Appendix D: Guidelines for Collaborative Study

Procedures To Validate Characteristics of a

Method of Analysis
{Note: These guidelines incorporate symbols, terminology, and characteristics of a method, although many sections are also appro-
recommendations accepted by consensus by the participants at the priate for other types of studies.
IUPAC Workshop on Harmonization of Collaborative Analytical
Alternatives for Method Selection
Studies, Geneva, Switzerland, May 4–5, 1987 [Pure Appl. Chem.
60, 855–864(1988); published as “Guidelines for Collaborative (1) Sometimes obvious (only method available).
Study of Procedure to Validate Characteristics of a Method of Anal- (2) Critical literature review (reported within-laboratory attrib-
ysis,” J. Assoc. Off. Anal. Chem. 72, 694–704(1989)]. The original utes are often optimistic).
guidelines were revised at Lisbon, Portugal, August 4, 1993, and at (3) Survey of laboratories to obtain candidate methods; compari-
Delft, The Netherlands, May 9, 1994, Pure Appl. Chem. 67, son of within-laboratory attributes of candidate methods (sometimes
331–343(1995). These revised, harmonized guidelines have been choice may still not be objective).
adopted by AOAC INTERNATIONAL as the guidelines for the (4) Selection by expert [AOAC-preferred procedure (selection
AOAC Official Methods Program, J. AOAC Int. 78(5), by Study Director with concurrence of General Referee)].
143A–160A(1995). Although the directions were developed for (5) Selection by Committee (ISO-preferred procedure; often
chemical studies, some parts may be applicable to all types of collab- time-consuming).
orative studies.} (6) Development of new method or modification of existing
method when an appropriate method is not available. (Proceed as a
Summary Statement of AOAC Recommendation research project.) (This alternative is time-consuming and re-
for Design of a Collaborative Study source-intensive; use only as a last resort.)

Minimum Criteria for Quantitative Study 1.2 Optimize Either New or Available Method

Minimum number of materials (see Note 1 on p. 4).—Five (only Practical Principles

when a single level specification is involved for a single matrix may
this minimum be reduced to 3). (1) Do not conduct collaborative study with an unoptimized
Minimum number of laboratories.—Eight reporting valid data for method. An unsuccessful study wastes a tremendous amount of col-
each material (only in special cases involving very expensive equip- laborators’ time and creates ill will. This applies especially to meth-
ment or specialized laboratories may the study be conducted with a ods that are formulated by committees and have not been tried in
minimum of 5 laboratories, with the resulting expansion in the confi- practice.
dence interval for the statistical estimates of the method characteristics). (2) Conduct as much experimentation within a single laboratory
as possible with respect to optimization, ruggedness, and interfer-
Minimum number of replicates.—One, if within-laboratory re-
ences. Analysis of the same material on different days provides con-
peatability parameters are not desired; 2, if these parameters are re-
siderable information on variability that may be expected in
quired. Replication should ordinarily be attained by blind replicates
practice.
or split levels (Youden pairs).
Minimum Criteria for Qualitative Analyses Alternative Approaches to Optimization

Ten laboratories reporting on 2 analyte levels per matrix, 6 test (1) Conduct trials by changing one variable at a time.
samples per level, and 6 negative controls per matrix. (Note: AOAC (2) Conduct formal ruggedness testing for identification and con-
criteria for qualitative analyses are not part of the harmonized guide- trol of critical variables. See Youden and Steiner (pp 33–36, 50–55).
lines.) The actual procedure is even simpler than it appears. (This is an ex-
tremely efficient way for optimizing a method.)
1. Preliminary Work (Within One Laboratory) (3) Use Deming simplex optimization to identify critical steps.
See Dols and Armbrecht. The simplex concept can be used in the op-
1.1 Determine Purpose and Scope of the Study and Method
timization of instrument performance and in application to analyti-
cal chemical method development.
Determine purpose of the study (e.g., to determine attributes of a
method, proficiency of analysts, reference values of a material, or to 1.3 Develop Within-Laboratory Attributes of Optimized Method
compare methods), the type of method (empirical, screening, practi-
cal, reference, definitive), and the probable use of the method (en- (Some items can be omitted; others can be combined depending
forcement, surveillance, monitoring, acceptance testing, quality on whether study is qualitative or quantitative.)
control, research). Also, on the basis of the relative importance of the Determine calibration function (response vs concentration in pure
various method attributes (bias, precision, specificity, limit of deter- or defined solvent) to determine useful measurement range of
mination), select the design of the collaborative study. The direc- method. For some techniques, e.g., immunoassay, linearity is not a
tions in this document pertain primarily to determining the precision prerequisite. Indicate any mathematical transformations needed.

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 2

Determine analytical function (response vs concentration in ma- tors (e.g., sum equals 100%) that suggest analysis is proceeding
trix, including blank) to determine applicability to commodity(ies) properly.
of interest. If time and resources are available, conduct pilot study involving
Test for interferences (specificity): (1) Test effects of impurities, 3 laboratories.
ubiquitous contaminants, flavors, additives, and other components
expected to be present and at usual concentrations. (2) Test nonspe- 1.5 Invite Participation
cific effects of matrices. (3) Test effects of transformation products, Selection of Collaborators/Candidate Laboratories
if method is to indicate stability, and metabolic products, if tissue
residues are involved. Laboratories invited to participate should have personnel experi-
Conduct bias (systematic error) testing by measuring recoveries enced in the basic techniques employed; experience with the method
of analyte added to matrices of interest and to extracts, digests, or itself is not a prerequisite for selection. Lists of possible participants
other treated solutions thereof. (Not necessary when method defines can be developed through personal contacts, technical societies,
property or component.) trade associations, or literature search, and advertisements in the
Develop performance specifications for instruments and suitabil- Referee section of AOAC’s magazine, Inside Laboratory Manage-
ity tests for systems (which utilize columns or adsorbents) to ensure ment. Collaborators are chosen by the organizers of the collaborative
satisfactory performance of critical steps (columns, instruments, study from a diversity of laboratories with interest in the method, in-
etc.) in method. cluding regulatory agencies, industry, and universities.
Conduct precision testing at the concentration levels of interest,
including variation in experimental conditions expected in routine Letter of Invitation
analysis (ruggedness). In addition to estimating the “classical” re-
peatability standard deviation, sr, the initiating laboratory may esti- Address a formal letter to the individual responsible for assign-
mate the total within-laboratory standard deviation (se) whereby se is ment of laboratory effort. State reason for selecting that laboratory
the variability at different days and with different calibration curves, (e.g., as a volunteer or has responsibility or familiarity with the prob-
by the same or different analysts within a single laboratory. This to- lem or method), estimated number of person-hours required for per-
tal within-laboratory estimate reflects both between-run (be- formance, number of test samples to be sent, number of analyses to
tween-batch) and within-run (within-batch) variability. be required, expected date for test sample distribution, and target
Delineate the range of applicability to the matrices or commodi- date for completion of the study. Emphasize the importance of man-
ties of interest. agement support in assigning the necessary time for the project. En-
Compare the results of the application of the method with exist- close a copy of the method and a return form or card (with postage
ing, studied methods intended for the same purposes, if other meth- affixed, if appropriate), requiring only a check mark for acceptance
ods are available. or refusal of the invitation, a signature, space for address corrections,
If any of the preliminary estimates of the relevant performance of telephone and fax numbers, e-mail, and date.
these characteristics are unacceptable, revise the method to improve Laboratory Coordinator
them, and re-study as necessary.
Have method tried by analysts not involved in its development. With large studies, involving several analysts per laboratory, sev-
Revise method to handle questions raised and problems encoun- eral familiarization samples, receipt of items at different times, or
tered. similar recurrent situations, acceptance of the invitation should be
followed by a letter suggesting that a Laboratory Coordinator be ap-
1.4 Prepare Description of Method
pointed. The Laboratory Coordinator should be responsible for re-
ceiving and storing the study materials, assigning the work,
Note: A collaborative study of a method involves practical testing
dispensing study materials and information related to the study, see-
of the written version of the method, in its specific style and format,
ing that the method is followed as written, accumulating the data, as-
by a number of laboratories on identical materials.
suring that the data are correctly reported, and submitting the
Prepare method description as closely as possible to format and
collaborative study manuscript within the deadline.
style that will be used for eventual publication.
Clearly specify requirements for chromatographic materials, en- 1.6 Instructions and Report Forms
zymes, antibodies, and other performance-related reagents.
Clearly describe and explain every step in the analytical method Carefully design and prepare instructions and forms, and scruti-
so as to discourage deviations. Use imperative directions; avoid sub- nize them before distribution. A pilot study is also useful for uncov-
junctive and conditional expressions as options as far as possible. ering problems in these documents.
Clearly describe any safety precautions needed. Send instructions and report forms immediately on receipt of ac-
Edit method for completeness, credibility (e.g., buffer pH consis- ceptance, independent of study materials, if selection of laboratories
tent with specified chemicals, volumes not greater than capacity of is not to be based on performance in pilot or training studies. The in-
container), continuity, and clarity. structions should include in bold face or capital letters a statement:
Check for inclusion of performance specifications and system THIS IS A STUDY OF THE METHOD, NOT OF THE LABO-
suitability tests, defined critical points, and convenient stopping RATORY. THE METHOD MUST BE FOLLOWED AS
points. Incorporate physical or chemical constants of working stan- CLOSELY AS PRACTICABLE, AND ANY DEVIATIONS
dards solutions, e.g., absorptivities, half-scale deflections, recover- FROM THE METHOD AS DESCRIBED, NO MATTER HOW
ies, etc., or properties of operating solutions and chromatographic TRIVIAL THEY MAY SEEM, MUST BE NOTED ON THE RE-
materials, e.g., pH, volumes, resolution, etc., and any other indica- PORT FORM.

© 2002 AOAC INTERNATIONAL

AOAC OFFICIAL METHODS OF ANALYSIS (2002) INTERLABORATORY COLLABORATIVE STUDY
Appendix D, p. 3

Include instructions on storage and handling, markings, and iden- Reproducibility is a composite measure of variation, sR2, which in-
tifications to be noted, any special preparation for analysis, and crite- cludes the between-laboratory and within-laboratory variations. It
ria for use of practice or familiarization samples, if included. measures how well an analyst in a given laboratory can check the re-
Pre-code the form for each laboratory and provide sufficient space sults of another analyst in another laboratory using the same method
for as much sequential data as may be required for proper evaluation to analyze the same test material under different conditions (e.g., dif-
of the results, including a check of the calculations. ferent apparatus and different time). The between-laboratory varia-
The initiating laboratory should indicate the number of significant tion represents a systematic error that reflects variation arising from
figures to be reported, usually based on the output of the measuring environmental conditions (e.g., condition of reagent and instru-
instrument. ments, variation in calibration factors, and interpretations of the
Note: In making statistical calculations from the reported data, the steps of the method) associated with the laboratories used in the
full power of the calculator or computer is to be used with no round- study. Therefore, it is important to identify the causes of the differ-
ing or truncating until the final reported mean and standard devia- ences among laboratories so that they may be controlled. Otherwise
tions are achieved. At this point the standard deviations are rounded they will be summed into sR2.
to 2 significant figures and the means and relative standard devia- Present test samples sent for analysis as unknowns (blind) and
tions are rounded to accommodate the significant figures of the stan- coded in a random pattern. If necessary to conserve analyst time, an
dard deviation. For example, if the reproducibility standard indication of the potential range of concentration or amount of
deviation sR = 0.012, the mean is reported as 0.147, not as 0.1473 or analyte may be provided. If spiking solutions are used, provide one
0.15, and RSDR, relative reproducibility standard deviation, is re- coded solution for each material. All spiking solutions should be
ported as 8.2%. If standard deviation calculations must be conducted identical in appearance and volume. Do not provide a single solution
manually in steps, with the transfer of intermediate results, the num- from which aliquots are to be removed for spiking. Any information
ber of significant figures to be retained for squared numbers should with regard to concentration (e.g., utilizing factorial aliquots or se-
be at least 2 times the number of figures in the data plus 1. rial dilutions of the same spiking solutions) or known replication is
When recorder tracing reproductions are required to evaluate likely to lead to an underestimate of the variability.
method performance, request their submission both in the instruc- The study must be extensive enough to assure sufficient data sur-
tions and as a check item on the form. Provide instructions with re- viving in the face of possible loss of materials during shipment, in-
gard to labeling of recorder tracings, such as identification with ability of collaborators to participate after acceptance, and a
respect to item analyzed, axes, date, submitter, experimental condi- maximum outlier rate of 2/9 and still maintain valid data from a min-
tions, and instrument settings. imum of 8 laboratories.
Include in the report form a signature line for the analyst and lines Improper preparation of reference standards and standard solu-
for a printed or typed version of the name and address for correct ac- tions can cause a significant portion of the analytical error. A deci-
knowledgement. sion must be made whether such error is to be considered separately
Provide for a review by the laboratory supervisor. An example of or as part of the method, i.e., will the analysts procure their own stan-
a completed form is helpful. A questionnaire may be included or sent dard solutions or will standards be provided by the Study Director.
after completion of the analyses in which the questions can be de- The decision depends primarily on the availability of the standard. If
signed to reveal if modifications have been made at critical steps in the standard is readily available, the analysts should prepare their
the method. own. If the standard is not readily available, the standard may be sup-
Request a copy of the calibration curve or other relationship be- plied, but physical constants, e.g., absorptivity of working standard
tween response and concentration or amount of analyte so that if dis- solutions, should be incorporated into the description as a check on
crepancies become apparent after examining all of the data, it can be proper preparation of the solution.
determined whether the problem is in the calibration or in the analysis. Obtain the necessary administrative and operational approvals.
Review by potential users of the method is also desirable.
1.7 Familiarization or Practice Samples
2.2 Laboratories
If deemed necessary, supply as far ahead as practicable, familiar-
ization samples, with instructions, before actual materials are sent. Laboratories must realize the importance of the study. A large in-
When familiarization samples have been submitted, supply forms vestment is being made in studying the method and this probably
for reporting progress toward satisfactory performance. will be only collaborative study of the method that will performed.
2. Design of the Collaborative Study Therefore, it is important to have a fair and thorough evaluation of
the method.
2.1 General Principles
Type
The purpose of a collaborative study is to determine estimates of
the attributes of a method, particularly the “precision” of the method The most appropriate laboratory is one with a responsibility re-
that may be expected when the method is used in actual practice. The lated to the analytical problem. Laboratory types may be representa-
AOACI uses 2 terms to define the precision of a method under 2 cir- tive (selection of laboratories that will be using the method in
cumstances of replication: repeatability and reproducibility. Repeat- practice), reference (assumed to be “best”), or the entire population
2
ability is a measure of the variation, sr , between replicate of laboratories (usually certified or accredited) that will be using the
determinations by the same analyst. It defines how well an analyst method. Final selection of participants should be based on a review
can check himself using the same method on blind replicates of the with the General Referee and others of each laboratory’s capabilities
same material or split levels (Youden pairs), under the same condi- and past performance in collaborative studies, followed up, if possi-
tions (e.g., same laboratory, same apparatus, and same time). ble, by telephone conversations or by personal visits. Selection may

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 4

also be based on performance with familiarization samples. Some- Number of Materials

times only laboratories with dedicated or very specialized instru-
ments must be used. If the study is intended for international A minimum of 5 materials must be used in the collaborative study.
consideration, laboratories from different countries should be in- Three materials are allowed but only when a single specification is
vited to participate. involved for a single matrix.
Note 1: A material is an analyte (or test component)/matrix/con-
Number of Laboratories centration combination to which the method-performance parame-
ters apply. This parameter determines the applicability of the
Minimum of 8 laboratories submitting valid data (to avoid unduly method.
large confidence bands about the estimated parameters). Only in Note 2: The 2 test samples of blind or open duplicates are a single
special cases of very expensive equipment or specialized laborato- material (they are not independent).
ries may the study be conducted with a minimum of 5 laboratories. The 2 test samples constituting a matched pair (called X and Y)
Fewer laboratories widen the confidence limits of the mean and of are considered Youden matched pairs only if they are sufficiently
close in composition. “Sufficiently close” would be considered as
the variance components (see design considerations). The optimum
≤5% difference in composition between X and Y. That is, given that
number of laboratories, balancing logistics and costs against infor- the concentration of analyte in X (xc) is higher than the concentration
mation obtained, often is 8–10. However, larger studies are not dis- of the analyte in Y (yc) then:
couraged.
For qualitative analyses, a minimum of 10 laboratories is needed; xc yc
≤ 0.05
collaborative study must be designed to include 2 analyte levels per xc
matrix, 6 test samples per level, and 6 negative controls per matrix.
(Note 1: AOAC criteria for qualitative analyses are not part of the or:
harmonized guidelines.)
yc ≥ (xc – 0.05xc)
Analysts
Note 3: The blank or negative control may or may not be a mate-
Most designs require only 1 analyst per laboratory. If rial, depending on the usual purpose of the analysis. For example, in
analyst–within-laboratory variability is a desired variance compo- trace analysis, where very low levels (near the limit of quantitation)
nent, multiple analysts should be requested from all participating are often sought, the blanks are considered as materials, and are nec-
laboratories. Ordinarily 2 analysts from the same laboratory cannot essary to determine certain statistical “limits of measurement;” how-
be substituted for different laboratories, unless standard solutions, ever, if the blank is merely a procedural control, in macro-level
reagents, chromatographic columns and/or materials, instrument analysis (e.g., fat in cheese), it would not be considered a material.
calibrations, standard curves, etc., are prepared independently, and
no consultation is permitted during the work. Different laboratories Nature of Materials
from the same organization may be used as separate laboratories if
they operate independently with their own instruments, standards, Materials should be representative of commodities usually ana-
reagents, and supervision. lyzed, with customary and extreme values for the analyte.

2.3 Test Materials Size of Test Samples

Homogeneous Materials Furnish only enough test sample to provide the number of test por-
tions specified in the instructions. If additional test portions are re-
Materials must be homogeneous; this is critical. Establish homoge- quired, the collaborator must request them, with an explanation.
neity by testing a representative number of laboratory samples taken
at random before shipment. (A collaborator who reports an outlying Interferences
value will frequently claim receipt of a defective laboratory sample.)
The penalty for inhomogeneity is an increased variance in the analyti- If pertinent, some materials, but not all, should contain contami-
cal results that is not due to the intrinsic method variability. nants and interferences in concentrations likely to be encountered,
unless they have been shown to be unimportant through
Test Sample Coding within-laboratory study. The success of the method in handling in-
terference on an intralaboratory basis will be demonstrated by pass-
Code test samples at random so that there is no pre-selection from ing systems suitability tests.
order of presentation.
Familiarization Samples
Concentration Range
With new, complex, or unfamiliar techniques, provide material(s)
Choose analyte levels to cover concentration range of interest. If of stated composition for practice, on different days, if possible. The
concentration range of interest is a tolerance limit or a specification valuable collaborative materials should not be used until the analyst
level, bracket it and include it with materials of appropriate concen- can reproduce the stated value of the familiarization samples within
tration. If design includes the determination of absence of analyte, a given range. However, it should be pointed out that one of the as-
include blank (not detectable) materials as part of range of interest. sumptions of analysis of variance is that the underlying distribution

© 2002 AOAC INTERNATIONAL

AOAC OFFICIAL METHODS OF ANALYSIS (2002) INTERLABORATORY COLLABORATIVE STUDY
Appendix D, p. 5

of results is independent of time (i.e., there is no drift). The Study example, the standard deviation, s r, which measures within
Director must be satisfied that this assumption is met. laboratory or repeatability precision has associated with it a standard
deviation (STD = sr) describing its scatter about the true value σr.
2.4 Replication 2
Therefore, sr, whose STD (sr) is a function of sr , number of laborato-
When within-laboratory variability is also of interest, as is usually ries, and number of analyses per laboratory, will vary about σr from
the case, independent replication can be ensured by applying at least occasion-to-occasion even for the same test conditions and material.
one of the following procedures (listed in suggested order of desir- The STD sR, which measures among laboratory or reproducibility
2
ability; the nature of the design should not be announced before- precision, has a STD (sR) that is a function of the random variables sr
2
hand): and sL , number of laboratories, and number of analyses per labora-
(1) Split levels (Youden pairs).—The 2 test materials, nearly tory. sR will vary about its true value σR from occasion-to-occasion
for the same test material.
identical but of slightly different composition (e.g., ≤5% difference
in composition, see 2.3 Number of Materials, Note 2) are obtained The validity of extrapolating the use of a method beyond concen-
either naturally or by diluting (or by fortifying) one portion of the trations and components tested can be estimated only on the basis of
material with a small amount of diluent (or of analyte). Both portions the slope of the calibration curve (sensitivity) observed as a function
are supplied to the participating laboratories as test samples, each of the nature and concentration of the matrix and contaminant com-
under a random code number, and each test sample should be ana- ponents. If the signal is more or less independent of these variables, a
lyzed only once; replication defeats the purpose of the design. reasonable amount of extrapolation may be utilized. The
(2) Split levels for some materials and blind duplicates for other extrapolator assumes the burden of proof as to what is reasonable.
materials in the same study.—Obtain only single values from each 3. Preparation of Materials for Collaborative Studies
test sample supplied.
3.1 General Principles
(3) Blind duplicate test samples, randomly coded.—Note: Tripli-
cate and higher replication are relatively inefficient when compared
Heterogeneity between test samples from a single test material
with duplicate test samples because replication provides additional
must be negligible compared to analytical variability, as measured
information only on individual within-laboratory variability, which
within the Study Director’s laboratory.
is usually the less important component of error. It is more effective
to utilize resources for the analysis of more levels and/or materials The containers must not contribute extraneous analytes to the con-
rather than for increasing the number of replicates for the individual tents, and they must not adsorb or absorb analytes or other compo-
materials. nents from the matrix, e.g., water.
PRACTICAL PRINCIPLE: With respect to replication, the great- If necessary, the materials may be stabilized, preferably by physi-
est net marginal gain is always obtained in going from 2 to 3 as com- cal means (freezing, dehydrating), or by chemical means (preserva-
pared to going from 3 to 4, 4 to 5, etc. tives, antioxidants) which do not affect the performance of the
(4) Independent materials.—(Note: Unrelated independent ma- method.
terials may be used as a split level in the calculations of the precision Composition changes must be avoided, where necessary, by the
parameters or for plotting. There should be ≤5% difference in com- use of vapor-tight containers, refrigeration, flushing with an inert
position for such materials (see 2.3 Number of Materials, Note 2). gas, or other protective packaging.
The more they differ in concentration, the less reliable the informa- 3.2 Materials Suitable for Collaborative Studies
tion they provide on within-laboratory variability.)
(5) Known replicates.—Use of known replicates is a common Material and analyte stability: Ensure analyte and matrix stability
practice.—It is much preferable to use the same resources on blind over projected transport and projected length of study.
replicates or split levels. Single batch of homogenous, stable product such as milk powder,
(6) Quality control materials.—Instead of obtaining repeatabil- peanut butter, vegetable oil, starch, etc., is the best type of material.
ity parameters through the collaborative study, information can be
Reference materials supplied by standards organizations such as
obtained from use of quality control materials in each laboratory in-
National Institute of Standards and Technology (NIST,
dividually, for its own use, independent of the collaborative study,
Gaithersburg, MD) and EC’s Joint Research Center and Institute on
for a separate calculation of sr, using 2 (or more) replicates from each
Reference Materials and Methods (IRMM, Belgium) are excellent,
quality control test, according to the pattern developed for each
unless they have easily recognizable characteristics (e.g., odor and
product.
color of NIST Orchard Leaves). However, they are of limited avail-
2.5 Other Design Considerations ability, composition, and analyte level. If available, they are expen-
sive. Sometimes the certification organization may be interested in
The design can be reduced in the direction of less work and less making reference materials available for the analyte under study, in
cost, but at the sacrifice of reduced confidence in the reliability of the which case it may assist in providing the material for the study.
developed information. Synthetic materials may be especially formulated with known
More work (values) is required if more confidence is needed, e.g., amounts of analytes by actual preparation for the study. This proce-
greater confidence is required to enforce a tolerance at 1.00 mg/kg dure is best used for macro-constituents such as drugs or pesticide
than at 1.0 mg/kg. (The distinction is a precision requirement of the formulations.
order of 1% rather than 10%.) Spiked materials consisting of normal or blank materials to which
The estimate of the standard deviation or the corresponding rela- a known amount of analyte has been added may be used. The amount
tive standard deviation obtained from a collaborative study is a ran- of analyte added should not be excessive in relation to the amount
dom variable that varies about its corresponding true value. For present (e.g., about 2×), and the analyte added should be in the same

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 6

chemical form as present in the commodities to be analyzed subse- 3.4 Limit of Detection/Quantitation
quently.
If the limit of detection/quantitation is important, it is necessary to
In drug and pesticide residue-type problems, it is often necessary provide a design which gives special attention to the number of
to use spiked materials in order to assess recovery. However, be- blanks, and to the necessity for interpreting false positives and false
cause incurred residues are likely to present different problems from negatives. In all cases, the definition of limit of detec-
those of spiked residues, collaborative studies should include some tion/quantitation used in the study must be given by the Study
test samples with incurred residues to ensure that the method is ap- Director.
plicable under these conditions as well.
(1) Preparation in bulk.—This requires thorough and uniform 3.5 Controls
incorporation of analyte, often by serial dilution of solids. The dan-
ger of segregation due to differences in densities always exists. Fluid When separation from interferences is critical to the analysis, ap-
materials susceptible to segregation should be prepared under con- propriate materials incorporating these interferences must be in-
stant agitation. Uniformity should be checked by direct analysis, cluded.
with an internal standard, or by a marker compound (dye or radioac- PRACTICAL ADVICE: Always allow for contingencies and pre-
tive label). pare more sets (e.g., 25% more) of laboratory samples than there are
collaborators. Some packages may never arrive, some materials may
(2) Test samples, individually prepared.—A known amount of spoil, and some may be lost or the container broken. New laboratories
analyte is either weighed directly or added as an aliquot of a prepared may have to be substituted for those which are unable to complete the
solution to pre-measured portions of the matrix in individual con- promised work. Some sets may have to be analyzed at a later time for
tainers. The collaborator is instructed to use each entire portion for different purposes, such as to verify stability on storage.
the analysis, transferring the contents of the container quantitatively
4. Submission of Test Samples
or a substantial weighed fraction of the portion. (This is the preferred
alternative to spiked solid materials at trace [mg/kg] levels, at the ex- 4.1 Sending Collaborative Study Material
pense of considerably more work.)
(3) Concentrated unknown solutions for direct addition by col- Notify collaborators of shipping arrangements, including waybill
laborators to their own commodities.—Should be used only as a last numbers, arrival time, and required storage conditions.
resort when instability of the analyte precludes distribution from a Label test samples legibly and without ambiguity.
central point. To preclude direct analysis of the spiking solution, Pack shipping cartons well and label properly to avoid transpor-
supply individual coded solutions to be added in their entirety to por- tation delays. If the containers are breakable, pack well to minimize
tions of the matrix for single analyses by each laboratory. All solu- possibility of breakage. If material is perishable, ship frozen with
tions should have the same volume and appearance. This type of solid CO2, sufficient to last several days longer than anticipated
material is analogous to that of test samples except for the source of travel time. Use special transportation services, if necessary. For in-
matrix. This case should be used only for perishable commodities ternational delivery, mark as “Laboratory samples—no commercial
that are altered by all available preservation techniques. value” or other designation as required by customs regulations of the
country to which the package is being sent. Hazardous materials
Materials analyzed by another, presumably accurate, method, if
must be packed and labeled as required by transportation regula-
available, in the Study Director’s laboratory or by some or all the
tions. Animal and plant products sent across international borders
collaborators.
may require special certification from health authorities.
Only as an absolutely last resort (usually with unstable materials Include a return slip, to confirm safe receipt, with each package.
and preparation of material studies) should the collaborators be If not sent previously, include copy of method, instructions, and re-
permitted to prepare their own materials for analysis. Since it is im- port forms.
possible to avoid the personal bias introduced by knowledge of the Provide instructions for proper storage of test samples between
composition of the material, the materials should be prepared in each unpacking and analysis. Note that analysts should not use thawed or
laboratory by an individual who will not be involved in the analyses. decomposed test samples without consulting the Study Director.
When it is important to have instruments calibrated with the same
3.3 Blanks
reference material, supply reference material to collaborators. Pro-
vision for supplying reference standards is particularly important
When the absence of a component is as important as its presence,
when commercial sources of standards have not yet been developed.
when determinations must be corrected for the amount of the com-
The inclusion of a working standard solution as an unknown is use-
ponent or the presence of background in the matrix, or when recov-
ful to establish a consensus value for standardization of quality con-
ery data are required, provision must be made for the inclusion of
trol parameters, such as absorptivity, retention time, and sensitivity
blank materials containing “none” (not detected) of the analyte. It is
(change in signal intensity divided by the change in concentration).
also important to know the variability of the blank and the tendency
of the method to produce false positives. There are 2 types of blanks: 4.2 Obligations of Collaborators
matrix blanks and reagent blanks. Since laboratories often will uti-
lize reagents from different sources, each laboratory should perform Analyze test samples at times indicated, according to submitted
reagent blanks. Matrix blanks, when required, are an intrinsic part of protocol. With unstable materials (e.g., with microbial or decompo-
the method, and the number of blanks needed depends on the com- sition problems), analyses must be started at specified times.
bined variance of the material (sM) and of the blank (sB). Standard de- FOLLOW METHOD EXACTLY (this is critical). If method is
viation reflecting the total variability of a blank corrected value will unclear, contact Study Director. Any deviation, such as the necessity
be s = (sM + sB )1/2.
2 2
to substitute reagents, columns, apparatus, or instruments, must be

© 2002 AOAC INTERNATIONAL

AOAC OFFICIAL METHODS OF ANALYSIS (2002) INTERLABORATORY COLLABORATIVE STUDY
Appendix D, p. 7

recorded at the time and reported. If the collaborator has no intention unexpected reactions occur; or (7) other atypical phenomena materi-
of following the submitted method, he or she should not participate alize. Other potential causes of invalid data are noted previously.
in the study. If the collaborator wishes to check another method on
the same materials, additional test samples should be requested for 5.2 Outliers
that purpose, to be analyzed separately.
Conduct exactly the number of determinations stated in the in- Collaborative studies seem to have an inherent level of outliers,
structions. Any other number complicates the statistical analysis. the number depending on the definition of outliers and the basis for
Too few determinations may require discarding the results from that calculation (analytes, materials, laboratories, or determinations).
laboratory for that material or inserting “missing values”; too many Rejection of more than 2/9 of the data from each material in a study,
values may require discarding the contribution of that laboratory or without an explanation (e.g., failure to follow the method), is ordi-
at least some of the values. If a laboratory cannot follow instructions narily considered excessive. Study must maintain valid data from a
as to number of analyses to perform, it raises a question as to its abil- minimum of 8 labs. For larger studies, a smaller acceptable percent-
ity to follow the method. age of rejections may be more appropriate. Determine the probabil-
ity that the apparent aberrant value(s) is part of the main group of
Report individual values, including blanks. Do not average or do values considered as a normal population by applying the following
other data manipulations unless required by the instructions. Undis- tests in order:
closed averaging distorts statistical measures. If blank is larger than
(1) Cochran test for removal of laboratories (or indirectly for re-
determination, report the negative value; do not equate negative val-
moval of extreme individual values from a set of laboratory values)
ues to zero. Follow or request instructions with regard to reporting
showing significantly greater variability among replicate
“traces” or “less than.” Descriptive (i.e., nonquantitative) terms are
(within-laboratory) analyses than the other laboratories for a given
not amenable to statistical analysis and should be avoided. When re-
material. Apply as a 1-tail test at a probability value of 2.5%.
sults are below the limit of determination, report actual calculated
To calculate the Cochran test statistic: Compute the
result, regardless of its value.
within-laboratory variance for each laboratory and divide the largest
Supply raw data, graphs, recorder tracings, photographs, or of these by the sum of all of these variances. The resulting quotient is
other documentation as requested in the instructions. the Cochran statistic which indicates the presence of a removable
Since collaborators may have no basis for judging whether a value outlier if this quotient exceeds the critical value listed in the Cochran
is an outlier, the results should be communicated to the Study table for P = 2.5% (1-tail) and L (number of laboratories), Appen-
Director as soon as the protocol is complete and before time and dix 1.
equipment are reassigned, so that repeat assays may be performed at (2) Grubbs tests for removal of laboratories with extreme aver-
once, if necessary and if permitted by the protocol. ages. Apply in the following order: single value test (2-tail; P =
Note: The sooner an apparent outlier is investigated, the greater 2.5%); then if no outlier is found, apply pair value test (2 values at the
the likelihood of finding a reason for its occurrence. highest end, 2 values at the lowest end, and 2 values, one at each end,
at an overall P = 2.5%).
The most frequent causes of correctable outliers are:
To calculate the single Grubbs test statistic: Compute the average
• Incorrect calculations and arithmetic errors. for each laboratory and then calculate the standard deviation (SD) of
• Errors in reporting, such as transposition of numbers, these L averages (designate as the original s). Calculate the SD of the
misplacement of the decimal point, or use of the wrong set of averages with the highest average removed (sH); calculate the
units. SD of the set averages with the lowest average removed (sL). Then
• Incorrect standards due to weighing or volumetric errors calculate the percentage decrease in SD as follows:
(check physical constants or compare against freshly
prepared standard solutions). 100 × [1 – (sL/s)] and 100 × [1 – (sH/s)]
• Contamination of reagents, equipment, or test samples.
5. Statistical Analysis The higher of these 2 percentage decreases is the single Grubbs
statistic, which signals the presence of an outlier to be omitted if it
5.1 Initial Review of Data (Data Audit) exceeds the critical value listed in the single Grubbs tables at the P =
2.5% level, 2-tail, for L laboratories, Appendix 2.
The Study Director may first plot the collaborative study results, To calculate the Grubbs pair statistic, proceed in an analogous
material by material (or one value against the other for a split level fashion, except calculate the standard deviations s2L, s2H, and sHL, fol-
[Youden pair]), value vs laboratory, preferably in ascending or de- lowing removal of the 2 lowest, the 2 highest, and the highest and the
scending order of reported average concentration. Usually major lowest averages, respectively, from the original set of averages.
discrepancies will be apparent: displaced means, unduly spread rep- Take the smallest of these 3 SD values and calculate the correspond-
licates, outlying values, differences between methods, consistently ing percentage decrease in SD from the original s. A Grubbs outlier
high or low laboratory rankings, etc. pair is present if the selected value for the percentage decrease from
Only valid data should be included in the statistical analysis. Valid the original s exceeds the critical value listed in the Grubbs pair value
table at the P = 2.5% level, for L laboratories, Appendix 2.
data are values that the Study Director has no reason to suspect as be-
(3) If the single value Grubbs test signals the need for outlier re-
ing wrong. Invalid data may result when: (1) the method is not fol-
moval, remove the single Grubbs outlier and recycle back to the
lowed; (2) a nonlinear calibration curve is found although a linear Cochran test as shown in the flow chart, Appendix 3.
curve is expected; (3) system suitability specifications were not met; If the single value Grubbs test is negative, check for masking by
(4) resolution is inadequate; (5) distorted absorption curves arise; (6) performing the pair value Grubbs test. If this second test is positive,

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 8

remove the 2 values responsible for activating the test and recycle Notes: (1) Youden equates “true” or “pure” between-laboratory
back to the Cochran test as shown in the flow chart, Appendix 3, and variability (not including the within-laboratory variability) to the
repeat the sequence of Cochran, single value Grubbs, and pair value variability in bias (or variability in systematic error) of the individual
Grubbs. Note, however, that outlier removal should stop before laboratories. Technically, this definition refers to the average
more than 2/9 laboratories are removed. squared difference between individual laboratory biases and the
(4) If no outliers are removed for a given cycle (Cochran, single mean bias of the assay.
Grubbs, pair Grubbs), outlier removal is complete. Also, stop outlier (2) The presence of random error limits the ability to estimate the
removal whenever more than 2/9 of the laboratories are flagged for systematic error. To detect the systematic error of a single laboratory
removal. With a higher removal rate, either the precision parameters when the magnitude of such error is comparable to that laboratory’s
must be taken without removal of all outliers or the method must be random error, at least 15 values are needed, under reasonable confi-
considered as suspect. dence limit assumptions.
Note: The decision as to whether a value(s) should be removed as 5.4 Precision
an outlier ultimately is not statistical in nature. The decision must be
made by the Study Director on the basis of the indicated probability The precision of analytical methods is usually characterized for
given by the outlier test and any other information that is pertinent. 2 circumstances of replication: within laboratory or repeatability and
(However, for consistency with other organizations adhering to the among laboratories or reproducibility. Repeatability is a measure of
how well an analyst in a given laboratory can check himself using the
harmonized outlier removal procedure, the estimate resulting from
same analytical method to analyze the same test sample at the same
rigid adherence to the prescribed procedure should be reported.) time. Reproducibility is a measure of how well an analyst in one labo-
5.3 Bias (Systematic Deviation) of Individual Results
ratory can check the results of another analyst in another laboratory
using the same analytical method to analyze the same test sample at
Bias is defined as follows: the same or different time. Given that test samples meet the criteria for
a single material, the repeatability standard deviation (sr) is:
(Estimated) bias =
mean amount found – amount added (or known or assigned value) sr = (Σdi2/2L)1/2

Single-value error and recovery are defined as follows: where di is the difference between the individual values for the pair
in laboratory i and L is the number of laboratories or number of pairs.
The reproducibility standard deviation (sR) is computed as:
Error of a single value =
the single value – amount added (true value)
sR = (1/2(sd2 + sr2))1/2
There are 2 methods for defining percent recovery: marginal and
where sd = Σ(Ti – T)2/(2(L – 1)), Ti is the sum of the individual values
2
total. The formulas used to estimate these percent recoveries are pro-
vided in the following: for the pair in laboratory i, T is the mean of the Ti across all laborato-
2
ries or pairs, L is the number of laboratories or pairs, and sr is the
2 1/2
Marginal %Rec = 100RM = 100((Cf – Cu)/CA) square of sr = (Σdi /2L) .
When the pairs of test samples meet the criteria for Youden
Total %Rec = 100RT = 100(Cf)/(Cu + CA) matched pairs, i.e., when:

where Cf is the amount found for the fortified concentration, Cu is the [(xc – yc )/xc ] ≤ 0.05
amount present originally for the unfortified concentration, and CA
is the amount added for the added concentration. The amount added or
is known or fixed and should be a substantial fraction of, or more
than, the amount present in the unfortified material; all other quanti- yc ≥ (xc – 0.05xc),
ties are measured and are usually reported as means, all of which
have variations or uncertainties. The variation associated with the sr, a practical approximation for repeatability standard deviation, is
marginal percent recovery is var(100RM) = (1002/CA2)[var(Cf) + calculated as:
var(Cu)] is larger than the variation associated with the total percent
recovery. The variation associated with total percent recovery is sr = [Σ(di – d)2/(2(L – 1))]1/2
var(100RT) = [1002/(Cu + CA)2][var(Cf) + (RT )var(Cu)]. In each for-
2

mula var means variance and refers to the concentration variation for where di is the difference between the individual values for the pair
the defined concentrations. in laboratory i, d is the mean of the di across all laboratories or pairs,
A true or assigned value is known only in cases of spiked or forti- and L is the number of laboratories or pairs. The reproducibility
fied materials, certified reference materials, or by analysis by an- standard deviation, sR, which reflects the square root of the average
other (presumably unbiased) method. Concentration in the of the reproducibility variances for the individual materials (i.e., sR =
[½(sRx + sRy )]1/2), previously called X and Y, should be determined
2 2
unfortified material is obtained by direct analysis by the method of
additions. In other cases, there is no direct measure of bias, and con- only if the individual variances are not significantly different from
2 2
sensus values derived from the collaborative study itself often must each other. To compare sRx and sRy , the following formula may be
be used for the reference point. used.

© 2002 AOAC INTERNATIONAL

AOAC OFFICIAL METHODS OF ANALYSIS (2002) INTERLABORATORY COLLABORATIVE STUDY
Appendix D, p. 9

1
(s2Rx s2Ry )(L 2) 2 analyte or property?), and discuss this in the collaborative
t= 2 2 2 1 study report.
2[(s )(s ) (covxy ) ]
Rx Ry
2

• HORRAT > 2.0—Method reproducibility is problematic.

2 2 2 2 2 A high HORRAT may result in rejection of a method
where sRx = [1/(L – 1)][Σxi – (Σxi) /L], sRy = [1/(L – 1)][Σyi – because it may indicate unacceptable weaknesses in the
(Σyi)2/L], and covxy = [1/(L – 1)][Σxiyi – (ΣxiΣyi)/L]. If t is greater than method or the study. Some organizations may use
or equal to the tabular t-value for L – 2 degrees of freedom for a sig- information about the HORRAT as a criterion not to
nificance level of α = 0.05, this may be taken to indicate that sRx and
2
accept the method for official purposes (e.g., this is
2
sRy are not equivalent and should not be pooled for a single estimate currently the case in the EU for aflatoxin methods for
2 2 2
of sR . That is, sRx and sRy should be taken as the reproducibility vari- food analysis, where only methods officially allowed are
ance estimates for the individual test materials X and Y, respec- those with HORRATs ≤ 2).
2
tively. This means that there is no rigorous basis for calculating sr
because the within laboratory variability cannot be estimated di- 5.6 Incorrect, Improper, or Illusory Values (False Positive and
False Negative Values)
rectly.
Though sr and sR are the most important types of precision, it is the
relative standard deviations (RSDr % = 100sr/mean and RSDR % = These results are not necessarily outliers (no a priori basis for de-
100sR/mean) that are the most useful measures of precision in chemi- cision), since there is a basis for determining their incorrectness (a
cal analytical work because the RSD values are usually independent positive value on a blank material, or a zero (not found) or negative
of concentration. Therefore, the use of the RSD values facilitates value on a spiked material). There is a statistical basis for the pres-
comparison of variabilities at different concentrations. When the ence of false negative values: In a series of materials with decreasing
RSD increases rapidly with decreasing concentration or amount, the analyte concentration, as the RSD increases, the percent false nega-
rise delineates the limit of usefulness of the method (limit of reliable tives increases from an expected 2% at an RSD = 50% to 17% at an
measurement). RSD = 100%, merely from normal distribution statistics alone.

5.5 HORRAT When false positives and/or false negatives exceed about 10% of
all values, analyses become uninterpretable from lack of confidence
HORRAT value is the ratio of the reproducibility relative stan- in the presence or absence of the analyte, unless all positive labora-
dard deviation, expressed as a percent (RSDR, %) to the predicted tory samples are re-analyzed by a more reliable (confirmatory)
reproducibility relative standard deviation, expressed as a percent method with a lower limit of determination than the method under
(PRSDR, %), i.e., study. When the proportion of zeros (not necessarily false negatives)
becomes greater than approximately 30%, the distribution can be-
RSDR,% come bimodal and even more uninterpretable (is the analyte present
HORRAT = or absent?).
PRSDR,%
5.7 Final Collaborative Study Manuscript
where PRSDR, % = 2C–0.1505 and C = the estimated mean concentra-
tion. HORRAT values between 0.5 to 1.5 may be taken to indicate
that the performance value for the method corresponds to historical The final manuscript should contain a description of the materials
performance. The limits for performance acceptability are 0.5–2. used, their preparation, any unusual features in their distribution,
The precision of a method must be presented in the collaborative and a table of all valid data, including outliers. When replication is
study manuscript. The HORRAT will be used as a guide to deter- performed, the individual values, not just averages, must be given,
mine the acceptability of the precision of a method. unless the method requires averages (e.g., microbiological meth-
ods). Values not used for specified reasons, such as decomposition,
The HORRAT is applicable to most chemical methods.
failure to follow method, or contamination, should not be included in
HORRAT is not applicable to physical properties (viscosity, RI,
the table since they may be included erroneously in subsequent re-
density, pH, absorbance, etc.) and empirical methods [e.g., fiber, en-
calculations. AOAC INTERNATIONAL requires the calculation
zymes, moisture, methods with indefinite analytes (e.g., polymers)
and reporting of mean, percent recovery (% Rec), HORRAT, repeat-
and “quality” measurements, e.g., drained weight]. Deviations may
ability (within-laboratory, sr) and reproducibility (interlaboratory,
also occur at both extremes of the concentration scale (near 100%
sR) standard deviations, and repeatability and reproducibility rela-
and .10 ). In areas where there is a question if the HORRAT is ap-
–8

tive standard deviations (RSDr and RSDR, respectively). The accu-

plicable, the General Referee will be the determining judge.
racy (bias, trueness) of a method measuring a specific, identifiable
The following guidelines should be used to evaluate the assay pre- analyte should be presented in the collaborative study manuscript as
cision: a recovery of added (spiked) analyte, as the results of analysis of a
• HORRAT ≤ 0.5—Method reproducibility may be in reference material, or by comparison with results by a reference
question due to lack of study independence, unreported method. Methods that are unable to report accuracy because of the
averaging, or consultations. unavailability of an accepted “true” value, or because of the nature
• 0.5 < HORRAT ≤ 1.5—Method reproducibility as of the method (empirical, microbiological, quality factors) should
normally would be expected. mention the reason in the manuscript. Proofread tables very care-
• HORRAT > 1.5—Method reproducibility higher than fully since many errors are of typographical origin. Give the names
normally expected: the Study Director should critically of the participants and their organizations, including complete con-
look into possible reasons for a “high” HORRAT (e.g., tact information (name, preliminary address, telephone and fax
were test samples sufficiently homogeneous, indefinite numbers, and e-mail address).

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 10

The final manuscript should be published in a generally accessible

publication, or availability of the report from the organization spon-
soring the method should be indicated in the published method.
Without public documentation, the significance of the study is very
limited.
The manuscript should be sent to all participants, preferably at the
preliminary stage, so that clerical and typographical errors may be
corrected before publication. If changes in values from the original
submission are offered, they must be accompanied by an explanation.
Example of Table of Interlaboratory Study Results: See Table 1.
The summary table as it will appear in the Official Methods of
Analysis of AOAC INTERNATIONAL is given in Table 2.
6. References
(1) W.J. Youden & E.H. Steiner (1975) Statistical Manual of the AOAC,
AOAC INTERNATIONAL, 481 N. Frederick Ave, Suite 500,
Gaithersburg, MD 20877-7077, USA. The fifth printing (1987) con-
tains several explanatory footnotes.
(2) G.T. Wernimont (1985) Use of Statistics to Develop and Evaluate
Analytical Methods, W. Spendley (Ed.) AOAC INTERNATIONAL,
481 N. Frederick Ave, Suite 500, Gaithersburg, MD 20877-7077,
USA.
(3) T. Dols & B. Armbrecht (1976) J. Assoc. Off. Anal. Chem. 59,
1204–1207.
(4) International Organization for Standardization Guide 18, ISO, Case
Postale 56, CH-1211 Geneva, Switzerland, and other national stan-
dards organizations.
(5) International Organization for Standardization ISO 5725, ISO, Case
Postale 56, CH-1211 Geneva, Switzerland, and other national stan-
dards organizations.

© 2002 AOAC INTERNATIONAL

AOAC OFFICIAL METHODS OF ANALYSIS (2002) INTERLABORATORY COLLABORATIVE STUDY
Appendix D, p. 11

Appendix 1 Critical values for the Cochran maximum vari- Appendix 2 Critical values for the Grubbs extreme devia-
ance ratio at the 2.5% (1-tail) rejection level, tion outlier tests at the 2.5% (2-tail), 1.25%
expressed as the percentage the highest vari- (1-tail) rejection level, expressed as the per-
ance is of the total variance cent reduction in the standard deviations
caused by removal of the suspect value(s)
L = number of laboratories at a given level (concentration)
(see text for calculating the Grubbs statistics)
r = number of replicates per laboratory
L r=2 r=3 r=4 r=5 r=6 L = number of laboratories at a given level (concentration)
One highest or Two highest or One highest
4 94.3 81.0 72.5 65.4 62.5 L lowest two lowest and one lowest
5 88.6 72.6 64.6 58.1 53.9
6 83.2 65.8 58.3 52.2 47.3 4 86.1 98.9 99.1
7 78.2 60.2 52.2 47.3 42.3 5 73.5 90.3 92.7
8 73.6 55.6 47.4 43.0 38.5 6 64.0 81.3 84.0
9 69.3 51.8 43.3 39.3 35.3 7 57.0 73.1 76.2
8 51.4 66.5 69.6
10 65.5 48.6 39.9 36.2 32.6 9 46.8 61.0 64.1
11 62.2 45.8 37.2 33.6 30.3
12 59.2 43.1 35.0 31.3 28.3 10 42.8 56.4 59.5
13 56.4 40.5 33.2 29.2 26.5 11 39.3 52.5 55.5
14 53.8 38.3 31.5 27.3 25.0 12 36.1 48.5 51.6
13 33.8 46.1 49.1
15 51.5 36.4 29.9 25.7 23.7 14 31.7 43.5 46.5
16 49.5 34.7 28.4 24.4 22.0
17 47.8 33.2 27.1 23.3 21.2 15 29.9 41.2 44.1
18 46.0 31.8 25.9 22.4 20.4 16 28.3 39.2 42.0
19 44.3 30.5 24.8 21.5 19.5 17 26.9 37.4 40.1
18 25.7 35.9 38.4
20 42.8 29.3 23.8 20.7 18.7 19 24.6 34.5 36.9
21 41.5 28.2 22.9 19.9 18.0
22 40.3 27.2 22.0 19.2 17.3 20 23.6 33.2 35.4
23 39.1 26.3 21.2 18.5 16.6 21 22.7 31.9 34.0
24 37.9 25.5 20.5 17.8 16.0 22 21.9 30.7 32.8
23 21.2 29.7 31.8
25 36.7 24.8 19.9 17.2 15.5 24 20.5 28.8 30.8
26 35.5 24.1 19.3 16.6 15.0
27 34.5 23.4 18.7 16.1 14.5 25 19.8 28.0 29.8
28 33.7 22.7 18.1 15.7 14.1 26 19.1 27.1 28.9
29 33.1 22.1 17.5 15.3 13.7 27 18.4 26.2 28.1
28 17.8 25.4 27.3
30 32.5 21.6 16.9 14.9 13.3 29 17.4 24.7 26.6
35 29.3 19.5 15.3 12.9 11.6
40 26.0 17.0 13.5 11.6 10.2 30 17.1 24.1 26.0
50 21.6 14.3 11.4 9.7 8.6 40 13.3 19.1 20.5
Cochran statistic = (largest individual within-laboratory variance)/(sum of all the 50 11.1 16.2 17.3
within-laboratory variances). Source: Both tables were calculated by R. Albert (October 1993) by computer
simulation involving several runs of approximately 7000 cycles each for each
value, and then smoothed. Although the table of Appendix 1 is strictly applica-
ble only to a balanced design (same number of replicates from all laboratories), it
can be applied to an unbalanced design without too much error, if there are only
a few deviations.

© 2002 AOAC INTERNATIONAL

INTERLABORATORY COLLABORATIVE STUDY AOAC OFFICIAL METHODS OF ANALYSIS (2002)
Appendix D, p. 12

Table 1 [x] Collaborative tests carried out at the inter-

national level in [year(s)] by [organization(s)] in
which [y and z] laboratories participated, each
performing [k] replicates, gave the following
statistical results [results expressed in (units)]:
Material [description and listed across the top in increasing order
of magnitude of means]
Number of laboratories retained after eliminating outliers
Number of outlying laboratories removed

Mean ( )
True or accepted value, if known

Repeatability standard deviation (sr)

Repeatability relative standard deviation (RSDr)
Repeatability value, r (2.8 × sr)
Total within laboratory standard deviation (se)—optional if sr is not
valid.

Reproducibility standard deviation (sR)

Reproducibility relative standard deviation (RSDR)
HORRAT
Reproducibility value, R (2.8 × sR)
Percent recovery (% Rec), if applicable

The repeatability and reproducibility values may also be expressed

as a relative value (as a percentage of the determined mean
value), when the results so suggest.
If the recovery and precision values are more or less constant for
all materials or for group of materials, an overall average value
may be presented. Although such averaging may not have
statistical validity, it does have practical value.

Appendix 3—Flowchart.

Table 2 Model table for presentation of chemistry results from AOAC Official Methods

Table 200X.XX Interlaboratory results for [analyte] by [technique]

Material Reproducibility
a(b) Repeatabiltiy RSDr, %
Matrix Level (units) No. of labs Mean (units) Recovery, % RSDR, % HORRAT

a(b)
a = Number of laboratories remaining after removal of the number of outliers indicated by (b).

© 2002 AOAC INTERNATIONAL