EXAMPLES OF

VALIDATION PROTOCOLS
OF THE ALTERNATIVE
MICROBIOLOGICAL METHODS
ACCORDING TO CHAPTER
5.1.6 “Alternative methods
for control of micro-
biological quality”

European Pharmacopoeia EDQM

Edition 2018
EXAMPLES OF VALIDATION PROTOCOLS OF THE
ALTERNATIVE MICROBIOLOGICAL METHODS
ACCORDING TO CHAPTER 5.1.6 “Alternative
methods for control of microbiological quality”
European Pharmacopoeia

Edition 2018

European Directorate for the Quality of Medicines & HealthCare

 English version
2018
Making copies of this file for commercial
purposes or posting it on a website
for which access is charged is strictly
prohibited. Re-use of the file, in whole
or in part, requires that the source
be clearly cited and the EDQM
([email protected])
be informed.
European Directorate for the Quality
of Medicines & HealthCare (EDQM)
Council of Europe
7, allée Kastner
CS 30026
F-67081 STRASBOURG
FRANCE
Cover images: (© Fotolia.com):
© catalin
© motorolka
© angellodeco
Director of the Publication: Dr S. Keitel
Page layout: EDQM
www.edqm.eu
© Council of Europe, 2018
 Table of contents

1. EXAMPLE OF A VALIDATION PROTOCOL FOR A RAPID STERILITY TEST...... 4

1.1 PRINCIPLE OF THE ALTERNATIVE METHOD ................................................... 4
1.2 VALIDATION OF THE ALTERNATIVE METHOD............................................... 4
1.2.1 Specificity............................................................................................................. 4
1.2.2 Detection limit ...................................................................................................... 5
1.2.3 Robustness ............................................................................................................ 5
1.2.4 Suitability testing.................................................................................................. 6
1.2.5 Equivalence testing .............................................................................................. 6
2. EXAMPLE OF A VALIDATION PROTOCOL FOR A QUANTITATIVE METHOD
(ENUMERATION OF MICRO-ORGANISMS) ....................................................................... 6
2.1 PRINCIPLE OF THE ALTERNATIVE METHOD ................................................... 6
2.2 VALIDATION OF THE ALTERNATIVE METHOD............................................... 6
2.2.1 Accuracy, precision, quantitation limit, linearity and range ................................ 6
2.2.2 Specificity............................................................................................................. 7
2.2.3 Robustness ............................................................................................................ 7
2.2.4 Suitability testing.................................................................................................. 8
2.2.5 Equivalence testing .............................................................................................. 8
3. EXAMPLE OF A VALIDATION PROTOCOL FOR A MOLECULAR-BASED
MICROBIAL IDENTIFICATION METHOD .......................................................................... 8
3.1 PRINCIPLE OF THE ALTERNATIVE METHOD ................................................... 8
3.2 VALIDATION OF THE ALTERNATIVE METHOD............................................... 8
3.2.1 Accuracy............................................................................................................... 8
3.2.2 Specificity............................................................................................................. 8
3.2.3 Robustness ............................................................................................................ 9

3
 EXAMPLES OF VALIDATION PROTOCOLS OF THE ALTERNATIVE
MICROBIOLOGICAL METHODS ACCORDING TO CHAPTER 5.1.6“Alternative
methods for control of microbiological quality”
The following are examples of alternative methods developed, validated and used by various
laboratories. The examples are based on the methods and equipment used by the laboratories
involved and are published for information only and not for general application. They are not
provided as a recommendation from the Pharmacopoeia and cannot be cited as references.
Although the proprietary names of the instruments are not given, the characteristics of some
are so specific that they may be identifiable. It is not the intention here to advocate the use of
a specific technology, or to give preference to one manufacturer over another. The sole
intention of this section is to give users an insight into the level of information required for
regulatory acceptance.

1. EXAMPLE OF A VALIDATION PROTOCOL FOR A RAPID STERILITY

TEST

1.1 PRINCIPLE OF THE ALTERNATIVE METHOD

This alternative sterility test method is based on membrane filtration. Following a pooling
step, the sample is filtered through filtration funnels. The membranes are subsequently
transferred onto cassettes containing suitable solid nutrient media and incubated. Incubation
conditions are chosen in order to reproduce the incubation conditions of the pharmacopoeial
sterility test. Incubation time is defined taking into account the time required to ensure
detection of a worst-case stressed micro-organism, while including an adequate safety margin.
After incubation, the membranes are sprayed with the appropriate reagents and examined
using a system that detects (micro)colonies by adenosine triphosphate (ATP)
bioluminescence.

1.2 VALIDATION OF THE ALTERNATIVE METHOD

1.2.1 Specificity
The specificity of a qualitative microbiological method is defined as its ability to detect only
the micro-organisms sought (i.e., it does not generate false positive results). In the case of a
sterility test, this implies that the method is capable of detecting all kinds of micro-organisms
without producing a positive test result in the absence of micro-organisms.
To demonstrate this ability, both methods, alternative and pharmacopoeial, are performed in
parallel with an inoculum containing less than 100 CFU of a representative panel of micro-
organisms, including pharmacopoeial reference strains (see chapter 2.6.1, table 2.6.1.-1) and
in-house isolates.
The in-house isolates are selected to represent the flora obtained during environmental
monitoring, and include slow-growing micro-organisms.

4
If necessary, specific stress conditions, such as heat treatment (for non-spore-forming micro-
organisms) or nutrient depletion (for spore-forming micro-organisms), may be applied before
the suspensions are used 1.
Acceptance criterion: all the micro-organisms in each run show growth.
In order to demonstrate that no false positive results are obtained in the absence of micro-
organisms, the background bioluminescence is evaluated by examining at least 3 sterile
membranes through which the product to be examined has been filtered. A risk analysis must
have previously shown that ATP from other sources can be excluded as a cause of false
positive results.
Acceptance criterion: no false positive results occur due to background bioluminescence.

1.2.2 Detection limit

The detection limit is evaluated using the pharmacopoeial reference strains recommended in
chapter 2.6.1, and stressed in-house isolates representative of a wide variety of potential
contaminants (including endospores and very slow-growing micro-organisms). Three 10-fold
dilutions are prepared from a starting quantity of 50 CFU for each micro-organism (i.e., a
range from approximately 50 CFU to 0.05 CFU). A significant number of replicates (e.g.,
minimum 10 replicates of each dilution) are tested using the alternative method and the
pharmacopoeial method, respectively. The 95 per cent confidence limits of the most probable
number (MPN) per gram is deduced from a MPN table for each method. The 95 per cent
confidence limits for the ratio between the respective proportions of micro-organisms detected
by the methods is also estimated.
Acceptance criterion: The detection limit value of the alternative method must not be greater
than that of the pharmacopoeial method.
The detection limit is evaluated based on the results of at least two independent test runs per
micro-organism.

1.2.3 Robustness
Two parameters have been found to have a major impact on test results and may also present
significant variability: the incubation time and the time required to transfer the filter
membrane from the spraying station to the detection system. As regards materials and
environments, robustness is tested using different filter funnel lots, reagent lots, media lots,
rinsing fluid lots, anaerobic incubation systems, analysts and laboratories. Robustness
experiments are conducted with both a fast-growing and a slow-growing micro-organism in
inocula containing 10-100 CFU. Three test runs with 3 replicates are performed for each
micro-organism and robustness parameter.
Acceptance criterion: no statistically significant difference is observed between the conditions
of interest (e.g., reagent lot A and reagent lot B) for a predefined micro-organism recovery
after 14 days of incubation.

1
Gray, J.C., Staerk, A., Berchtold, M., Hecker, W., Neuhaus, G., Wirth, A. Growth-promoting Properties of
Different Solid Nutrient Media Evaluated with Stressed and Unstressed Micro-organisms: Prestudy for the
Validation of a Rapid Sterility Test. PDA Journal of Pharmaceutical Science and Technology 2010, 64: 249-263
5
1.2.4 Suitability testing
The product-specific suitability study focuses on identification of an adequate rinse protocol
and demonstration that the product to be examined does not interfere with the alternative
method by inducing high background bioluminescence or inhibiting the bioluminescence
reaction. To demonstrate absence of a residual inhibitory effect after rinsing the membrane, 3
test runs are carried out with the pharmacopoeial reference strains recommended in chapter
2.6.1 and with in-house isolates. A control (identically inoculated membrane through which
no product has been filtered) is prepared for comparison. Possible inhibition of the
bioluminescence reaction is excluded by inoculating 3 membranes through which the product
to be examined has been filtered with 10-100 CFU of a stressed, slow-growing micro-
organism.
Acceptance criterion: an equivalent number of bioluminescent (micro)colonies is recovered
compared to the control.

1.2.5 Equivalence testing

Contaminated test samples are prepared by inoculation with fewer than 5 CFU of 3 different
test micro-organisms. Three test runs are then performed with 10 replicates of each of the
three contaminated samples, with the alternative and the pharmacopoeial method in parallel.
Routine test conditions are simulated as closely as possible, for example by also applying a
standard rinse protocol. The results obtained are subjected to statistical evaluation.
Acceptance criterion: statistical analysis demonstrates that the results of the alternative
method enable an unequivocal decision as to whether compliance with the standards of the
monographs would be achieved if the pharmacopoeial method was used.

2. EXAMPLE OF A VALIDATION PROTOCOL FOR AN ALTERNATIVE

QUANTITATIVE METHOD (ENUMERATION OF MICRO-ORGANISMS)
2.1 PRINCIPLE OF THE ALTERNATIVE METHOD
The alternative method uses solid phase cytometry which combines 3 well-known
technologies: membrane filtration, fluorescent cell staining and laser scanning. After standard
membrane filtration, all viable micro-organisms retained on the membrane are fluorescently
stained through enzymatic cleavage of a non-fluorescent substrate to liberate free
fluorochrome into the cytoplasm. Only metabolically active (viable) cells and spores with
intact membranes will appear fluorescent. . After viability staining, a fully overlapping laser
scan is performed and any viable cells present on the membrane are individually detected and
automatically counted. Given that the solid phase cytometry test for the enumeration of
micro-organisms counts viable cells individually, the result is expressed as the number of
viable cells per unit volume and not in CFU per unit volume.

2.2 VALIDATION OF THE ALTERNATIVE METHOD

2.2.1 Accuracy, precision, quantitation limit, linearity and range

The alternative method is validated using a representative panel of different micro-organisms,
including pharmacopoeial reference strains (see chapter 2.6.12, table 2.6.12.-1) and in-house
isolates.

6
The accuracy, precision, quantitation limit, linearity and range of both the alternative method
and the pharmacopoeial quantitative method (membrane filtration method) for the
enumeration of micro-organisms are determined using an independent dilution series –
300 CFU/mL, 100 CFU/mL, 50 CFU/mL, 10 CFU/mL, 5 CFU/mL and 1 CFU/mL –
individually prepared for each micro-organism by three different analysts. Five replicates
(Note: a 6th replicate of the 100 CFU/mL dilution is prepared for the determination of
precision) of each concentration of the independent dilution series are tested in parallel by 3
analysts using the alternative and the pharmacopoeial methods. The experiments are
performed using different equipment and different lots of reagents.
Acceptance criteria:
(i) Accuracy: the alternative method is shown to recover at least as many micro-
organisms as the pharmacopoeial method, using appropriate statistical analysis;
(ii) Precision: the coefficient of variation is calculated and the precision of the
alternative method is comparable to that of the pharmacopoeial method.
(iii) Linearity: the regression line is calculated and the method is linear if the estimated
slope is significant and the deviation from linearity is non-significant (see general
chapter 5.3).
(iv) Range: the range of the alternative method is equal or greater to the range of the
pharmacopoeial method.
(v) Quantitation limit: the lowest concentration in the linear range is considered to be
the quantitation limit of the method. The value obtained for this limit with the
alternative method must not be greater than that of the pharmacopoeial method.

2.2.2 Specificity
To determine the specificity of both methods, a suspension of S. aureus with a titre of
approximately 100 CFU/mL is prepared and divided into 2 aliquots for each assay. One
aliquot of the suspension is treated to inactivate/kill all viable micro-organisms using
conditions that avoid lysis of the non-viable cells (for example, by treatment with 100 per cent
ethanol). In both methods (solid phase cytometry and membrane filtration, for the alternative
and pharmacopoeial methods, respectively), 3 replicates of 1 mL each of the treated
suspension, representing approximately 100 non-viable cells are filtered individually using
membrane filtration equipment. After filtration, 3 replicates of 1 mL each of the untreated
suspension, representing approximately 100 viable cells or CFUs, are filtered using the same
membranes. Consequently, about 100 non-viable cells and about 100 viable cells or CFUs are
retained by each of the membranes.
Acceptance criterion: regardless of the different principles on which the alternative and the
pharmacopoeial methods for enumeration of micro-organisms are based, the final test result
for both these methods is about 100 viable cells or CFUs per membrane.

2.2.3 Robustness
The repeatability, intermediate precision and reproducibility of the alternative and the
pharmacopoeial methods are determined by deliberately varying a number of key parameters,
for example, the validation experiments can be performed on different days, by different
analysts, using different systems or different reagent batches and in different laboratories.
Acceptance criterion: these deliberate variations do not have a significant effect on the test
results.

7
2.2.4 Suitability testing
Suitability testing on the alternative and pharmacopoeial methods for enumeration of micro-
organisms is performed in accordance with section 4-5 of general chapter 2.6.12. It must
therefore be proven that the product does not inhibit the growth or detection of viable micro-
organisms under the conditions of the test. In order to do so, test suspensions of not more than
100 CFU for each test micro-organism are added separately into the rinse during the
membrane filtration step of both methods (at the same rinsing step and in the same manner).
Acceptance criterion: a recovery of 50-200 per cent is obtained in accordance with general
chapter 2.6.12.

2.2.5 Equivalence testing

The equivalence of the enumeration tests is demonstrated directly by performing the assays
side-by-side for a predetermined period of time (or number of samples) and then carrying out
statistical analysis to compare the results.
Acceptance criterion: statistical analysis demonstrates that the results of the alternative
method enable an unequivocal decision as to whether compliance with the standards of the
monographs would be achieved if the pharmacopoeial method was used.

3. EXAMPLE OF A VALIDATION PROTOCOL FOR A MOLECULAR-

BASED MICROBIAL IDENTIFICATION METHOD
3.1 PRINCIPLE OF THE ALTERNATIVE METHOD
The gene that encodes rRNA (rDNA) is the most conserved DNA in all cells and is
extensively used to identify micro-organisms and determine their taxonomy and phylogeny.
The 16S rDNA sequence has hypervariable regions that have diverged during evolution,
flanked by strongly conserved regions. Primers for the identification of micro-organisms are
designed to bind to these conserved regions and to amplify the variable regions between them.
The amplicons of the PCR reaction are purified and subsequently forward- and reverse-
sequenced. Based on these forward and reverse sequences, a consensus sequence is assembled
and compared to sequence databases.

3.2 VALIDATION OF THE ALTERNATIVE METHOD

3.2.1 Accuracy
The accuracy of the method is investigated using several well-characterised bacterial-type,
yeast and mould strains, which are found in the database.
Different subcultures of well-characterised micro-organisms are harvested (at least 3
individual colonies are collected), prepared and subjected to identification. All strains are
sequenced 3 times on 3 successive days by 1 analyst.
Acceptance criterion: Micro-organisms are identified correctly to species level and above the
pre-defined reporting level for sequence homology (for example, see Table 1).

3.2.2 Specificity
The specificity of an identification method is its ability to discriminate micro-organisms
actually present from interfering factors that generate false identification results, e.g.,

8
mixtures of micro-organisms that give positive identification results for micro-organisms that
are not present in the sample. Mixtures of appropriate micro-organisms can be used to
demonstrate the ability of the test to reveal either an invalid result or a valid result for a
micro-organism actually present in the mixture. The assay must be able to differentiate
between valid and invalid results which are due to mixed cultures or DNA.
Acceptance criterion: the results of the specificity test using mixed cultures (quantified and
titred based on CFU or using an analytical method to standardise the DNA titre) lead to valid
identifications. These results must clearly demonstrate that the analysis of mixed cultures does
not result in false identifications (validation example, see Table 2).

3.2.3 Robustness
Method robustness can be determined at the same time as method accuracy and precision.
During multiple identifications of the same micro-organism, several critical parameters can be
deliberately modified.
An intra-laboratory study can be performed using modifications of the critical parameters for
microbial identification. These include carrying out the identification on different
working days; with different analysts within one laboratory; using different of test-kit and
reagent lots; using cultures of different ages (e.g. 3- and 7-day-old cultures of the reference
strains).
Different subcultures of well-characterised micro-organisms are harvested (at least 3
individual colonies are collected), prepared and subjected to identification. All strains are
sequenced 3 times on 3 successive days by the same analyst (accuracy and intra-laboratory
robustness) and on 3 successive days by 2 analysts (intra-laboratory robustness). Sample
preparation and instrument set-up are carried out according to the internal procedures of the
laboratory.
It is also possible to modify this study to include an inter-laboratory study, in which case the
microbial identification is performed on the same strains by different laboratories using:
– different types of equipment, if available;
– slightly different procedures for the purification of the target DNA;
– different test-kits and test-kit lots (for the purification of the PCR-amplicons as well as for
purification of the sequencing products);
– different versions of the data-processing software, if applicable.
Acceptance criterion: Micro-organisms are identified correctly to species level, with a pre-
defined percentage match between the database and the test strains (validation example, see
Table 1).
If feasible, identical acceptance criteria for accuracy and robustness are applied by each
collaborating laboratory.

9
Table 1 – Summary of the validation studies for accuracy and robustness for a molecular-based microbial identification method
Repeatability Reproducibility

Sequence homology/

Average of sequence
Average of sequence
sequence homology/

sequence homology/
Standard deviation

Standard deviation
Max. difference in

Max. difference in
[% match]

Worst sequence

homology/ [%]
homology/ [%

homology/ [%
homology/ [%

Best sequence
Run 1 Run 2 Run 3

match]

match]
match]

[%]
[%]
Species
Pseudomonas aeruginosa (ATCC 10145 T/DSM 50071T) 100.0 99.8 99.7 0.3 99.8 0.15 99.68 100.00 0.32 99.94 0.12
Stenotrophomonas maltophilia (ATCC 13637 T/ DSM 100.0 99.8 100.0 0.2 99.9 0.12 99.79 100.00 0.21 99.98 0.07
50170 T)
Escherichia coli (ATCC 11775 T/ DSM 30083T) 99.6 99.9 99.8 0.3 99.8 0.15 99.58 99.90 0.32 99.80 0.11
Salmonella choleraesuis (ATCC 13312 T/DSM 14846 T) 99.8 100.0 100.0 0.2 99.9 0.12 99.79 100.00 0.21 99.98 0.07
Staphylococcus aureus (ATCC 12600 T/DSM 20231T) 99.8 100.0 100.0 0.2 99.9 0.12 99.79 100.00 0.21 99.98 0.07
Staphylococcus epidermidis (ATCC 14990 T/ DSM 100.0 100.0 100.0 0 100.0 0 99.70 100.00 0.30 99.91 0.13
20044T)
Streptococcus agalactiae (ATCC 13813 T/DSM 2134 T) 100.0 100.0 99.9 0.1 100.0 0.06 99.38 100.00 0.62 99.85 0.20
Enterococcus faecalis (ATCC 19433 T/ DSM 20478 T) 100.0 99.9 99.9 0.1 99.9 0.06 99.80 100.00 0.20 99.88 0.06
Bacillus cereus (ATCC 14579 T/ DSM 31T) 100.0 100.0 100.0 0 100.00 0 99.60 100.00 0.40 99.93 0.14
Bacillus subtilis subsp. Subtilis (ATCC 6051 T/ DSM 10T) 99.8 99.8 99.8 0.2 99.8 0 99.60 99.80 0.20 99.77 0.07
Corynebacterium bovis (ATCC 7715 T/DSM 20582T) 100.0 100.0 100.0 0 100.0 0 100.00 100.00 0 100.00 0
Carnobacterium gallinarum (ATCC 49517 T/ DSM 4847 T) 99.9 99.9 99.9 0.1 99.0 0 99.70 99.90 0.20 99.81 0.11
Clostridium perfringens (ATCC 13124 T/DSM 756 T) 99.7 99.7 99.7 0.3 99.7 0 99.67 99.78 0.11 99.71 0.05
Candida albicans (ATCC 18804 T/ CBS 562T) or (ATCC 100.0 100.0 100.0 0 100 0 100.00 100.00 0 100.00 0
10231/CBS6431)
Aspergillus brasiliensis (ATCC 16404/CBS 733.88) 100.0 100.0 100.0 0 100 0 100.00 100.00 0 100.00 0

10
 Table 2 – Summary of specificity validation studies (mixed cultures quantified and
standardised using a DNA quantification method)

Experiment Micro-organism(s) Ratio ID Result

Pure Culture 1 S. aureus S. aureus

Pure Culture 2 M. luteus M. luteus

Pure Culture 3 R. pickettii R. pickettii

Pure Culture 4 P. aeruginosa P. aeruginosa

Mixed Culture 1 S. aureus + M. luteus 1:1 S. aureus

Mixed Culture 2 S. aureus + M. luteus 1:9 S. aureus

Mixed Culture 3 S. aureus + M. luteus 9:1 S. aureus

Mixed Culture 4 R. pickettii + P. aeruginosa 1:1 R. pickettii

Mixed Culture 5 R. pickettii + P. aeruginosa 1:9 P. aeruginosa

Mixed Culture 6 R. pickettii + P. aeruginosa 9:1 R. pickettii

Mixed Culture 7 S. aureus + R. pickettii 1:1 R. pickettii

Mixed Culture 8 S. aureus + R. pickettii 1:9 R. pickettii

Mixed Culture 9 S. aureus + R. pickettii 9:1 B. gladioli*

Mixed Culture 10 S. aureus + P. aeruginosa 1:1 P. aeruginosa

Mixed Culture 11 S. aureus + P. aeruginosa 1:9 P. aeruginosa

Mixed Culture 12 S. aureus + P. aeruginosa 9:1 P. aeruginosa

Mixed Culture 13 M. luteus + R. pickettii 1:1 R. pickettii

Mixed Culture 14 M. luteus + R. pickettii 1:9 R. pickettii

Mixed Culture 15 M. luteus + R. pickettii 9:1 R. pickettii

Mixed Culture 16 M. luteus + P. aeruginosa 1:1 P. aeruginosa

Mixed Culture 17 M. luteus + P. aeruginosa 1:9 P. aeruginosa

Mixed Culture 18 M. luteus + P. aeruginosa 9:1 P. aeruginosa

Mixed Culture 19 S. aureus + M. luteus + R. pickettii + P. aeruginosa 1:1:1:1 R. pickettii

*ID does not fulfil test acceptance criteria - consensus length is only 188 bp → invalid result

11
 ENG
The Council of Europe is the continent’s leading human rights
organisation. It comprises 47 member states, 28 of which are members
of the European Union. The European Directorate for the Quality of
www.edqm.eu Medicines & HealthCare (EDQM) is a directorate of the Council of Europe.
Its mission is to contribute to the basic human right of access to good
quality medicines and healthcare and to promote and protect public
health.