QAS/14.

598
Supplement 10

WHO Vaccine

Checking the accuracy of

temperature control
and monitoring devices
Technical supplement to
WHO Technical Report Series, No. 961, 2011

Annex 9: Model guidance for the storage and transport of time and
temperature–sensitive pharmaceutical products

August 2014

© World Health Organization 2014

WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22
791 4857; e-mail: [email protected]). Requests for permission to reproduce or translate WHO publications – whether for
sale or for noncommercial distribution – should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; e-
mail: [email protected]).
The designations employed and the presentation of the material in this publication do not imply the expression of any
opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city
or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent
approximate border lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or
recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors
and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by the World Health Organization to verify the information contained in this
publication. However, the published material is being distributed without warranty of any kind, either expressed or
implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World

`
 Health Organization be liable for damages arising from its use. The named authors alone are responsible for the views
expressed in this publication.

`
Acknowledgments
The author of this document is Claude Hafner of Berlinger & Co. AG., Switzerland.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 1
Contents
Acknowledgments ................................................................................................................................ 1
Contents ................................................................................................................................................... 2
Abbreviations ........................................................................................................................................ 3
Glossary ................................................................................................................................................... 4
1. Introduction .................................................................................................................................. 5
1.1 Requirements ...................................................................................................................................... 5
1.2 Objectives.............................................................................................................................................. 6
1.3 Target readership .............................................................................................................................. 6
2. Guidance ......................................................................................................................................... 7
2.1 Associated materials and equipment ........................................................................................ 7
2.2 Procedure ............................................................................................................................................. 8
2.2.1 Prerequisites ............................................................................................................................................ 8
2.2.2 Establishing the ice-point bath (excerpt from ASTM E563-11) ...................................... 8
2.2.3 Placing the device in the bath.......................................................................................................... 9
2.2.4 Carrying out the accuracy check, step-by-step ..................................................................... 10
2.2.5 Maintaining the bath temperature ............................................................................................ 10
2.2.6 Actions to take following the test ............................................................................................... 11
References ............................................................................................................................................12
Annex 1 - Generic temperature accuracy check form ...........................................................13
Revision history ..................................................................................................................................14

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 2
Abbreviations
DUT Device Under Test
SOP Standard Operating Procedure
TTSPP Time and Temperature-Sensitive Pharmaceutical Product

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 3
Glossary
Standard Operating Procedure (SOP): A set of instructions having the force of a
directive, covering those features of operations that lend themselves to a definite or
standardized procedure without loss of effectiveness. Standard operating policies and
procedures can be effective catalysts to drive performance improvement and improve
organizational results.
Temperature control device: A device which actively controls the operation of cooling
plant used to store or transport TTSPPs.
Temperature monitoring device: A device which monitors the temperature of spaces
used to store or transport TTSPPs.
Thermal time constant: The most common definitions of the thermal reaction time are
the so-called "Tau" (τ, the 19th letter of the Greek alphabet) and "T90". "Tau" stands for
the time a device needs to adapt to 63% of the end value of a temperature change whereas
"T90" represents the time to adapt to 90% of the change. "T90" approximately equals to
2.5 * "Tau". These constants are commonly evaluated by experiment on the test device
under well-defined conditions as described in EN12830.
Time and temperature sensitive pharmaceutical product (TTSPP): Any
pharmaceutical good or product which, when not stored or transported within pre-
defined environmental conditions and/or within pre-defined time limits, is degraded to
the extent that it no longer performs as originally intended.
Triple point: The temperature and pressure at which a substance can exist in equilibrium
in the liquid, solid, and gaseous states. The triple point of pure water is at 0.01 degrees
Celsius and 4.58 millimetres of mercury and is used to calibrate thermometers.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 4
1. Introduction
This technical supplement has been written to amplify the recommendations given in
Section 4.10 of WHO Technical Report Series No. 961, 2011, Annex 9: Model guidance for
the storage and transport of time- and temperature-sensitive pharmaceutical products1. It
describes a way to check the accuracy of temperature monitoring and temperature control
devices using the “ice-water” procedure, and it outlines the limitations of this approach.
This method can be used in situations where it is not possible to carry out a full three
point calibration using the services of a of a nationally or internationally accredited
calibration laboratory.

1.1 Requirements
As a general rule, temperature measurement and control devices must periodically be
calibrated in order to prove their accuracy over the full operating temperature range and
according to the device’s data sheet definitions. Proven accuracy is mandatory because
inaccurate readings can lead to a false sense of security and place TTSPPs at risk.
Some devices are covered by calibration certificates from the device manufacturer. These
certificates are valid for a defined period of time and the associated devices may be used
throughout this period without additional calibration. An example of this is a single use
temperature monitoring device which is designed to be discarded at the end of a journey,
or when the battery powering the device expires. In order to ensure conformity, the
manufacturer of such products must supply a calibration certificate with the device.
However, there are circumstances under which proper device calibration or re-calibration
is needed. The list below is not comprehensive but illustrates some of these
circumstances:
 A calibration certificate is not available because it has been lost.
 The device is used for longer than the period covered by the calibration certificate,
either at the user’s risk or with the approval of the manufacturer.
 The device was used or treated beyond the manufacturer's data sheet limitations
(e.g. excessive temperature, shock, etc.).
 The battery powering the device was replaced.
 The device's measurements are suspect.
 The device manufacturer specifies that a calibration procedure should be carried
out at regular intervals.
 Regulatory bodies require regular proof of calibration - e.g. at 12 month intervals –
and proof of calibration cannot be provided by the manufacturer’s certificate.

1 http://apps.who.int/medicinedocs/documents/s18683en/s18683en.pdf

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 5
1.2 Objectives
Wherever possible, calibration should be carried out in accordance with the device
manufacturer’s instructions, or by following a device-specific SOP. Ideally, a full three-
point calibration should be carried out by a nationally or internationally accredited
calibration laboratory with proven accuracy standards and appropriate equipment.
However, there are many circumstances where accredited calibration is not possible
because no suitable laboratory is available. The simple and accurate method described in
this Technical Supplement can be used to prove the device’s functionality and accuracy at
one single point of temperature using the so-called "ice-water procedure".

1.3 Target readership

This document is intended to be read by managers and technical staff who are responsible
for the monitoring, installation and maintenance of temperature measurement and control
equipment throughout the cold chain. Responsible managers must understand the
necessity of calibration; in the absence of a suitable calibration laboratory, technical
personnel must be able to carry out and/or supervise the accuracy checking procedure
described below.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 6
2. Guidance
The method described below is relatively simple to carry out but requires close attention
to detail. The accuracy of the results are also dependent upon the use of a high quality
reference thermometer with a valid calibration certificate.

2.1 Associated materials and equipment

Reliable results will be achieved if the following equipment and materials are used:
a. Wherever possible, use reference temperature measurement equipment regularly
calibrated by an accredited laboratory, e.g. Fluke Hart Scientific precision
equipment2. The illustration below shows an example of this type of instrument.
Figure 1 – Example of a reference thermometer

Source: Fluke Hart

b. Always use a temperature measurement reference instrument which is of higher

accuracy than the device to be checked – for example, a thermometer with a rated
accuracy of ±0.2°C should be used to check a device with a rated accuracy of
±0.3°C.
c. A thermally insulated container, tub open to the atmosphere, and large enough to
contain enough melting ice-water in order to provide stable temperature
conditions and allow full immersion of the device under test (DUT) as described in
Section 3. In the cold chain operating environment, a vaccine carrier or small cold
box with a hinged or separate lid would be a suitable choice. The required size
depends on the dimensions of the DUT.
d. Disposable latex gloves.
e. Enough clean distilled water must be available to make ice cubes and to set up a
proper and stable ice-water triple-point mixture. A transparent water-proof
sealable pouch should be used to seal the DUTs before immersing it in the liquid in
order to avoid water ingress and resulting damage. As much air as possible should
be extracted to ensure that the DUT is in good contact with the ice-water mixture.
Sealing is always recommended, even for DUT's with a high IEC 60529 IP

2http://us.flukecal.com/products/temperature-calibration Note that mention of this company

does not imply any endorsement or recommendation by the World Health Organization.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 7
 protection class rating because the protection might have been damaged – for
example by dropping or battery replacement. Where the DUT has an external
sensor, the sensor may be immersed directly in the bath, provided it has an IP7 or
IP8 protection class rating. Otherwise it should be sealed in a pouch as described
above, with the pouch tied tightly around the lead above the level of the ice-water.

2.2 Procedure
The ice-water bath provides an accurate reference temperature at 0.0°C if the melting ice -
water mixture is properly set up, handled and maintained. An accurate temperature is
achieved by this method because an ice-water mixture in a container which is open to the
atmosphere will stabilize at its own “triple point”. At this point all three aggregate states of
water coexist: liquid, solid and gaseous. For more physical details refer ASTM E563-11.

2.2.1 Prerequisites
a. Only place clean equipment and distilled water inside the container. Use clean
latex gloves to handle ice and equipment.
b. Although the temperature of the ice-water mixture stabilizes itself, its temperature
must still be monitored before and during the preocedure using the calibrated
reference thermometer. This is a mandatory requirement in order to prove the
functionality and stability of the ice-water bath.
c. During extended testing periods enough ice must be added to the bath to maintain
the water–ice equilibrium temperature and to prevent a possible temperature rise
caused by excessive ice melting.
d. Note that only DUTs which display a temperature reading can be checked by this
method because the temperature indicated by the DUT must be compared to the
ice-water bath temperature. Devices without a display need additional equipment
to capture an immediate measurement read-out so that this can be compared with
the reference temperature reading. This additional equipment is manufacturer-
specific.

2.2.2 Establishing the ice-point bath (excerpt from ASTM E563-11)

a. Before using the bath, chill the required amount of distilled water close to 0.0°C.
b. Freeze a suitable amount of the same water to produce ice-cubes, making sure that
there is a sufficient quantity for the complete test run.
c. Prepare shaved or fine crushed ice with a maximum 2mm to 3mm particle
diameter; the finer the ice particles, the more accurate the ice-water temperature.
d. Prepare the bath in the clean thermally insulated container. The container should
be large enough not to affect the water-ice equilibrium temperature. The width,
length or diameter and the overall depth must ensure that, when the thermal
equilibrium state is reached, the test objects will not significantly modify the
temperature of the bath over the region to which the ice point is to be applied. For
normal applications, a width, length or diameter at least 100mm larger than the
maximum DUT dimension size, and a depth of at least 300mm should be sufficient.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 8
 e. Alternately add shaved ice and chilled water to the vessel, using just enough water
to saturate the ice but not enough to float it. As the vessel fills, compress the ice-
water mixture to force out excess water. The objective is to surround each particle
of ice with water, filling all voids, but to keep the ice particles as close together as
possible. Continue adding ice and water and compressing until the vessel is filled
to the required level. Decant or siphon off excess water.
f. Use the reference thermometer throughout the entire test period to confirm that a
stable temperature is maintained.
g. Cover the ice-point bath to protect it during the test period. Use an opaque and
thermally insulating cover or stopper that is suitable for the application. This
reduces heat transfer to the ambient environment through the surface of the bath.
Allow the bath and vessel to equilibrate for at least 30 minutes before using.

2.2.3 Placing the device in the bath

a. Seal the DUT in a clean plastic bag. Make sure that as little air as possible is
trapped inside the bag in order to avoid false results by floating and/or lack of
contact between the DUT and the ice-water.
b. Pre-cool the DUT in water at less than +3.0°C before immersing it in the bath. Pre-
cooling the DUT reduces the time to reach equilibrium at the ice point; it also helps
to preserve the bath at the ice point for a prolonged time. Additionally it ensures
that the water-ice interface will be in close contact with the DUT; negligible
melting is important, otherwise the water film thickness between the DUT and the
ice-water will increase and distort the test results.
c. Form a well in the ice-water bath that has the dimensions and intended immersion
depth of the DUT.
d. Insert the sealed DUT to a depth of at least ten object diameters or heights
respectively below the surface. Keep the DUT a minimum of 30mm above the
bottom of the container in order to avoid the zone at the bottom where denser
melt water tends to accumulate.
e. Replace the lid. For devices with an external sensor, make sure that the lid seals
well around the sensor lead.
f. Allow the bath and DUT to come to thermal equilibrium. Allow for the thermal
time constant of the DUT.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 9
 Figure 2 – Ice-water bath arrangement

2.2.4 Carrying out the accuracy check, step-by-step

a. Establish and maintain the ice-water bath, prove its temperature with the
reference thermometer.
b. Ensure that the DUT remains immersed in the ice-water bath as described in point
3.3d above.
c. Directly read the DUT temperature display while it is still immersed in the bath. If
this is not possible, remove the DUT and immediately read its temperature
indicator; prompt action avoids false readings due to exposure to the ambient
temperature.
d. Record the readings on a accuracy check reference data chart as per sample taken
from EVM-SOP-E2-2. Store the chart at least until the next accuracy check takes
place, and preferably for a minimum of three years.

2.2.5 Maintaining the bath temperature

a. As ice particles in the bath melt, excess water begins to accumulate. This melt
water has a temperature slightly above 0.0°C. Since the density of water is at its
maximum at +4.0°C, the slightly warm melt water will collect at the bottom of the
bath and, hence, around the DUT. Under these conditions, the bath will no longer
be at 0.0°C and cannot serve as an ice-point bath. For this reason surplus water
should be removed, as it accumulates, from the bottom of the bath by decanting or
siphoning. The presence of excess water can be detected if water overspill occurs
when the ice is depressed. Add ice particles, and chilled water, as necessary so that

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 10
 the ice slush column always extends to at least 30mm below the lowest point of the
test object.
b. In order to sustain the ice point over prolonged periods, the ice-point bath can be
immersed in another larger insulated bath that is kept near to 0.0°C.

2.2.6 Actions to take following the test

There are two possible test outcomes: pass or fail.
1. The DUT passes the test
A pass is achieved if the DUT temperature indication deviates from the reference
thermometer reading3 by less than the tolerance allowed in the data sheet. In this case the
DUT should be physically labelled as follows:
Accuracy valid until: <enter the date one year after the accuracy check>.
Note: If the manufacturer or the regulatory authority stipulates a shorter or longer
period of validity, enter the appropriate date to take account of this.
After this period expires the accuracy check must be repeated, as described in this
document.
2. The DUT fails the test
If the DUT temperature indication deviates from the reference thermometer reading4 by
more than the tolerance allowed in the data sheet. In this case, take one if the following
actions:
a. If the sensor can be adjusted, carry out the adjustment according to the
manufacturer's instructions. Then repeat the accuracy check according to the
procedure described above.
b. If the sensor can be replaced, but the accuracy of the device is not solely dependent
on the performance of the sensor head, replace the sensor according to the
manufacturer's instructions. Then repeat the accuracy check.
c. If the sensor can be replaced, and the accuracy of the device depends entirely on
the performance of the sensor head, the sensor can simply be replaced with a new
factory-calibrated sensor in accordance with the manufacturer’s instructions. No
further check is then required until the next accuracy/calibration date falls due.
d. If the sensor can neither be replaced nor adjusted, the whole DUT must be
replaced and a non-conformity report must be issued to the relevant decision
makers in order to ensure that the DUT is removed from service and replaced.

3 For example, if the reference instrument has an accuracy of ±0.5°C and the DUT deviates 0.3°C
from the reference instrument reading, this is a pass.
4 For example, if the reference instrument has an accuracy of ±0.5°C and the DUT deviates 0.6°C

from the reference instrument reading, this is a failure.

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 11
References
 ASTM E563-11 Standard Practice for Preparation and Use of an Ice-Point Bath as a
Reference Temperature
 EN 12830:1999 Temperature recorders for the transport, storage and distribution of
chilled, frozen, deep-frozen/quick-frozen food and ice cream. Tests, performance and
suitability.
 IEC 60529: Consolidated Edition 2.1 (incl. am1): Degrees of protection provided by
enclosures (IP Code).
 WHO. EVM-SOP-E2-2. Checking the accuracy of temperature monitoring devices.
Available online at:
http://www.who.int/immunization/programmes_systems/supply_chain/evm/en
/index2.html
 WHO Technical Report Series No. 961, 2011, Annex 9: Model guidance for the
storage and transport of time- and temperature-sensitive pharmaceutical
http://apps.who.int/medicinedocs/documents/s18683en/s18683en.pdf

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 12
Annex 1 - Generic temperature accuracy check form
Store location: …………………………………………………………………………......
Form start date: …………………………………………………………………………....
Form finish date: .…………………………………………………………………………..
Date Device Device Reference Check Reference Difference from Initials Comments Verified
description location thermometer method temperature reference by/ date
12 Dec 2013 DUT Cold room # 1 XYZ 500 Water-ice 0.0°C +0.2°C UK checked - OK SJ 15
#12341234 bath Dec2013

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 13
Revision history
Date Change summary Reason for change Approved

Technical Supplement: Checking the accuracy of temperature control and monitoring devices 14