Performance evaluation of computed radiography systems

Ehsan Sameia)
Department of Radiology, Duke University Medical Center, DUMC Box 3302,
Durham, North Carolina 27710
J. Anthony Seibert
Department of Radiology, UC Davis Medical Center, Sacramento, California 95817
Charles E. Willis
Department of Radiology, Baylor College of Medicine and Edward B. Singleton
Diagnostic Imaging Service, Texas Children’s Hospital, Houston, Texas 77030
Michael J. Flynn
Department of Radiology, Henry Ford Health System, Detroit, Michigan 49202
Eugene Mah
Department of Radiology, Medical University of South Carolina, Charleston, South Carolina 29425
Kevin L. Junck
Department of Radiology, University of Alabama Medical Center, Birmingham, Alabama 35233
共Received 19 September 2000; accepted for publication 15 December 2000兲
Recommended methods to test the performance of computed radiography 共CR兲 digital radiographic
systems have been recently developed by the AAPM Task Group No. 10. Included are tests for dark
noise, uniformity, exposure response, laser beam function, spatial resolution, low-contrast resolu-
tion, spatial accuracy, erasure thoroughness, and throughput. The recommendations may be used for
acceptance testing of new CR devices as well as routine performance evaluation checks of devices
in clinical use. The purpose of this short communication is to provide a tabular summary of the tests
recommended by the AAPM Task Group, delineate the technical aspects of the tests, suggest
quantitative measures of the performance results, and recommend uniform quantitative criteria for
the satisfactory performance of CR devices. The applicability of the acceptance criteria is verified
by tests performed on CR systems in clinical use at five different institutions. This paper further
clarifies the recommendations with respect to the beam filtration to be used for exposure calibration
of the system, and the calibration of automatic exposure control systems. © 2001 American
Association of Physicists in Medicine. 关DOI: 10.1118/1.1350586兴

Key words: computed radiography, photostimulable phosphor radiography, acceptance testing,

quality control, automatic exposure control

I. INTRODUCTION devices, Agfa Medical Systems 共Ridgefield Park, NJ兲, Fuji

Medical Systems 共Stamford, CT兲, Eastman Kodak Health
Computed radiography 共CR兲, scientifically known as photo- Imaging 共Rochester, NY兲, Konica Imaging Systems 共Wayne,
stimulable phosphor radiography, is a digital technology for NJ兲, and Lumisys, Inc 共Sunnyvale, CA兲. There are currently
the acquisition of radiographic images.1,2 CR is the most no industry standards for specifying the performance of these
common digital radiography modality in radiology depart-
ments today, with an estimated 7000 systems in use world-
wide. The technology uses a conventional radiographic ac- TABLE I. CR systems evaluated in this study.
quisition geometry to deposit x-ray energy in a
photostimulable phosphor screen with delayed luminescence Manufacturer CR device Phosphor screen
properties. After irradiation, the screen is stimulated by a Agfa ADC-70 MD-10
scanning laser beam, to release the deposited energy in the ADC-Compact
form of visible light. The released photostimulated light is ADC-Solo
captured by a light detector, converted to digital signals, and
Fuji FCR-9501 ST-VA and ST-VN
registered with the location on the screen from which it has FCR-9501-HQ
been released. The digital data are then postprocessed for AC3-CS
appropriate presentation, and are sent to a hard-copy printer FCR-5000 ST-VN
or a soft-copy display monitor for medical evaluation.
Kodak CR-400 GP-25 and HR
Upon installation and prior to clinical use, CR devices
should be evaluated for satisfactory performance.3,4 As of Lumisys ACR-2000 MD-10
September 2000, there are five manufacturers of CR imaging

361 Med. Phys. 28 „3…, March 2001 0094-2405Õ2001Õ28„3…Õ361Õ11Õ$18.00 © 2001 Am. Assoc. Phys. Med. 361
362 Samei et al.: Performance evaluation 362

TABLE II. Testing devices required to perform the acceptance testing of a port the response using indices which have different depen-
CR imaging device.
dences on exposure. In a large medical institution in which
Testing device CR devices of different kinds might be employed, it is im-
Calibrated x-ray source
portant to assure that the patient images are acquired within
Calibrated hard/soft-copy display devices a certain exposure range to prevent over- and underexpo-
Densitometer 共if a hard-copy display is to be used兲 sures. However, the lack of calibration uniformity makes the
Copper and aluminum filters
definition of the acceptable exposure ranges from the CR
Calibrated ion chamber
Stand for the ion chamber response values cumbersome.
Screen cleaning solution and cloths In general, in order to achieve a consistent level of clini-
Two metric 30 cm steel rulers 共for laser-beam function and spatial cal performance, acceptance testing should utilize a uniform
accuracy tests兲
Three sector-type 共0.4°兲 line-pair phantoms of up to 5 lp/mm frequency cross-platform methodology and uniform criteria so that the
共⭓0.05 mm lead thickness兲 results of the tests can be correlated with clinical perfor-
Low-contrast phantom 共e.g., Leeds TO.12兲 mance standards. Currently, Task Group No. 10 of the
Screen-contact wire-mesh pattern
Screen-contact fine wire-mesh pattern 共e.g., mammography screen-film
American Association of Physicists in Medicine 共AAPM
contact tool兲 TG10兲5 is making an effort to provide a comprehensive stan-
Small lead block 共⬎3 mm thick兲 dardized testing protocol for acceptance testing and quality
Antiscatter grid 共10:1 or 12:1, 103 ln/in.兲 共if the x-ray system does not
control of CR systems. In this work, we have used the pre-
have one兲
Anthropomorphic phantoms 共foot, hand, pelvis, chest, etc.兲 liminary guidelines established by the AAPM Task Group to
Timer evaluate the performance of CR systems currently in use at
Measuring tape different institutions represented by the co-authors. The pa-
Flashlight
Role of masking tape per provides a summary of the tests recommended by the
AAPM Task Group, delineates the specific technical aspects
of the tests, suggests quantitative measures of the perfor-
mance results, and recommends uniform quantitative criteria
devices. The lack of uniformity in measurement procedures for satisfactory performance. The recommendations provided
among different manufacturers has introduced ambiguity in in this paper are a first step toward meeting a need perceived
the meaning of the system specifications. For example, dif- by practicing clinical medical physicists for quantitative
ferent manufacturers calibrate the response of the system to a guidelines to be used in conjunction with AAPM TG10 rec-
given exposure value using different beam qualities and re- ommended testing procedures.

TABLE III. Testing protocol and acceptance criteria for the dark noise test.

Agfa Fuji Kodak Lumisys

Exposure condition No exposures. Erase a single screen and read it without exposing it.

Screen processing System diagnostics/flat field, Test/sensitivity 共L⫽1兲, Pattern Standard

speed class⫽200 fixed EDR 共S⫽10 000兲

Image postprocessing None ‘‘Linear’’ ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 共GA⫽1.0, GT⫽A, RE⫽0.0兲 enhancement’’ settings,
Sensitometry⫽linear window⫽512,
level⫽exposure index

Measurements to be IgM, average pixel value 共PV兲 Avg. pixel value 共PV兲 Exposure index 共EI兲, average Average pixel value 共PV兲
made and its standard deviation 共PVSD兲, and its standard deviation pixel value 共PV兲, and its standard and standard deviation
and scan average level 共SAL兲 共PVSD兲 within 80% deviation 共PVSD兲 within 80% 共PVSD兲 within 80%
within 80% of the image of the image area of the image area of the image area

Qualitative criteria Uniform image without any artifacts Uniform without any artifacts except Uniform image
for acceptance for collector profile bands in the without any artifacts
screen-movement direction

Quantitative criteria IgM⬍0.28 PV⬍280a EIGP⬍80, EIHR⬍380 PV⬎3425

for acceptance SAL⬍130 PVSD⬍4 PVGP⬍80, PVHR⬍80 PVSD⬍4
PV⬍350 PVSD⬍4
PVSD⬍5
a
For those systems in which there is a direct relationship between PV and log共E兲. In the case of an inverse relationship, PV should be greater than 744.

Medical Physics, Vol. 28, No. 3, March 2001

363 Samei et al.: Performance evaluation 363

TABLE IV. Testing protocol and acceptance criteria for uniformity 共CR screen test兲.

Agfa Fuji Kodak Lumisys

Exposure condition This test is applied to all the screens. Visually inspect the screens for physical defects. Verify that the cassette label matches
the type of screen inside. Expose the screen to 10 mR (2.58⫻10⫺6 C/kg) a entrance exposure using 80 kVp, 0.5 mm Cu and
1 mm Al filtration, and 180 cm source-to-image distance 共SID兲. If significant heel effect is present, test can be performed with
two sequential half-exposures between which the orientation of the cassette is reversed.

Screen processing System diagnosis/flat field, Test/sensitivity 共L⫽1兲, Semi EDR Pattern Standard
speed class⫽200

Image postprocessing None, ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
Musica parameters⫽0.0 enhancement’’ settings,
Sensitometry⫽linear window⫽512,
level⫽exposure index

Measurements to Average pixel value 共PV兲 Average pixel value 共PV兲 Average pixel value 共PV兲 Average pixel value 共PV兲
be made and its standard deviation and its standard deviation and its standard deviation and its standard deviation
共PVSD兲 within 80% 共PVSD兲 within 80% 共PVSD兲 within 80% 共PVSD兲 within 80%
of the image area of the image area of the image area of the image area

Screen-to-screen variations: Screen-to-screen variations: Screen-to-screen variations: Screen-to-screen variations:

Standard deviation of IgM Standard deviation/mean Standard deviation Standard deviation
共LMSDs兲, and mean sensitivity 共SD/Ss兲 of exposure index of average PV
and standard deviation and standard deviation of among screens 共EISDs兲 among screens 共PVSDs兲
of PV among screens average PV among screens 共PVSDs兲
共PVs and PVSDs兲

Qualitative criteria Uniform image without any artifacts

for acceptance

Quantitative criteria PVSD⬍25 共single screen兲 PVSD⬍20 共single screen兲 PVSD⬍20 共single screen兲 PVSD⬍20 共single screen兲
for acceptance LMSDs⬍0.02 SD/Ss⬍5% EISDs⬍20 PVSDs⬍20
PVSDs⬍25 PVSDs⬍20
a
Throughout these tables, for convenience, all exposures are expressed in units of mR 共1 mR⫽2.58⫻10⫺7 C/kg兲.

II. METHODS AND RECOMMENDATIONS specific parameters, as reported in Tables III–XIII, using the
response relationships of the systems tabulated in Table XV.
As listed in Table I, CR devices in use at five different
institutions from four major CR manufacturers were evalu- None of the clinically acceptable systems tested in this col-
ated. The inventory of equipment used for testing is listed in laborative effort generated results beyond the established cri-
Table II. Each system was evaluated for dark noise, screen teria. In most instances, the acceptance criteria were at least
uniformity, exposure indicator calibration, linearity and au- 20% beyond the extremes of the evaluation results, a reason-
toranging response, laser beam function, limiting resolution, able margin considering that the evaluated systems were not
noise and low-contrast resolution, spatial accuracy, erasure operating at the borderline of clinical acceptability.
thoroughness, aliasing and grid response, and throughput.6 Several experimental precautions were observed in the
Special attention was paid to applying a uniform testing pro- evaluation of the systems. All the phosphor screens were
tocol for different CR systems, following the recommenda- cleaned and erased prior to executing the testing procedures.
tions of the AAPM TG10 as closely as practicable. The data Consistent delay times between 1 to 15 min were observed
from different institutions were collected and processed in a between exposing and reading the screens. Care was taken to
single database. Prior to or shortly after the evaluations, each reduce backscattered radiation by utilizing cross-table expo-
system’s performance was judged clinically acceptable by sures and significant interspace behind the screens. A large
attending radiologists based on image quality of clinical im- source-to-image distance 共SID⬃180 cm兲 was used to mini-
ages acquired with the system. Tables III–XIII tabulate the mize the heel effect. The ‘‘raw’’ signal values which were
testing protocol and the acceptance criteria derived from the proportional to the log of the incident exposure without any
results. For a full description of the tests and the rationale for postprocessing were used in the evaluations.
performing each test, the reader is advised to consult the All exposures were measured in a consistent fashion: The
AAPM TG10 report. collimators were set to expose the whole cassette with addi-
The quantitative acceptance criteria were established tional 7 cm margins on each side in the direction perpendicu-
based on the results of the tests performed on the clinical lar to the anode–cathode axis. The ion chamber was then
systems and a uniform level of tolerance in system response placed at the center of the beam at 2/3 of the SID. The
across different systems. Table XIV tabulates the response exposure was measured in five consecutive exposures and
tolerance levels based upon which the acceptance criteria the values averaged, E 1 . Keeping the ion chamber at 2/3
were established. These levels were translated to system- SID, the chamber was shifted on the central axis perpendicu-

Medical Physics, Vol. 28, No. 3, March 2001

364 Samei et al.: Performance evaluation 364

TABLE V. Testing protocol and acceptance criteria for exposure indicator calibration.

Agfa Fuji Kodak Lumisgysb

Recommended Use multiple screens 共at least three兲 of a given size/type. Expose the screens to approximately 1 mR (2.58⫻10⫺7 C/kg兲
exposure conditiona enhance exposure using 80 kVp and 0.5 mm Cu/1 mm Al filtration. Screens should be read with a precise 10 min delay.

Exposure condition Expose a screen to Expose a screen Expose a screen Expose a screen to approximately 8
共manufacturer approximately 1 mR to approximately to approximately mR (2.064⫻10⫺6 C/kg兲 entrance
specifieda兲 (2.58⫻10⫺7 C/kg兲 entrance 1 mR (2.58⫻10⫺7 C/kg兲 1 mR (2.58⫻10⫺7 C/kg兲 exposure using 80 kVp with 1 mm
exposure using 75 kVp entrance exposure entrance exposure Cu filtration. Screen should be read
and 1.5 mm Cu filtration. using 80 kVp without using 80 kVp promptly.
Screen should be read filtration. Screen should and 0.5 mm Cu/1 mm
promptly. be read with a Al filtration. Screen
precise 10 min delay. should be read with
a precise 15 min delay.

Screen processing System diagnosis/flat field, Test/sensitivity 共L⫽1兲, Pattern Standard

speed class⫽200 semi-EDR

Image postprocessing None, Irrelevant None

musica parameters⫽0.0

Measurements to be IgM and IgM normalized Sensitivity and sensitivity Exposure index 共EI兲 Mean pixel value 共PV兲 within 80%
made to exactly 1 mR exposure normalized to exactly and exposure index of the image area normalized to
to the screen (IgM1 mR) 1 mR exposure normalized to exactly 1 mR exactly 1 mR (PV1 mR)
using IgM1 mR⫽IgM⫺log(exposure), to the screen (S1 mR) exposure to the screen or 8 mR (PV8 mR) exposure
SAL and SAL normalized using S1 mR⫽S exposure (EI1 mR) using to the screen using
to exactly 1 mR exposure EI1 mR⫽EI⫺1000 PV1 mR⫽PV⫹1000 log 共exposure)
to the screen (SAL1 mR) ⫻log 共exposure兲 PV8 mR⫽PV⫹1000 log 共exposure/8)
using SAL1 mR⫽SAL/共exposure兲0.5

Qualitative criteria None

for acceptance

Quantitative criteria IgM1 mR⫺2.2⬍⫾0.045 single screen S1 mR⫺200⬍⫾20 EI1 mR⫺2000⬍⫾45 Pv8 mR⫺600⬍⫾45 single screen
for acceptance IgM1 mR⫺2.2⬍⫾0.023 for all single screen single screen PV1 mR⫺1505⬍⫾45 single screen
screens averaged S1 mR⫺200⬍⫾10 EI1 mR⫺2000⬍⫾23 PV1 mR⫺1505⬍⫾23 for all screens
SAL1 mR⫺1192⬍⫾60 single screen for all screens averaged for all screens averaged averaged
SAL1 mR⫺1192⬍⫾30
for all screens averaged
a
There is currently a strong consensus that CR systems should be calibrated with a standard filtered beam. Until such time as manufacturers change their
recommendations, the calibration procedure can be performed both with the manufacturer-defined technique, to verify conformance with the manufacturer’s
specifications, and with 0.5 mm Cu/1 mm Al filtration and 10 min delay time, for benchmarking and constancy checks.
b
The Lumisys ACR-2000 software did not make use of an exposure index at the time of testing. The system is calibrated to produce a pixel value of 600 in
response to an 8 mR (2.064⫻10⫺6 C/kg兲 exposure to the screen.

lar to the anode–cathode axis toward the edge of the field when the evaluation is quantitative and the results are com-
just outside the useful beam area 共the shadow of the ion pared against specific quantitative acceptance criteria. In this
chamber was still fully within the beam without projecting work, an attempt was made to outline a cross-platform uni-
over the cassette area兲. The exposure was measured in five form methodology based on the guidelines being developed
consecutive exposures again and the values were averaged, by the American Association of Physicists in Medicine Task
E 2 . The chamber was kept at the second location during the Group 10. Furthermore, a first attempt was made to recom-
tests for verification of the exposure values. The average mend quantitative acceptance criteria for satisfactory perfor-
exposure to the cassette in each single exposure was calcu- mance of a CR system based on the current state of practice.
lated as (E 1 /E 2 )(2/3) 2 共measured exposure兲.
The criteria were established using uniform tolerance levels
and test results acquired from CR systems in clinical use at
III. DISCUSSION five different institutions. The user specificity 共as opposed to
To achieve a consistent level of clinical performance from the conventional manufacturer specificity兲 of the acceptance
CR systems, acceptance testing procedures should be per- criteria suggested in this paper was necessitated by the de-
formed according to a uniform cross-platform methodology. sired uniformity of the testing procedures. The criteria, how-
As in any medical physics survey, the performance evalua- ever, do not guarantee optimal clinical performance, which
tion of a CR system is also more definitive and objective may not be ascertained without comprehensive clinical trials.

Medical Physics, Vol. 28, No. 3, March 2001

365 Samei et al.: Performance evaluation 365

TABLE VI. Testing protocol and acceptance criteria for linearity and autoranging response.a

Agfa Fuji Kodak Lumisys

Exposure condition Use a single screen 共multiple screens may also be used if the screen-to-screen variations in the previous test were found minimal兲.
Expose the screen to approximately 0.1, 1, and 10 mR (2.58⫻10⫺8 , 2.58⫻10⫺7 , 2.58⫻10⫺6 C/kg兲 entrance exposures
in a sequence of three exposure-reading cycles using 80 kVp, 0.5 mm Cu and 1 mm Al filtration, and 180 cm SID. Each time
read the screen with a consistent delay time.

Screen processing System diagnosis/flat field, Test/ave 4.0 Pattern Standard

speed class⫽200 Semi-EDR and fixed EDR⫽200
repeat also with
Test/contrast
semi-EDR and fixed EDR⫽200

Image postprocessing None, ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 enhancement’’ settings

Measurements to be IgM, average pixel value 共PV兲, For Semi EDR, correlation Exposure index 共EI兲 Mean pixel value 共PV兲
made and scan average level 共SAL兲 coefficient 共CC兲 of a linear fit and avg. pixel value 共PV兲 within 80% of
within 80% of the image area. to log共S兲 vs log 共E兲 plot. within 80% of the image area. the image area.
Slopes and correlation For fixed EDR, avg. pixel value 共PV兲 Slope and correlation Slope, intercept,
coefficients 共CCs兲 within 80% of the image area, slope coefficient 共CC兲 of a linear fit and correlation
of linear fits to log共SAL兲 and correlation coefficient 共CC兲 of a to EI vs log共E兲 and PV vs coefficient 共CC兲
vs log共E兲, PV vs log共E兲, linear fit to PV vs log共E兲 log 共E兲 plots of a linear fit
and IgM vs log共E兲 to P vs log共E兲

Qualitative criteria SAL vs exposure For semi-EDR, slope and correlation, The plot of EI and PV The plot of PV
for acceptance on a linear-log plot sensitivity vs exposure on a log–log vs exposure on a vs exposure on a
should result plot should result in a linear-log scale should linear-log scale
in a straight line straight line. result in straight should result in a
For fixed EDR, to PV vs exposure lines straight line
on a linear-log plot should result
in a straight line

Quantitative criteria SlopeIgM⫺1⬍⫾0.1 Slopes⫹1⬍⫾0.1 SlopeEI/1000⫺1⬍⫾0.1 Slopes/1000⫹1⬍⫾0.1

for acceptance SlopeSAL/0.5⫺0.1⬍⫾0.1 SlopePV/256⫺1⬍⫾0.1 共Ave 4兲b SlopePV/1000⫺0.1⬍⫾0.1 CCs⬎-0.95
SlopePV/1250⫺0.1⬍⫾0.1 SlopePV/511⫺1⬍⫾0.1 共Con.兲b CCs⬎0.95
CCs⬎0.95 CCs⬎0.95
a
If this test is performed with hard copy prints, the relationship between the pixel value 共PV兲 and optical density 共OD兲 should be established beforehand using
an electronic test pattern. The relationship between OD and PV should then be incorporated as a transformation in the quantitative analysis of the results.
b
Note that in some Fuji systems, there is an inverse relationship between PV and log共E兲. For those systems, the polarity of the slope in these equations should
be reversed.

TABLE VII. Testing protocol and acceptance criteria for the laser beam function.

Agfa Fuji Kodak Lumisys

Exposure condition Place a steel ruler roughly perpendicular to the laser-scan direction on a screen. Expose the screen to about 5 mR
(1.29⫻10⫺6 C/kg兲 entrance exposure using a 60 kVp beam without any filtration 共SID⫽180 cm兲. Examine the edges of the
ruler on the image for laser beam jitters using 10–20⫻ magnification.

Screen processing System diagnosis/flat field, Test/sensitivity Pattern Standard

speed class⫽200 Semi-EDR

Image postprocessing None, ‘‘Linear’’ ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 共GA⫽1.0, GT⫽A, RE⫽0.0兲 enhancement’’ settings,
sensitometry⫽linear window⫽512,
level⫽exposure index

Measurements to be made If any jitter is present, jitter dimension using workstation’s ‘‘measurement’’ or ROI tool.

Qualitative criteria Ruler edges should be straight and continuous without any under- or overshoot of the scan lines in light to dark transitions.
for acceptance

Quantitative criteria There should not be more than occasional ⫾1 jitters.

for acceptance

Medical Physics, Vol. 28, No. 3, March 2001

366 Samei et al.: Performance evaluation 366

TABLE VIII. Testing protocol and acceptance criteria for the limiting resolution and resolution uniformity.a

Agfa Fuji Kodak Lumisys

Exposure condition This test should be done for each type and size of the screens. Use a 60 kVp, unfiltered x-ray beam 共SID⫽180 cm兲. Place
three line-pair pattern devices on the cassette, two in orthogonal directions and one at 45°. Expose the screen with an
exposure of about 5 mR (1.20⫻10⫺6 C/kg兲. Also acquire an image of a fine wire mesh 共e.g., mammography screen–film
contact test tool兲 in contact with the cassette to examine the consistency of the resolution response across the image.

Screen processing System diagnosis/flat field, Test/sensitivity Pattern Standard

speed class⫽200 semi-EDR

Image postprocessing None, ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 enhancement’’ settings,
sensitometry⫽linear window⫽512,
level⫽exposure index

Measurements to be made Maximum discernible spatial frequencies in the three directions (R hor , R ver , R 45) using a magnified 共⬎10⫻兲, narrowly
windowed presentation of the images

Qualitative criteria The image of the wire mesh should be uniform without any blurring across the image
for acceptance

Quantitative criteria R hor / f Nyquist⬎0.9

for acceptance R ver / f Nyquist⬎0.9
R 45/1.41 f Nyquist⬎0.9

Note that the spatial resolution response of a CR system can be more comprehensively evaluated by measuring the modulation transfer function 共MTF兲 of the
a

system 共Refs. 7–9, 11–14兲.

TABLE IX. Testing protocol and acceptance criteria for noise and low-contrast resolution.a

Agfa Fuji Kodak Lumisys

Exposure condition This test should be done for each type and size of the screens. A low-contrast resolution pattern is used 共e.g., Leeds TO.12,
75 kVp beam with 1 mm of Cu filtration兲. For each screen type/size, acquire three images of the low-contrast phantom using
0.1, 1, and 10 mR (2.58⫻10⫺8 , 2.58⫻10⫺7 , 2.58⫻10⫺6 C/kg兲 exposures to the screens. Use a constant delay time of 10 min in
reading each of the screens.

Screen processing System diagnosis/flat field, Test/contrast Pattern Standard

speed class⫽200 Semi-EDR

Image postprocessing None, ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 enhancement’’ settings,
Sensitometry⫽linear window⫽512,
level⫽4096⫺EI 共for GP screens兲
or level⫽3796⫺EI
共for HR screens兲

Measurements Minimum discernible contrast for each object size 共contrast detail threshold兲, Standard deviation of pixel value 共PVSD兲
to be made within a fixed 共size and location兲 small region of the images, correlation coefficient 共CC兲 of the linear fit to log共PVSD兲 vs log共E兲.b

Qualitative criteria Contrast-detail threshold should be proportionately lower at Contrast-detail threshold Contrast-detail threshold
for acceptance higher exposures. should be proportionately should be proportionately
lower at higher exposures, lower at higher exposures.
with higher contrast thresholds
for standard-resolution screens.

Quantitative criteria CC⬎0.95b

for acceptance
a
Note that the noise response of a CR system can be more comprehensively evaluated by measuring the noise power spectrum 共NPS兲 and the detective
quantum efficiency 共DQE兲 of the system at different exposure levels 共Refs. 8 and 9, 11–14兲.
b
The quantitative evaluation is more valid with uniform images acquired for the linearity test 共Table VI兲 because of the absence of scattering material in the
beam. The expected quantitative response is based on the assumption of a logarithmic relationship between pixel value and exposure 共Table XV兲.

Medical Physics, Vol. 28, No. 3, March 2001

367 Samei et al.: Performance evaluation 367

TABLE X. Testing protocol and acceptance criteria for spatial accuracy.

Agfa Fuji Kodak Lumisys

⫺6
Exposure condition Place a regular wire-mesh screen–film contact test tool over cassette. Expose the cassette to about 5 mR (1.29⫻10 C/kg兲
entrance exposure using a 60 kVp beam without any filtration 共SID⫽180 cm兲. Repeat the acquisition with two steel rulers
in the vertical and the horizontal directions.

Screen processing System diagnosis/flat field, Test/contrast Pattern Standard

speed class⫽200 Semi-EDR

Image postprocessing None ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 enhancement’’ settings,
window⫽512, level⫽EI

Measurements to be Distances in the orthogonal directions 共15 cm minimum length兲 measured using the measurement tool of the workstation.a
made

Qualitative criteria Grid pattern spacing should be uniform without any distortion across the image.
for acceptance

Quantitative criteria Measured distance should be within 2% of the actual values.

acceptance
a
Alternatively, length measurements can be made on a hard-copy film printed in ‘‘true-size.’’

TABLE XI. Testing protocol and acceptance criteria for erasure thoroughness.

Agfa Fuji Kodak Lumisys

Exposure condition Place a thick lead block at the center of a 14⫻17 cassette and expose the screen to about 50 mR (1.29⫻10⫺5 C/kg兲 using a
60 kVp x-ray beam without any filtration 共SID⫽180 cm兲. Read the screen, and expose it a second time to 1 mR (2.58⫻10⫺7 C/kg兲
entrance exposure without the lead object using the same beam quality collimated in by about 5 cm on each side of the screen. For
a quantitative test re-read the screen after the second exposure without exposing it.

Screen processing System diagnosis/flat field, Test/sensitivity Pattern Standard

speed class⫽200 semi-EDR

Image postprocessing None, ‘‘Linear’’ ‘‘Raw data’’ and ‘‘No edge Window setting default
musica parameters⫽0.0 共GA⫽1.0, GT⫽A, RE⫽0.0兲 enhancement’’ settings, or equivalent
Sensitometry⫽linear Window setting default level⫽EI, window setting to 1 log共exposure兲 unit
Window setting default or equivalent to default or equivalent
or equivalent 1 log共exposure兲 unit to 1 log共exposure兲 unit
to 1 log共exposure兲 unit

Measurements to be IgM, average pixel value 共PV兲 Avg. pixel value 共PV兲 Exposure index 共EI兲, Average pixel value 共PV兲
made and its standard deviation 共PVSD兲, and its standard deviation average pixel Value 共PV兲, and standard deviation 共PVSD兲
and scan average level 共SAL兲 共PVSD兲 within 80% and its standard deviation within 80% of the
within 80% of the of the reread/unexposed 共PVSD兲 within 80% of the reread/unexposed image
reread/unexposed image image reread/unexposed image

Qualitative criteria Absence of a ghost image of the lead block from the first exposure in the reexposed image.a,b
for acceptance

Quantitative criteria IgM⫽0.28 PV⬍280c EIGP⬍80, EIHR⬍380 PV⬎3425

for acceptance SAL⬍130 PVSD⬍4 PVGP⬍80, PVHR⬍80 PVSD⬍4
PV⬍630 PVSD⬍4
PVSD⬍5
a
In our tests on the ACR-2000 system, the length of the standard erasure cycle was sufficient for exposures up to 32 mR (8.256⫻10⫺6 C/kg兲. Higher
exposures to the screen required an additional erasure cycle for complete screen erasure.
b
Note that erasure time in some systems 共e.g., Agfa兲 is configurable on an exam-by-exam basis.
c
For those systems in which there is an direct relationship between PV and log共E兲. In the case of inverse relationship, PV should be greater than 744.

Medical Physics, Vol. 28, No. 3, March 2001

368 Samei et al.: Performance evaluation 368

TABLE XII. Testing protocol and acceptance criteria for the aliasing/grid response.

Agfa Fuji Kodak Lumisys

Exposure condition This test should be performed for each type and size of screens that will be commonly used. Place the screen in a bucky that con-
tains an antiscatter grid so that the grid lines are parallel to the laser-scan direction. Alternatively, a grid may be placed directly on
the screen. Make sure the grid movement is disabled. Expose the screen to 1 mR (2.58⫻10⫺7 C/kg兲 using an 80 kVp beam filtered
with 0.5 mm Cu/1 mm Al filter and a SID according to the specification of the grid. Repeat, placing the screen perpendicular to the
laser-scan direction. Repeat the exposures with a moving grid.

Screen processing System diagnosis/flat field, Test/contrast Pattern Standard

Speed class⫽200 semi-EDR

Image postprocessing None, ‘‘Linear’’ 共GA⫽1.0, GT⫽A, RE⫽0.0兲 ‘‘Raw data’’ and ‘‘no edge None
musica parameters⫽0.0 A narrow window setting enhancement’’ settings,
sensitometry⫽linear level⫽EI, a narrow window setting
A narrow window setting

Measurements to be None
made

Qualitative criteria Moiré pattern should not be present when the grid lines are perpendicular to the laser-scan direction. For moving grids, no moiré
for acceptance pattern should be apparent when the screen is placed in either direction.a

Quantitative criteria None

for acceptance

Moiré patterns caused by display sampling 共not addressed in this protocol兲 can be distinguished by their changing behavior with changing the magnification
a

of the image on the soft-copy display device.

In light of this limitation, the recommended quantitative cri- the proposed quantitative test does not evaluate the spatial
teria should only be considered as helpful suggestions that characteristics of image noise. Ideally, the resolution and
require further clinical validation in the future. noise characteristics of a CR system should be more objec-
Another limitation of the current work is the fact that tively evaluated by measuring the frequency-dependent
many of the evaluation procedures were not fully quantita- modulation transfer function, the noise power spectrum, and
tive or can be influenced by the subjectivity of the examiner. the detective quantum efficiency of these systems. A number
The evaluations of limiting resolution and noise performance of investigators have been able to successfully and reproduc-
共Tables VIII and IX兲 are two important examples. The reso- ibly characterize the resolution and noise performance of CR
lution tests used do not evaluate the system transfer charac- systems using these indices,11–13 and more recently repro-
teristics but only establish that some modulation can be de- ducible measurements have been made in the field.7,14 How-
tected at the limiting frequency. The noise tests subjectively ever, a routine implementation of these measurements awaits
evaluate the contrast-detail characteristics of the system, and further standardization of measurement methods, and the de-

TABLE XIII. Testing protocol and acceptance criteria for the throughput.

Agfa Fuji Kodak Lumisys

⫺7 a
Exposure condition Expose 4 screens to 80 kVp, 2 mR (5.18⫻10 C/kg兲. Process the screens sequentially without delay.

Screen processing System diagnosis/flat field, Test/contrast Pattern Standard

speed class⫽200 semi-EDR

Image postprocessing musica parameters typical of those Irrelevant None

in clinical usage

Measurements to be Time interval 共t, in minutes兲 between putting the first screen in and the last image appearing on the CR viewing stationb
made Throughput 共screens/h兲⫽60⫻4/t

Qualitative criteria None

for acceptance

Quantitative criteria Throughput should be within 10% of the system’s specifications.

for acceptance
a
The test can be performed multiple times with different size cassettes.
b
Contribution of the network configuration is not considered.

Medical Physics, Vol. 28, No. 3, March 2001

369 Samei et al.: Performance evaluation 369

TABLE XIV. The CR response tolerance levels based upon which the uniform quantitative acceptance criteria were derived 共using the equations tabulated in
Table XV兲. All signal levels and standard deviations are expressed in terms of corresponding exposure 共E兲 values deduced from those quantities.

Characteristics Quantity of interest Acceptable tolerance

Dark noise Average signal and its standard deviation within 80% of the image area E⬍0.012 mR
(E⬍3.1⫻10⫺9 C/kg兲
␴ E /E⬍1%

Uniformity Signal standard deviation within 80% of the image area, and the standard ␴ E ⬍5%
deviation of the average screen signal among screens

Exposure calibration The exposure indicator response 共expressed in terms of exposure兲 to 1 mR E measured⫺1⬍⫾10%
(2.58⫻10⫺7 C/kg兲 entrance exposure

Linearity and autoranging The slope of the system response 共expressed in terms of logarithm of Slope ⫺1⬍⫾10%
exposure兲 vs logarithm of actual exposure Correlation coefficient ⬎0.95

Laser beam function Jitter dimension in pixels Occasional jitters

⬍⫾1 pixel

Limiting resolution Maximum discernible spatial frequencies of a high-contrast line-pair R hor / f Nyquist⬎0.9
pattern in two orthogonal and 45° angle directions R ver / f Nyquist⬎0.9
R 45 /1.41 f Nyquist⬎0.9

Noise and low-contrast A linear fit of system noise 共expressed in terms of logarithm of Correlation coefficient ⬎0.95
resolution corresponding ␴ E /E) to logarithm of actual exposure

Spatial accuracy The difference between the measured (d m ) and actual distances (d 0 ) in (d m ⫺d 0 )/d 0 ⬍2%
the orthogonal directions

Erasure thoroughness Average signal and its standard deviation within 80% of the reread/ E⬍0.012 mR
unexposed image (E⬍3.1⫻10⫺9 C/kg兲
␴ E /E⬍1%

Aliasing/grid response No quantitative tolerance levels

Throughput Measured throughput in screens per hours (T m ) and the specified (T 0 ⫺T m )/T 0 ⬍10%
throughput (T 0 )

velopment of automated commercial QC products. may vary as a function of radiographic technique factors, the
In this study, the exposures for quantitative measurements specific recipe of image processing parameters applied to the
were made with 0.5 mm copper and 1 mm additive alumi- images, and the type and calibration of the display media.
num filtration in the beam. The use of filtration was based on The default image processing parameters of the system for
prior studies10,15,16 indicating that the use of 0.5 mm Cu filter various anatomical sites and views 共e.g., chest PA, chest lat-
minimizes the dependency of the results on the kVp inaccu- eral, chest portable, knee, etc.兲 should be tested and custom-
racy and on the variations in the x-ray generator type, as the ized by the application specialists of the manufacturer with
filter attenuates the ‘‘soft’’ portion of the spectrum, predomi- assistance of the diagnostic medical physicist and under the
nantly responsible for tube-to-tube variations 共Fig. 1兲. The direction of the radiologist who is ultimately responsible for
use of this filtration also makes the spectrum a more accurate
the clinical acceptability of the images. Using radiographic
representative of primary x rays incident on the detector in
techniques provided by the manufacturer, images of various
clinical situations 共Fig. 2兲. The additional post-Cu, 1-mm-
anthropomorphic phantoms should be acquired with various
thick Al filter is used to attenuate any potential secondary
low-energy x rays generated in the Cu filter. The use of 0.5 combinations of collimation and positioning, utilizing the ap-
mm Cu/1 mm Al filtration, therefore, is advised for checking propriate prescribed anatomical menus of the system. In each
the consistency of the response in the acceptance testing and case, the proper processing of the image and the absence of
annual compliance inspections of CR systems. unexpected positioning and collimation errors should be
This paper outlines the steps for only the physical evalu- verified. Attending radiologists should be consulted for ac-
ation of CR systems. In a newly installed system, after ceptability of the image processing parameters for each ana-
completion of the physical acceptance testing and prior to a tomical menu. Since standard anthropomorphic phantoms
full clinical utilization, the system should also be evaluated have a limited ability to represent human anatomy and
for its clinical performance. The appearance of CR images patient-to-patient variations, the clinical evaluation and cus-

Medical Physics, Vol. 28, No. 3, March 2001

370 Samei et al.: Performance evaluation 370

TABLE XV. The relationship between exposure and pixel value/exposure indicator responses of various CR systems. The relationships which were provided
by the manufacturers or derived from their literature, were verified against experimental measurements at 80 kVp with 0.5 mm Cu/1 mm Al filtration. In these
relationships, PV is the pixel value, E is the exposure in mR, B is the speed class, and L is the latitude of the system.

Agfa Fuji Kodak Lumisys

Exposure indicator IgM and scan Sensitivity 共S兲 Expsoure index 共EI兲 None
quantities average level 共SAL兲

Exposure indicator SAL⫽90冑0.877cBE S⫽ 200/E EI⫽1000 log(E)⫹2000 None

relationship IgM⫽2log共SAL兲⫺3.9478
⫽log共cBE兲⫺0.0963
c⫽1.0 f or MD10 screens

Pixel value PV⫽2499 log共SAL兲⫺4933 PV⫽(1024/L) PV⫽1000 log(E)⫹c0 PV⫽1000 log共32/E兲

relationships ⫽1250 log共cBE兲⫺121a ⫻(log E⫹log(S/200)) c 0 ⫽2000 f or G P screens
c⫽1.0 for MD10 screens ⫹511b c 0 ⫽1700 f or HR screens

Exposure/reading 75 kVp and 1.5 mm Cu filtration, 80 kVp without filtration, 80 kVp and 0.5 mm Cu/1 mm Al 80 kVp with 1 mm Cu filtration,
condition no reading delay 10 min reading delay filtration, 15 min reading delay no reading delay
a
Using a 12 bit, linear log共E兲 data transfer from Agfa QC workstation.
b
Assuming a direct relationship between exposure and pixel value.

tomization of the image processing parameters should in- sorption characteristics and radiographic speed of CR and
clude actual clinical images. conventional screen–film radiography systems, an AEC cali-
Care should be taken that in the validation of the system brated for screen–film radiography is unlikely to be suitable
settings, all examinations performed at the facility are repre- for CR usage.18 For CR usage, the AEC can be calibrated
sented. The final customized image processing parameters using an approach similar to that for screen–film imaging
and system settings for different anatomical menus should be using the exposure indicator value of the system as the target
loaded into all units from the same manufacturer in place at variable to be controlled. The AEC should be adjusted to
the institution or associated medical facilities, where the result an exposure indicator value within a narrow acceptable
same exam may be performed on different machines, to as- range 共10%–15%兲 when the kVp or phantom thickness is
sure consistency of image presentations. They should also be varied within clinical operational limits. It may also be set to
documented in a list for future reference. provide a constant change in the exposure indicator value
Patient dose is one of the important implementation con-
siderations in the use of CR in a traditional film-based radi-
ology department.17 In screen–film radiography, film density
is a direct indicator of patient dose. In CR, however, because
of the dissociation of the detection and the display functions
of the imaging system, optical density can no longer be used
as an indicator of the patient dose. In reading a CR screen,
almost all CR systems provide an index that reflects the av-
erage exposure received by the screen during the image ac-
quisition 共Table XV兲. This exposure indicator can be used to
define and monitor patient exposures. Based on the manufac-
turer’s recommendations regarding the intrinsic speed of the
system and on the applicable standards of practice, the user
should establish, monitor, and enforce the acceptable range
of exposure indicator values for the clinical operation in the
facility. Note, however, that if a filtration other than that
suggested by the manufacturer is used for the exposure cali-
bration of the CR system, as suggested previously, the ac- FIG. 1. The relative variation in the response of a CR system 共signal per unit
cepted range of exposure indicator values should be derived exposure兲, where the energy of the beam is varied within 80 kVp⫾10%
range, as a function of Cu filtration in the beam for both single phase and
based on the comparative results of the two filtration condi- high-frequency/constant-potential generator x-ray systems 共12° anode angle,
tions. 2.6 mm intrinsic Al filtration兲. The data were generated by a computational
Automatic exposure control 共AEC兲 is the primary means model for simulation of the x-ray spectra, filter attenuation, and absorption
for controlling patient exposure in general radiography prac- characteristics of BaFBr0.85I0.15 :Eu phosphor screens 共98 mg/cm2 phosphor
coating weight兲. The model accuracy has been previously verified against
tice. For screen–film systems, the AEC is calibrated for con- experimental measurements 共Refs. 8, 10, 14兲. Note that Agfa CR systems
sistency in optical density resultant from varying exposure use a slightly different phosphor material (Ba0.86Sr0.14F1.1Br0.84I0.06) than the
techniques. Because of the dissimilarity between x-ray ab- one modeled here.

Medical Physics, Vol. 28, No. 3, March 2001

371 Samei et al.: Performance evaluation 371

facturers. The materials can be used as a handbook for ac-

ceptance testing and quality control inspection of CR sys-
tems to assure the consistency and reliability of their clinical
operation.
a兲
Electronic mail: [email protected]
1
R. Schaetzing, B. R. Whiting, A. R. Lubinsky, and J. F. Owen, ‘‘Digital
radiography using storage phosphors,’’ in Digital Imaging in Diagnostic
Radiology, edited by J. D. Newall and C. A. Kelsey 共Churchill Living
Stone, 1990兲, pp. 107–138.
2
M. Sonoda, M. Takano, J. Miyahara, and H. Kato, ‘‘Computed radiogra-
phy utilizing scanning laser stimulated luminescence,’’ Radiology 148,
833–838 共1983兲.
3
J. A. Seibert, ‘‘Photostimulable phosphor system acceptance testing,’’ in
Specification, Acceptance Testing and Quality Control of Diagnostic
X-ray Imaging Equipment, edited by J. A. Seibert, G. T. Barnes, and R.
G. Gould 共AIP, New York, 1994兲, pp. 771–800.
4
C. E. Willis, R. G. Leckie, J. Carter, M. P. Williamson, S. D. Scotti, and
G. Norton, ‘‘Objective measures of quality assurance in a computed
radiography-based radiology department,’’ SPIE Med. Imaging 2432,
588–599 共1995兲.
5
J. A. Seibert et al., ‘‘Acceptance testing and quality control of photo-
stimulable phosphor imaging systems,’’ Report of the American Associa-
tion of Physicists in Medicine 共AAPM兲 Task Group No. 10 共unpublished,
in the final review process兲.
6
A. R. Cowen, A. Workman, and J. S. Price, ‘‘Physical aspects of photo-
stimulable phosphor computed radiography,’’ Br. J. Radiol. 66, 332–345
共1993兲.
7
E. Samei, M. J. Flynn, and D. A. Reimann, ‘‘A method for measuring the
presampled MTF of digital radiographic systems using an edge test de-
vice,’’ Med. Phys. 25, 102–113 共1998兲.
8
E. Samei and M. J. Flynn, ‘‘Physical measures of image quality in pho-
tostimulable phosphor radiographic systems,’’ SPIE Med. Imaging 3032,
338 共1997兲.
9
J. T. Dobbins III, D. L. Ergun, L. Rutz, D. A. hinshaw, H. Blume, and D.
C. Clark, ‘‘DQE共f兲 of four generations of computed radiography acqui-
sition devices,’’ Med. Phys. 22, 1581–1593 共1995兲.
10
E. Samei, D. J. Peck, P. L. Rauch, E. Mah, and M. J. Flynn, ‘‘Exposure
FIG. 2. 共a兲 The model-calculated primary x-ray spectra emerging from a 0.5 calibration of computed radiography imaging systems 共abstract兲,’’ Med.
mm Cu filter and 24 cm tissue-equivalent material. The spectra were nor- Phys. 25, A155 共1995兲.
malized to have the same total area. b兲 The model-calculated equivalency of 11
C. D. Bradford, W. W. Peppler, and J. T. Dobbins III, ‘‘Performance
the CR signal per unit exposure for various Cu and tissue-equivalent mate- characteristics of a Kodak computed radiography system,’’ Med. Phys.
rial 共see Fig. 1 caption兲. 26, 27–37 共1999兲.
12
W. Hillen, U. Schiebel, and T. Zaengel, ‘‘Imaging performance of a
digital storage phosphor system,’’ Med. Phys. 14, 744–751 共1987兲.
13
when plus or minus density steps are applied. Because the C. E. Floyd, H. G. Chotas, J. T. Dobbins III, and C. E. Ravin, ‘‘Quanti-
CR exposure indicator is a quantity derived from analysis of tative radiographic imaging using a photostimulable phosphor system,’’
Med. Phys. 17, 454–459 共1990兲.
the image histogram, care must be exercised in the selection 14
M. J. Flynn and E. Samei, ‘‘Experimental comparison of noise and reso-
of phantoms and processing menus. The phantoms should lution for 2k and 4k storage phosphor radiography systems,’’ Med. Phys.
produce image histograms representative of clinical images, 26, 1612–1623 共1999兲.
15
not a very trivial requirement. Otherwise, inaccurate expo- C. E. Willis, J. C. Weiser, R. G. Leckie, J. Romlein, and G. Norton,
‘‘Optimization and quality control of computed radiography,’’ SPIE
sure indicator values may result, leading to faulty AEC cali- Med. Imaging 2164, 178–185 共1994兲.
bration. Further work on AEC calibration methodology for 16
D. M. Tucker and P. S. Rezentes, ‘‘The relationship between pixel value
CR is warranted. and beam quality in photostimulable phosphor imaging,’’ Med. Phys. 24,
887–893 共1997兲.
17
M. Freedman, E. Pe, S. K. Mun, S. C. B. La, and M. Nelson, ‘‘The
IV. CONCLUSIONS potential for unnecessary patient exposure from the use of storage phos-
phor imaging systems,’’ SPIE Med. Imaging 1897, 472–479 共1993兲.
The methods and acceptance criteria for the performance 18
C. E. Willis, ‘‘Computed Radiography: QA/QC,’’ in Practical Digital
evaluation of CR systems were presented in a comprehensive Imaging and PACS, Medical Physics Monograph No. 28 共Medical Phys-
tabular form for imaging systems from four major CR manu- ics Publishing, Madison, 1999兲, pp 157–175.

Medical Physics, Vol. 28, No. 3, March 2001