Scribd
Upload
85 views

Model For The Computation of Cutoff For "In House " or Modified Tests

The clinical decision point in qualitative tests is defined as the test threshold that differentiates positive from negative results , referred to as the cutoff

Uploaded by

Paulo Pereira
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
85 views

Model For The Computation of Cutoff For "In House " or Modified Tests

The clinical decision point in qualitative tests is defined as the test threshold that differentiates positive from negative results , referred to as the cutoff

Uploaded by

Paulo Pereira
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

P06 - Model for the computation of cutoff

for “in-house” or modified tests


12.ª Reunião Científica da Sociedade Portuguesa de Medicina Laboratorial
29 a 31 de Outubro de 2020

Paulo Pereira, Ph.D.


Introduction
Introduction
• The clinical decision point in qualitative tests is defined as the
test threshold that differentiates positive from negative
results, referred to as the “cutoff”
• Its selection is required for “in-house” or modified tests
• A modification should only occur in special and justified
situations
• A sensitivity-specificity tradeoff derived from the cutoff choice
usually happens, i.e., an increase in sensitivity is accompanied by
a reduction in specificity and vice versa

3 of 36
Introduction
• A “best” cutoff is associated with a “best” tradeoff for a claimed
performance - “better” condition sensitivity or “better” condition
specificity.

4 of 36
Objectives
Objectives
• The presentation briefly introduces and discusses an approach to
identify “the best” cutoff point based on the “the best”
sensitivity-specificity tradeoff

6 of 36
Material and methods
Material and methods
• The receiver operating characteristic (ROC) curve1 design is a
fundamental methodology for the identification of a cutoff that
fits the purpose of the test
• The overall efficiency is related to the area under the ROC curve
(AUC), but it should be clear that this area can not be
misunderstood with clinical sensitivity or specificity, and it has
very limited interest to define “the best” cutoff2
• We will assume a virology immunoassay for the screening of
human T-lymphotropic virus types I and II (HTLV I/II) in a
hospital medical laboratory
8 of 36
Material and methods
• Numerical results are expressed in absorbance (wavelength of
450 ± 5 nm)
• 47 infected samples and 179 non-infected samples are used
• A sensitivity of 100% and specificity ≥ 90% are claimed (better
sensitivity)

9 of 36
Material and methods
Table 1. Percentiles, absorbance, true and false positives and negatives
no. np A TP FN TN FP N
717 0.716 0.107 47 0 162 17 226
(…) (…) (…) (…) (…) (…) (…) (…)
795 0.794 0.402 47 0 179 0 226
Table 2. Clinical sensitivity and specificity, and 95% confidence intervals
by “score confidence interval” method for [0.107 A, 0.402 A]
A se LL HL sp LL HL
0.107 100.0% 92.4% 100.0% 90.5% 85.3% 94.0%
(…) (…) (…) (…) (…) (…) (…)
0.402 100.0% 92.4% 100.0% 100.0% 97.9% 100.0%
10 of 36
Material and methods
Figure 1. ROC, AUC  [0.999, 1.000)]
100%
90%
80%
70%
Sensitivity

60%
50%
40%
30%
20%
10%
0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100%
100%-Specificity

11 of 36
Material and methods
Figure 2. Se and sp versus criterion Figure 3. Efficiency and Youden index
100% 1,00
90% 0,90
80% 0,80
70% 0,70
60% 0,60
50% 0,50
40% 0,40
30% 0,30
20% 0,20
10% 0,10
0% 0,00
0,000 0,500 1,000 1,500 2,000 2,500 0 1 1 2 2 3
Criterion value Criterion value
Sensitivity SeLLof95%CI
Efficiency Youden Index
SeHLof95%CI Specificity
12 of 36
Discussion and conclusions
Discussion and conclusions
• The computation was done using spreadsheet software
• The results indicate that the test meets the claimed
requirements (see Tables and Figures), suggesting a large
number of possible cutoff points
• So, what is “the best” point?
• What fits the objective of the test is 0.107 A, since, theoretically,
it is least likely to be affected by a lack of sensitivity as it is the
lowest value
• Note that if the focus is a better specificity, “the best”
discriminator should be 0.206 A
14 of 36
Discussion and conclusions
• ROC curve design is critical to a reliable estimate of “the best”
cutoff point since selection is based on “the best” sensitivity-
specificity tradeoff
• Although the ROC curve concept is not systematically used in the
medical laboratory, it should be encouraged

15 of 36
References
1. Lusted LB. Introduction to medical decision making. Springfield
(Il): Charles C. Thomas, 1968.
2. Poiesz BJ, et al. Detection and isolation of type C retrovirus
particles from fresh and cultured lymphocytes of a patient with
cutaneous t-cell lymphoma. Proc Natl Acad Sci USA 1980;
77(12):7415-7419.
3. Chapter 6, Computation of the cutoff for “in-house” and modified
tests. In: Pereira P. Quality control of qualitative tests for medical
laboratories. Lisbon, 2019.

16 of 36
P06 - Model for the computation of cutoff
for “in-house” or modified tests
12.ª Reunião Científica da Sociedade Portuguesa de Medicina Laboratorial
29 a 31 de Outubro de 2020

Paulo Pereira, Ph.D., Sandra Xavier, Ph.D.

You might also like