DR. SASMITA KU.

BISOYI
SENIOR RESIDENT
DEPT. COMMUNITY MEDICINE
SLN. MEDICAL COLLEGE, KORAPUT
 HISTORY
 INTRODUCTION
 TESTS CLASSIFICATION

2
  1st used during World War II
 Following the attack on Pearl Harbor in 1941, the
United States Army began new research to increase
the prediction of correctly detected Japanese
Aircrafts from their radar signals.
 Now used in Medicine, Radiology, Biometrics, etc.

3
  Making a diagnosis is an important role for
clinicians.
 Often a range of tests are used to supplement
history-taking and examination.
 However, not all diagnostic tests are equal.
 Hence, it is important to evaluate the diagnostic
abilities of those tests.

4
 TEST REPORT

CATEGORICAL CONTINUOUS

POSITIVE/ NEGATIVE/ RANGE OF

PRESENT ABSENT VALUES

5
 CATEGORICAL TEST CONTINUOUS TEST
REPORT REPORT

 ACCURACY “ROC” CURVE

 SENSITIVITY
 SPECIFICITY
 PREDICTIVE VALUES : +/-
 LIKELIHOOD RATIOS : +/-

6
 DISEASE
TEST RESULT
PRESENT (+) ABSENT (-)
POSITIVE TRUE POSITIVE FALSE POSITIVE
(+) (TP) (a) (FP) (b)
NEGATIVE FALSE NEGATIVE TRUE NEGATIVE
(-) (FN) (c) (TN) (d)

7
(TP + TN) / (TP + FP + FN + TN)
(Entire Study Population)
= (a + d) / (a + b + c + d)

8
 a.k.a. TRUE POSITIVE FRACTION
 Probability that a test result will be positive when
the disease is present
 TP / (TP + FN) = a / (a + c)

9
 a.k.a. TRUE NEGATIVE FRACTION
 Probability that a test result will be negative when
the disease is not present
 TN / (TN + FP) = d / (b + d)

10
 • FP / (FP + TN) = b / (b + d)
= 1 - [d / (b + d)] = 1 - Specificity

11
 POSITIVE PREDICTIVE NEGATIVE PREDICTIVE
VALUE (PPV) VALUE (NPV)

 Probability that the  Probability that the

disease is present when disease is not present
the test result is positive when the test result is
negative
 TP / (TP + FP)  TN / (TN + FN)
= a / (a + b) = d / (c + d)
12
 POSITIVE LIKELIHOOD NEGATIVE LIKELIHOOD
RATIO RATIO
 Ratio between the  Ratio between the
probability of a positive test probability of a negative
result given the presence of test result given
the disease and the the presence of the disease
probability of a positive test and the probability of a
result given the absence of negative test result given
the disease the absence of the disease
 TP Rate / FP Rate  FN Rate / TN Rate
 Sensitivity/(1-Specificity)  (1-Sensitivity)/Specificity
13
 Diagnostic tests that produce continuous results are
often dichotomised because the outcome of interest
is generally binary.
 One of the challenges in interpreting the results of
these tests is the selection of the threshold to
distinguish a ‘positive’ test from a ‘negative’ test.
 The role of Receiver Operating Characteristic
(ROC) Curve is in choosing the threshold cut-offs.
14
 The Sensitivities and Specificities for different
values of a continuous test measure are first
tabulated.
 Then, the graphical ROC curve is produced by
plotting Sensitivity (True Positive Rate) on the y-axis
against 1 – Specificity (False Positive Rate) on the x-
axis for the various values tabulated.

15
16
 FEV1 in Pneumoconiosis
 How good is FEV1 as an indicator of
Pneumoconiosis?
 What should be the cut-off value of FEV1 for the
diagnosis of Pneumoconiosis?

17
 Men with Pneumoconiosis (n=27)
40, 43, 47, 49, 50, 50, 53, 57, 58, 58, 58, 62, 65, 69, 71,
73, 74, 75, 75, 77, 78, 79, 80, 87, 90, 100, 105
 Men without Pneumoconiosis (n=13)
60, 67, 73, 75, 79, 80, 83, 87, 89, 100, 105, 109, 115

18
 PNEUMOCONIOSIS
FEV1
PRESENT ABSENT TOTAL
<80% OF
22 5 27
NORMAL
≥80% OF
5 8 13
NORMAL
TOTAL 27 13 40
SENSITIVITY = 22/27 = 81%; SPECIFICITY = 8/13 = 62%
19
 Forced Expiratory Volume at 1st Sec
100

80
Sensitivity

60

40

20
AUC = 0.792
P < 0.001
0
0 20 40 60 80 100
100-Specificity 20
21
22
23
 Global measure of the ability of a test to
discriminate whether a specific condition is
present or not.
 > AUC, Better the test…

24
 Sensitivity 1
Specificity 1
1 - Specificity 0

Sensitivity
25
1 - Specificity
 Sensitivity 1
Specificity 0
1 - Specificity 1

Sensitivity
26
1 - Specificity
 Sensitivity 0
Specificity 1
1 - Specificity 0

Sensitivity
27
1 - Specificity
Sensitivity

1 - Specificity
28
 Sensitivity 0.75
Specificity 0.75
1 - Specificity 0.25

Sensitivity
29
1 - Specificity
 Sensitivity 1
Specificity 0
1 - Specificity 1

Sensitivity
30
1 - Specificity
 Sensitivity 0
Specificity 1
1 - Specificity 0

Sensitivity
31
1 - Specificity
Sensitivity

1 - Specificity
32
 Sensitivity 0.5
Specificity 0.5
1 - Specificity 0.5

Sensitivity
33
1 - Specificity
 Sensitivity 1
Specificity 0
1 - Specificity 1

Sensitivity
34
1 - Specificity
 Sensitivity 0
Specificity 1
1 - Specificity 0

Sensitivity
35
1 - Specificity
Sensitivity

1 - Specificity
36
 Forced Expiratory Volume at 1st Sec
100

80
Sensitivity

60

40

20
AUC = 0.792
P < 0.001
0
0 20 40 60 80 100
100-Specificity 37
 That test value should be considered as the cut-off
value for diagnosis, which has High Sensitivity as
well as High Specificity;
 In other words, High Sensitivity (y-coordinate) as
well as Low (1 - Specificity) (x-coordinate);
 That is, which has the Highest (y – x) value…

38
 And (y – x) = Sensitivity – (1 – Specificity)
= Sensitivity + Specificity – 1 = YOUDEN’S INDEX
 Graphically, the cut-off value is the point on the
ROC Curve that is farthest towards the Upper Left
Corner from the Diagonal Chance Line…

39
 Forced Expiratory Volume at 1st Sec
100

80
Sensitivity

60

40

20
AUC = 0.792
P < 0.001
0
0 20 40 60 80 100
100-Specificity 40
 Sensitivity + Specificity – 1
 The threshold, for which the Youden’s Index is
highest, is the cut-off value for diagnosis.
 For the example, the Youden’s Index (0.4701) is
highest for the threshold value of ≤ 78 with a
sensitivity of 77.78% and specificity of 69.23%.
 So, those men having an FEV1 of ≤ 78 should be
diagnosed as having Pneumoconiosis.

41
42
43