How to critically appraise a journal article on accuracy of a diagnostic test. This presentation spans issues regarding directness, validity, applicability and individualization. Also included are how to process information on sensitivity, specificity, likelihood ratios, predictive values and decision thresholds

1. Critical Appraisal of a
Diagnostic Article
MARC EVANS M. ABAT, MD, FPCP, FPCGM
Internal Medicine-Geriatric Medicine
Head, Center for Healthy Aging and Section of Geriatrics
Department of Medicine, The Medical City
Clinical Associate Professor, Division of Adult Medicine, Department of Medicine, PGH
Faculty, Asia-Pacific Evidence-Based Healthcare

2. Clinical Scenario
• A 65 year old male, with diabetes and hypertension, presented at the
ER with non-productive cough, fever of 38.5°C, headache and loss of
smell, of 6 days duration.
• Due to the low alert status of the metropolis, he is able to go out and
dine out in restaurants
• Does not recall exposure to any other person with signs and
symptoms of COVID-19
• On day of consult, had subjective complaint of shortness of breath
when walking <10 meters, hence consultation

3. • On PE
• BP 130/80, HR 102, RR 25, T 38.5°C, O2 sat 93% at room air
• (+)bibasal crackles
• No other pertinent findings
• CBG 158 mg/dl
• At the ER, he was triaged to the infectious area for further
assessment
• Both COVID-19 RAT and RTPCR were done on the patient

4. What is your personal guess of the
probability that this patient has
COVID-19??

5. Clinical Question
Among adult patients with signs and symptoms of
possible COVID-19, how accurate is the COVID-19 rapid
antigen test in diagnosing COVID-19?

6. Clinical Question
P
Adult patients with signs and symptoms of possible
COVID-19
I
COVID-19 rapid antigen test
C
COVID-19 RTPCR
O
Diagnosis of COVID-19

7. Article
Wertenauer C, Brenner Michael G, Dressel A, Pfeifer C, Hauser U,
Wieland E, et al. Diagnostic performance of rapid antigen testing for
SARS-COV-2: The COVid-19 antigen (COVAG) study [Internet]. Frontiers
in medicine. Frontiers Media S.A.; 2022 [cited 2022Apr15]. Available
from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8979030/

8. Directness
Clinical Question Research Question
P Adult patients with signs
and symptoms of possible
COVID-19
Adults 18 years and older
I COVID-19 rapid antigen test COVID-19 rapid antigen test
(Roche and Abbott)
C COVID-19 RTPCR COVID-19 RTPCR
O Diagnosis of COVID-19 Diagnosis of COVID-19

9. Validity Criteria
Was the reference standard an acceptable one?
Yes. RTPCR. (page 2)

10. Was “definition” of the index test and the
reference standard independent?
Yes. Reference and index tests were
different and independent from each other
(page 2)

11. Was “performance” of the index test and the
reference standard independent?
Yes. The swabs for BOTH RT-PCR and rapid tests
were done concomitantly and independently
(page 2)

12. Was “interpretation” of the index test and the
reference standard independent?
Yes. Processing of reference and index
tests were done in different locations
(page 3)

13. Probability Odds

14. P = 1% O = 1/99
P = 6% O = 6/94
P = 13.4% O = 13.4/86.6
O = 11/1 P = 11/12
O = 17.3/1 P = 17.3/18.3
O = 1/1000 P = 1/1001

15. DISEASE
(+) (-)
(-)
(+)
TEST
a b
c d
P of (+) test in pts. w/ ds. =
P of (- ) test in pts. w/o ds =
P of (+) ds in pts w (+) test =
P of (-) ds in pts w ( -) test =
a/(a+c) =
a/(a+b) =
d/(c+d) =
Sensitivity
Specificity
Pos pred val
Neg pred val
d/(b+d) =

16. DISEASE
(+) (-)
(-)
(+)
TEST
a b
c d
e f
(±)
Overall odds of disease = (a+c+e)/(b+d+f)
Odds if (+)=
Odds if (±)=
Odds if (-) =
a/b
c/d
e/f
LR = odds of disease given a test result
overall odds of disease

17. Results
What likelihood ratios were associated with
the range of possible test results?

18. N=2215 Roche: Sn 60.4%, Sp 99.7%
(+)COVID-19 via RTPCR=338 Abbott: Sn 56.8%, Sp 99.9%
Roche RAT
(+)disease (-)disease LR(+) LR(-)
(+)test 204 6 189
(-)test 134 1871 0.4
338 1877
Abbott RAT
(+)disease (-)disease LR(+) LR(-)
(+)test 192 2 533
(-)test 146 1875 0.4
338 1877

19. 1. Estimate the pre-test probability of the disease
you are suspecting.
2. Convert this to odds.
3. Multiply by the likelihood ratio of the test result
4. Convert this back to probability
30% or 30/100
30:70 or 3:7
3:7 x 189 = 567:7
567/574 = 98.7%
POSITIVE ROCHE TEST

20. NEGATIVE ROCHE TEST
1. Estimate the pre-test probability of the disease
you are suspecting.
2. Convert this to odds.
3. Multiply by the likelihood ratio of the test result
4. Convert this back to probability
30% or 30/100
30:70 or 3:7
3:7 x 0.4 = 1.2:7
1.2/8.2 = 14.6%

21. 0% 50% 100%

22. 0% 50% 100%
DIAGNOSTIC
THRESHOLD
TREATMENT
THRESHOLD
TESTING RANGE
STOP TESTS,
REASSURE
STOP TESTS,
TREAT
DISEASE SEVERITY
TREATMENT COST
TREATMENT SAFETY
DISEASE SEVERITY
TEST COST
TEST SAFETY

23. Applicability Criteria
• Biologic issues? SCRAP
• Sex
• Comorbidiy
• Race
• Age
• Pathology
• Socioeconomic issues?

24. Individualizing the result
• Apply the process to your patient
During a COVID-19 surge in the Philippines:
Positivity rate via RTPCR 25-30%
Theoretical number of ER suspect consults = 100
patients
Abbott RAT: Sn 56.8%, Sp 99.9%
Abbott RAT
(+)dse (-)dse PPV NPV LR(+) LR(-)
(+)test 14 0 100% 42
(-)test 11 75 87% 0.44
25 75
Abbott RAT
(+)dse (-)dse PPV NPV LR(+) LR(-)
(+)test 1 0 100% 49
(-)test 1 98 99% 0.5
2 98
Present situation in the Philippines:
Positivity rate via RTPCR 2%
Theoretical number of ER suspect consults = 100
patients
Abbott RAT: Sn 56.8%, Sp 99.9%

26. Thank you for listening!