 |
 |
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
3 February 2004 | Volume 140 Issue 3 | Pages 189-202
Background: Studies of diagnostic accuracy are subject to different sources of bias and variation than studies that evaluate the effectiveness of an intervention. Little is known about the effects of these sources of bias and variation.
Purpose: To summarize the evidence on factors that can lead to bias or variation in the results of diagnostic accuracy studies.
Data Sources: MEDLINE, EMBASE, and BIOSIS, and the methodologic databases of the Centre for Reviews and Dissemination and the Cochrane Collaboration. Methodologic experts in diagnostic tests were contacted.
Study Selection: Studies that investigated the effects of bias and variation on measures of test performance were eligible for inclusion, which was assessed by one reviewer and checked by a second reviewer. Discrepancies were resolved through discussion.
Data Extraction: Data extraction was conducted by one reviewer and checked by a second reviewer.
Data Synthesis: The best-documented effects of bias and variation were found for demographic features, disease prevalence and severity, partial verification bias, clinical review bias, and observer and instrument variation. For other sources, such as distorted selection of participants, absent or inappropriate reference standard, differential verification bias, and review bias, the amount of evidence was limited. Evidence was lacking for other features, including incorporation bias, treatment paradox, arbitrary choice of threshold value, and dropouts.
Conclusions: Many issues in the design and conduct of diagnostic accuracy studies can lead to bias or variation; however, the empirical evidence about the size and effect of these issues is limited.
Author and Article Information
From the University of York, York, United Kingdom, and the University of Amsterdam, Amsterdam, the Netherlands.
Disclaimer: The views expressed in this paper are those of the authors and not necessarily those of the Standing Group, the Commissioning Group, or the Department of Health.
Acknowledgments: The authors thank Kath Wright (Centre for Reviews and Dissemination) for carrying out literature searches. They also thank the advisory panel to the review for their help during various stages, including commenting on the protocol and draft report.
Grant Support: Commissioned and funded by the National Health Service R&D Health Technology Assessment Programme (project number 98/27/99).
Potential Financial Conflicts of Interest: None disclosed.
Requests for Single Reprints: Penny Whiting, MSc, Centre for Reviews and Dissemination, University of York, York YO10 5DD, United Kingdom; e-mail, pfw2{at}york.ac.uk.
Current Author Addresses: Ms. Whiting and Dr. Kleijnen: Centre for Reviews and Dissemination, University of York, York YO10 5DD, United Kingdom.
Ms. Rutjes and Drs. Reitsma, Glas, and Bossuyt: Department of Clinical Epidemiology & Biostatistics, Academic Medical Center, University of Amsterdam, P.O. Box 22700, 1100 DE Amsterdam, the Netherlands. ACADEMIA AND CLINIC
Sources of Variation and Bias in Studies of Diagnostic Accuracy
A Systematic Review
This article has been cited by other articles:
|
|
 |
|
  M. M.G. Leeflang, J. J. Deeks, C. Gatsonis, P. M.M. Bossuyt, and on behalf of the Cochrane Diagnostic Test Accuracy Systematic Reviews of Diagnostic Test Accuracy Ann Intern Med, December 16, 2008; 149(12): 889 - 897. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. P. Whitlock, J. S. Lin, E. Liles, T. L. Beil, and R. Fu Screening for Colorectal Cancer: A Targeted, Updated Systematic Review for the U.S. Preventive Services Task Force Ann Intern Med, November 4, 2008; 149(9): 638 - 658. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Gaffikin, J. McGrath, M. Arbyn, and P. D Blumenthal Avoiding verification bias in screening test evaluation in resource poor settings: a case study from Zimbabwe Clinical Trials, October 1, 2008; 5(5): 496 - 503. [Abstract] [PDF]
|
|
|
|
 |
|
 T Kaltenbach, S Friedland, and R Soetikno A randomised tandem colonoscopy trial of narrow band imaging versus white light examination to compare neoplasia miss rates Gut, October 1, 2008; 57(10): 1406 - 1412. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. H Willis Spectrum bias--why clinicians need to be cautious when applying diagnostic test studies Fam. Pract., October 1, 2008; 25(5): 390 - 396. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. M. M. Bossuyt STARD Statement: Still Room for Improvement in the Reporting of Diagnostic Accuracy Studies Radiology, September 1, 2008; 248(3): 713 - 714. [Full Text] [PDF]
|
|
|
|
|
|
N. L. Wilczynski Quality of Reporting of Diagnostic Accuracy Studies: No Change Since STARD Statement Publication--Before-and-after Study Radiology, September 1, 2008; 248(3): 817 - 823. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Hachamovitch and M. F. Di Carli Methods and Limitations of Assessing New Noninvasive Tests: Part I: Anatomy-Based Validation of Noninvasive Testing Circulation, May 20, 2008; 117(20): 2684 - 2690. [Full Text] [PDF]
|
|
|
|
|
|
D. Toll, R Oudega, Y Vergouwe, K. Moons, and A. Hoes A new diagnostic rule for deep vein thrombosis: safety and efficiency in clinically relevant subgroups Fam. Pract., February 1, 2008; 25(1): 3 - 8. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. Doubrovsky, R. A. Scolyer, R. Murali, P. R. McKenzie, G. F. Watson, C. S. Lee, D. J. McLeod, W. H. McCarthy, R. F. Uren, J. R. Stretch, et al. Diagnostic Accuracy of Fine Needle Biopsy for Metastatic Melanoma and Its Implications for Patient Management Ann. Surg. Oncol., January 1, 2008; 15(1): 323 - 332. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Terasawa, T. Nihashi, T. Hotta, and H. Nagai 18F-FDG PET for Posttherapy Assessment of Hodgkin's Disease and Aggressive Non-Hodgkin's Lymphoma: A Systematic Review J. Nucl. Med., January 1, 2008; 49(1): 13 - 21. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Cortellazzi, L. Minati, C. Falcone, M. Lamperti, and D. Caldiroli Predictive value of the El-Ganzouri multivariate risk index for difficult tracheal intubation: a comparison of Glidescope(R) videolaryngoscopy and conventional Macintosh laryngoscopy Br. J. Anaesth., December 1, 2007; 99(6): 906 - 911. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. Virgili, F. Menchini, A. F. Dimastrogiovanni, E. Rapizzi, U. Menchini, F. Bandello, and R. G. Chiodini Optical Coherence Tomography versus Stereoscopic Fundus Photography or Biomicroscopy for Diagnosing Diabetic Macular Edema: A Systematic Review Invest. Ophthalmol. Vis. Sci., November 1, 2007; 48(11): 4963 - 4973. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Pewsner, P. Juni, M. Egger, M. Battaglia, J. Sundstrom, and L. M Bachmann Accuracy of electrocardiography in diagnosis of left ventricular hypertrophy in arterial hypertension: systematic review BMJ, October 6, 2007; 335(7622): 711 - 711. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Poynard, P. Halfon, L. Castera, M. Munteanu, F. Imbert-Bismut, V. Ratziu, Y. Benhamou, M. Bourliere, V. de Ledinghen, and FibroPaca Group Standardization of ROC Curve Areas for Diagnostic Evaluation of Liver Fibrosis Markers Based on Prevalences of Fibrosis Stages Clin. Chem., September 1, 2007; 53(9): 1615 - 1622. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S Brealey and M Westwood Are you reading what we are reading? The effect of who interprets medical images on estimates of diagnostic test accuracy in systematic reviews Br. J. Radiol., August 1, 2007; 80(956): 674 - 677. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Holsinger, J. Deveau, M. Boustani, and J. W. Williams Jr Does This Patient Have Dementia? JAMA, June 6, 2007; 297(21): 2391 - 2404. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S.F.P.J. Coppus, B.C. Opmeer, S. Logan, F. van der Veen, S. Bhattacharya, and B.W.J. Mol The predictive value of medical history taking and Chlamydia IgG ELISA antibody testing (CAT) in the selection of subfertile women for diagnostic laparoscopy: a clinical prediction model approach Hum. Reprod., May 1, 2007; 22(5): 1353 - 1358. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. M. Harbord, J. J. Deeks, M. Egger, P. Whiting, and J. A. C. Sterne A unification of models for meta-analysis of diagnostic accuracy studies Biostat., April 1, 2007; 8(2): 239 - 251. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Leeflang, J. Reitsma, R. Scholten, A. Rutjes, M. Di Nisio, J. Deeks, and P. Bossuyt Impact of Adjustment for Quality on Results of Metaanalyses of Diagnostic Accuracy Clin. Chem., February 1, 2007; 53(2): 164 - 172. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. A. Katz, T. P. Aufderheide, M. Bogner, P. R. Rahko, R. L. Brown, L. M. Brown, M. E. Prekker, and H. P. Selker The Impact of Unstable Angina Guidelines in the Triage of Emergency Department Patients with Possible Acute Coronary Syndrome Med Decis Making, November 1, 2006; 26(6): 606 - 616. [Abstract] [PDF]
|
|
|
|
 |
|
 G Peat, E Thomas, R Duncan, L Wood, E Hay, and P Croft Clinical classification criteria for knee osteoarthritis: performance in the general population and primary care Ann Rheum Dis, October 1, 2006; 65(10): 1363 - 1367. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Dewey, F. Teige, D. Schnapauff, M. Laule, A. C. Borges, K.-D. Wernecke, T. Schink, G. Baumann, W. Rutsch, P. Rogalla, et al. Noninvasive detection of coronary artery stenoses with multislice computed tomography or magnetic resonance imaging. Ann Intern Med, September 19, 2006; 145(6): 407 - 415. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. Smidt, A.W.S. Rutjes, D. A.W.M. van der Windt, R. W.J.G. Ostelo, P. M. Bossuyt, J. B. Reitsma, L. M. Bouter, and H. C.W. de Vet The quality of diagnostic accuracy studies since the STARD statement: has it improved? Neurology, September 12, 2006; 67(5): 792 - 797. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Mallett, J. J Deeks, S. Halligan, S. Hopewell, V. Cornelius, and D. G Altman Systematic reviews of diagnostic tests in cancer: review of methods and reporting BMJ, August 26, 2006; 333(7565): 413. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. L. Wang, B. H. Avashia, and E. L. Petsonk Interpreting Periodic Lung Function Tests in Individuals: The Relationship Between 1- to 5-Year and Long-term FEV1 Changes. Chest, August 1, 2006; 130(2): 493 - 499. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Gatsonis and P. Paliwal Meta-analysis of diagnostic and screening test accuracy evaluations: methodologic primer. Am. J. Roentgenol., August 1, 2006; 187(2): 271 - 281. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Whiting, R. Harbord, C. Main, J. J Deeks, G. Filippini, M. Egger, and J. A C Sterne Accuracy of magnetic resonance imaging for the diagnosis of multiple sclerosis: systematic review BMJ, April 15, 2006; 332(7546): 875 - 884. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. T. Sica Bias in Research Studies Radiology, March 1, 2006; 238(3): 780 - 789. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. W.S. Rutjes, J. B. Reitsma, M. Di Nisio, N. Smidt, J. C. van Rijn, and P. M.M. Bossuyt Evidence of bias and variation in diagnostic accuracy studies. Can. Med. Assoc. J., February 14, 2006; 174(4): 469 - 476. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. A. Furukawa and G. H. Guyatt Sources of bias in diagnostic accuracy studies and the diagnostic process. Can. Med. Assoc. J., February 14, 2006; 174(4): 481 - 482. [Full Text] [PDF]
|
|
|
|
 |
|
 J-E C Holty, W G Kuschner, and M K Gould Accuracy of transbronchial needle aspiration for mediastinal staging of non-small cell lung cancer: a meta-analysis Thorax, November 1, 2005; 60(11): 949 - 955. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. M. Mitchell, M. D. Brown, I. B.A. Menown, and J. A. Kline Novel Protein Markers of Acute Coronary Syndrome Complications in Low-Risk Outpatients: A Systematic Review of Potential Use in the Emergency Department Clin. Chem., November 1, 2005; 51(11): 2005 - 2012. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. W.S. Rutjes, J. B. Reitsma, J. P. Vandenbroucke, A. S. Glas, and P. M.M. Bossuyt Case-Control and Two-Gate Designs in Diagnostic Accuracy Studies Clin. Chem., August 1, 2005; 51(8): 1335 - 1341. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Soletormos, A. Semjonow, P. E.C. Sibley, R. Lamerz, P. H. Petersen, W. Albrecht, P. Bialk, M. Gion, F. Junker, H.-P. Schmid, et al. Biological Variation of Total Prostate-Specific Antigen: A Survey of Published Estimates and Consequences for Clinical Practice Clin. Chem., August 1, 2005; 51(8): 1342 - 1351. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Stengel, K. Bauwens, G. Rademacher, S. Mutze, and A. Ekkernkamp Association between Compliance with Methodological Standards of Diagnostic Research and Reported Test Accuracy: Meta-Analysis of Focused Assessment of US for Trauma Radiology, July 1, 2005; 236(1): 102 - 111. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Tatsioni, D. A. Zarin, N. Aronson, D. J. Samson, C. R. Flamm, C. Schmid, and J. Lau Challenges in Systematic Reviews of Diagnostic Technologies Ann Intern Med, June 21, 2005; 142(12_Part_2): 1048 - 1055. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. F Woolacott, M. A Puhan, J. Steurer, and J. Kleijnen Ultrasonography in screening for developmental dysplasia of the hip in newborns: systematic review BMJ, June 18, 2005; 330(7505): 1413. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. H. Wang, J. Q. Huang, G. F. Zheng, W. M. Wong, S. K. Lam, J. Karlberg, H. H. X. Xia, R. Fass, and B. C. Y. Wong Is Proton Pump Inhibitor Testing an Effective Approach to Diagnose Gastroesophageal Reflux Disease in Patients With Noncardiac Chest Pain?: A Meta-analysis Arch Intern Med, June 13, 2005; 165(11): 1222 - 1228. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. R Richter and M. F Reinking How does evidence on the diagnostic accuracy of the vertebral artery test influence teaching of the test in a professional physical therapist education program? Physical Therapy, June 1, 2005; 85(6): 589 - 599. [Full Text] [PDF]
|
|
|
|
 |
|
 J. I. Westbrook, E. W. Coiera, and A. S. Gosling Do Online Information Retrieval Systems Help Experienced Clinicians Answer Clinical Questions? J. Am. Med. Inform. Assoc., May 1, 2005; 12(3): 315 - 321. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. P. Mulhall, G. R. Veerappan, and J. L. Jackson Meta-Analysis: Computed Tomographic Colonography Ann Intern Med, April 19, 2005; 142(8): 635 - 650. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Swarr and R. Keren Comparison of Alternative Diagnostic Approaches for Managing Appendicitis in Children: The Effect of Disease Prevalence and Spectrum Pediatrics, August 1, 2004; 114(2): 513 - 514. [Full Text] [PDF]
|
|
|
|
|
|
D. Pewsner, M. Battaglia, C. Minder, A. Marx, H. C Bucher, and M. Egger Ruling a diagnosis in or out with "SpPIn" and "SnNOut": a note of caution BMJ, July 24, 2004; 329(7459): 209 - 213. [Full Text] [PDF]
|
|
Article
