Sensitivity and specificity of alternative screening methods for systematic reviews using text mining tools.

Autor: Li J; Neurology Division, Centre Hospitalier de l'Université de Sherbrooke (CHUS), Sherbrooke, Canada; Centre de Recherche du Centre Hospitalier de l'Université de Montréal (CRCHUM), Montreal, Canada., Kabouji J; Department of Pharmacy, University of Laval, Quebec City, Canada., Bouhadoun S; Department of Neurology, McGill University, Montreal, Canada., Tanveer S; Department of Pharmaceutical Health Services Research, University of Maryland, Baltimore, MD, USA., Filion KB; Departments of Medicine and of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada; Centre for Clinical Epidemiology, Jewish General Hospital - Lady Davis Institute, Montreal, Canada., Gore G; Schulich Library of Physical Sciences, Life Sciences, and Engineering, McGill University, Montreal, Canada., Josephson CB; Department of Clinical Neurosciences, University of Calgary, Calgary, Canada; Department of Community Health Sciences, University of Calgary, Calgary, Canada; O'Brien Institute for Public Health, University of Calgary, Calgary, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Canada; Center for Health Informatics, University of Calgary, Calgary, Canada., Kwon CS; Department of Neurology, Epidemiology, Neurosurgery and the Gertrude H. Sergievsky Center, Columbia University, New York, NY, USA., Jette N; Department of Clinical Neurosciences, University of Calgary, Calgary, Canada; Department of Community Health Sciences, University of Calgary, Calgary, Canada., Bauer PR; Department of Psychosomatic Medicine and Psychotherapy, Faculty of Medicine, Medical Center - University of Freiburg, Freiburg, Germany., Day GS; Department of Neurology, Mayo Clinic Florida, Jacksonville, FL, USA., Subota A; Department of Community Health Sciences, University of Calgary, Calgary, Canada; Department of Medicine, University of Calgary, Calgary, Canada., Roberts JI; Department of Community Health Sciences, University of Calgary, Calgary, Canada., Lukmanji S; Department of Community Health Sciences, University of Calgary, Calgary, Canada., Sauro K; Department of Community Health Sciences, University of Calgary, Calgary, Canada; Department of Surgery, University of Calgary, Calgary, Canada; Department of Oncology & Arnie Charbonneau Cancer Institute, University of Calgary, Calgary, Canada., Ismaili AA; Department of Internal Medicine, McGill University, Montreal, Canada., Rahmani F; Faculty of Medicine, McGill University, Montreal, Canada., Chelabi K; Department of Pediatrics, McGill University, Montreal, Canada., Kerdougli Y; Department of Family Medicine, McGill University, Montreal, Canada., Seulami NM; Department of Emergency Medicine, McGill University, Montreal, Canada., Soumana A; Department of Family Medicine, McGill University, Montreal, Canada., Khalil S; Department of Family Medicine, McGill University, Montreal, Canada., Maynard N; Department of Internal Medicine, McGill University, Montreal, Canada., Keezer MR; Centre de Recherche du Centre Hospitalier de l'Université de Montréal (CRCHUM), Montreal, Canada; Departments of Medicine and of Epidemiology, Biostatistics, and Occupational Health, McGill University, Montreal, Canada; Department of Neurosciences, Université de Montréal, Montreal, Canada; School of Public Health, Université de Montréal, Montreal, Canada. Electronic address: mark.keezer@umontreal.ca.
Jazyk: angličtina
Zdroj: Journal of clinical epidemiology [J Clin Epidemiol] 2023 Oct; Vol. 162, pp. 72-80. Date of Electronic Publication: 2023 Jul 26.
DOI: 10.1016/j.jclinepi.2023.07.010
Abstrakt: Objectives: To evaluate the impact of text mining (TM) on the sensitivity and specificity of title and abstract screening strategies for systematic reviews (SRs).
Study Design and Setting: Twenty reviewers each evaluated a 500-citation set. We compared five screening methods: conventional double screen (CDS), single screen, double screen with TM, combined double screen and single screen with TM, and single screen with TM. Rayyan, Abstrackr, and SWIFT-Review were used for each TM method. The results of a published SR were used as the reference standard.
Results: The mean sensitivity and specificity achieved by CDS were 97.0% (95% confidence interval [CI]: 94.7, 99.3) and 95.0% (95% CI: 93.0, 97.1). When compared with single screen, CDS provided a greater sensitivity without a decrease in specificity. Rayyan, Abstrackr, and SWIFT-Review identified all relevant studies. Specificity was often higher for TM-assisted methods than that for CDS, although with mean differences of only one-to-two percentage points. For every 500 citations not requiring manual screening, 216 minutes (95% CI: 169, 264) could be saved.
Conclusion: TM-assisted screening methods resulted in similar sensitivity and modestly improved specificity as compared to CDS. The time saved with TM makes this a promising new tool for SR.
Competing Interests: Declaration of competing interest J.K., S.T., G.G., P.R.B., A Subota, S.L., K.S., A.A.I., F.R., K.C., Y.K., N.M.S., A Soumana, S.K., and N.M. report no conflicts of interest. J.L. is supported by the Fonds de Recherche Québec–Santé. K.B.F. is supported by a Senior salary support award from the Fonds de Recherche Québec–Santé and a William Dawson Scholar award from McGill University. He holds research grants from the Canadian Institutes of Health Research and has received honoraria from Quebec's Institut national d'excellence en santé et services sociaux and a stipend from the Canadian Network for Observational Drug Effect Studies. N.J. received grant funding paid to her institution for grants unrelated to this work from NINDS, NIH U24NS107201, NIH IU54NS100064, and NIH U24NS113849) during the study period and was the Bludhorn Professor of International Medicine at the Icahn School of Medicine at Mount Sinai. She receives an honorarium for her work as an Associate Editor of Epilepsia. GS Day's research is supported by NIH (K23AG064029, U01AG057195, U19AG032438), the Alzheimer's Association, and Chan Zuckerberg Initiative. He serves as a consultant for Parabon Nanolabs Inc., as a Topic Editor (Dementia) for DynaMed (EBSCO), and as the Clinical Director of the Anti-NMDA Receptor Encephalitis Foundation (Inc., Canada; uncompensated). He is the co-Project PI for a clinical trial in anti-NMDAR encephalitis, which receives support from Horizon Pharmaceuticals. He has developed educational materials for PeerView Media, Inc., and Continuing Education Inc. He owns stock in ANI pharmaceuticals. Dr. Day's institution has received support from Eli Lilly for Dr. Day's development and participation in an educational event promoting early diagnosis of symptomatic Alzheimer’s disease. J.I.R. reports fellowship salary support from Canadian Network of MS Clinics and travel support from the Rebecca Hotchkiss International Scholar Exchange. M.R.K. reports unrestricted educational grants from UCB, Eisai, and Jazz Pharmaceuticals, research grants for investigator-initiated studies from UCB and Eisai as well as from government entities (Canadian Institutes of Health Research, Fonds de Recherche Québec–Santé), academic institutions (Center Hospitalier de l’Université de Montréal), and foundations (TD Bank, TSC Alliance, Savoy Foundation, Quebec Bio-Imaging Network). M.R.K.’s salary is supported by the Fonds de Recherche Québec–Santé.
(Copyright © 2023 Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: