Machine learning to help researchers evaluate biases in clinical trials: a prospective, randomized user study

Autor:	Randolph G. Bias, Joël Kuiper, Frank Soboczenski, Thomas A Trikalinos, Iain J. Marshall, Byron C. Wallace
Jazyk:	angličtina
Rok vydání:	2019
Předmět:	020205 medical informatics MEDLINE Health Informatics 02 engineering and technology Machine learning computer.software_genre lcsh:Computer applications to medicine. Medical informatics Risk Assessment law.invention Task (project management) Feedback Machine Learning 03 medical and health sciences 0302 clinical medicine Randomized controlled trial Bias law 0202 electrical engineering electronic engineering information engineering Humans 030212 general & internal medicine Prospective Studies Randomized Controlled Trials as Topic business.industry Health Policy System usability scale Usability 3. Good health Computer Science Applications Clinical trial Systematic review lcsh:R858-859.7 Artificial intelligence Risk assessment business Psychology computer Research Article
Zdroj:	BMC Medical Informatics and Decision Making, Vol 19, Iss 1, Pp 1-12 (2019) BMC Medical Informatics and Decision Making
ISSN:	1472-6947
Popis:	Objective Assessing risks of bias in randomized controlled trials (RCTs) is an important but laborious task when conducting systematic reviews. RobotReviewer (RR), an open-source machine learning (ML) system, semi-automates bias assessments. We conducted a user study of RobotReviewer, evaluating time saved and usability of the tool. Materials and methods Systematic reviewers applied the Cochrane Risk of Bias tool to four randomly selected RCT articles. Reviewers judged: whether an RCT was at low, or high/unclear risk of bias for each bias domain in the Cochrane tool (Version 1); and highlighted article text justifying their decision. For a random two of the four articles, the process was semi-automated: users were provided with ML-suggested bias judgments and text highlights. Participants could amend the suggestions if necessary. We measured time taken for the task, ML suggestions, usability via the System Usability Scale (SUS) and collected qualitative feedback. Results For 41 volunteers, semi-automation was quicker than manual assessment (mean 755 vs. 824 s; relative time 0.75, 95% CI 0.62–0.92). Reviewers accepted 301/328 (91%) of the ML Risk of Bias (RoB) judgments, and 202/328 (62%) of text highlights without change. Overall, ML suggested text highlights had a recall of 0.90 (SD 0.14) and precision of 0.87 (SD 0.21) with respect to the users’ final versions. Reviewers assigned the system a mean 77.7 SUS score, corresponding to a rating between “good” and “excellent”. Conclusions Semi-automation (where humans validate machine learning suggestions) can improve the efficiency of evidence synthesis. Our system was rated highly usable, and expedited bias assessment of RCTs.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_dedup___::bb7f15bd7a05a7f3e42c497809f68cbc http://link.springer.com/article/10.1186/s12911-019-0814-z Zobrazit plný text záznamu Plný text ve formátu PDF Plný text ve formátu HTML
Nepřihlášeným uživatelům se plný text nezobrazuje	K zobrazení výsledku je třeba se přihlásit.