Automated title and abstract screening for scoping reviews using the GPT-4 Large Language Model

Autor:	Wilkins, David
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language
Druh dokumentu:	Working Paper
Popis:	Scoping reviews, a type of literature review, require intensive human effort to screen large numbers of scholarly sources for their relevance to the review objectives. This manuscript introduces GPTscreenR, a package for the R statistical programming language that uses the GPT-4 Large Language Model (LLM) to automatically screen sources. The package makes use of the chain-of-thought technique with the goal of maximising performance on complex screening tasks. In validation against consensus human reviewer decisions, GPTscreenR performed similarly to an alternative zero-shot technique, with a sensitivity of 71%, specificity of 89%, and overall accuracy of 84%. Neither method achieved perfect accuracy nor human levels of intraobserver agreement. GPTscreenR demonstrates the potential for LLMs to support scholarly work and provides a user-friendly software framework that can be integrated into existing review processes. Comment: 12 pages, 4 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.07918 Zobrazit plný text záznamu View this record from Arxiv