Engineering of Generative Artificial Intelligence and Natural Language Processing Models to Accurately Identify Arrhythmia Recurrence.

Autor:	Feng R; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Brennan KA; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Azizi Z; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Goyal J; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Deb B; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.).; School of Information Science, University of California, Berkeley (B.D., S.M.N.)., Chang HJ; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Ganesan P; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Clopton P; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Pedron M; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Ruipe Rez-Campillo S; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.).; Department of Computer Science, ETH Zurich, Switzerland (S.R.-C.)., Desai Y; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., De Larochellière H; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Baykaner T; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Perez M; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Rodrigo M; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.).; CoMMLab, Universitat Politècnica de València, Spain (M.R.)., Rogers AJ; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.)., Narayan SM; Department of Medicine, Stanford University, CA (R.F., K.A.B., Z.A., J.G., B.D., H.J.C., P.G., P.C., M. Pedron, S.R.-C., Y.D., H.D.L., T.B., M. Perez, M.R., A.J.R., S.M.N.).; School of Information Science, University of California, Berkeley (B.D., S.M.N.).
Jazyk:	angličtina
Zdroj:	Circulation. Arrhythmia and electrophysiology [Circ Arrhythm Electrophysiol] 2024 Dec 16, pp. e013023. Date of Electronic Publication: 2024 Dec 16.
DOI:	10.1161/CIRCEP.124.013023
Abstrakt:	Background: Large language models (LLMs), such as ChatGPT, excel at interpreting unstructured data from public sources, yet are limited when responding to queries on private repositories, such as electronic health records (EHRs). We hypothesized that prompt engineering could enhance the accuracy of LLMs for interpreting EHR data without requiring domain knowledge, thus expanding their utility for patients and personalized diagnostics. Methods: We designed and systematically tested prompt engineering techniques to improve the ability of LLMs to interpret EHRs for nuanced diagnostic questions, referenced to a panel of medical experts. In 490 full-text EHR notes from 125 patients with prior life-threatening heart rhythm disorders, we asked GPT-4-turbo to identify recurrent arrhythmias distinct from prior events and tested 220 563 queries. To provide context, results were compared with rule-based natural language processing and BERT-based language models. Experiments were repeated for 2 additional LLMs. Results: In an independent hold-out set of 389 notes, GPT-4-turbo had a balanced accuracy of 64.3%±4.7% out-of-the-box at baseline. This increased when asking GPT-4-turbo to provide a rationale for its answers, requiring a structured data output, and providing in-context exemplars, rose to a balanced accuracy of 91.4%±3.8% ( P <0.05). This surpassed the traditional logic-based natural language processing and BERT-based models ( P <0.05). Results were consistent for GPT-3.5-turbo and Jurassic-2 LLMs. Conclusions: The use of prompt engineering strategies enables LLMs to identify clinical end points from EHRs with an accuracy that surpassed natural language processing and approximated experts, yet without the need for expert knowledge. These approaches could be applied to LLM queries for other domains, to facilitate automated analysis of nuanced data sets with high accuracy by nonexperts.
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu