Poor performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction.

Autor: van Nuland M; Department of Clinical Pharmacy, Tergooi Medical Center, Laan van Tergooi 2, 1212 VG, Hilversum, The Netherlands. mvannuland@tergooi.nl., Snoep JD; Department of Nephrology, Tergooi Medical Center, Hilversum, The Netherlands., Egberts T; Department of Clinical Pharmacy, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.; Division of Pharmacoepidemiology and Clinical Pharmacology, Department of Pharmaceutical Sciences, Faculty of Science, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands., Erdogan A; Department of Clinical Pharmacy, Tergooi Medical Center, Laan van Tergooi 2, 1212 VG, Hilversum, The Netherlands., Wassink R; Department of Clinical Pharmacy, Tergooi Medical Center, Laan van Tergooi 2, 1212 VG, Hilversum, The Netherlands., van der Linden PD; Department of Clinical Pharmacy, Tergooi Medical Center, Laan van Tergooi 2, 1212 VG, Hilversum, The Netherlands.
Jazyk: angličtina
Zdroj: European journal of clinical pharmacology [Eur J Clin Pharmacol] 2024 Aug; Vol. 80 (8), pp. 1133-1140. Date of Electronic Publication: 2024 Apr 09.
DOI: 10.1007/s00228-024-03687-5
Abstrakt: Purpose: Clinical decision support systems (CDSS) are used to identify drugs with potential need for dose modification in patients with renal impairment. ChatGPT holds the potential to be integrated in the electronic health record (EHR) system to give such dosing advices. In this study, we aim to evaluate the performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal impairment.
Methods: This cross-sectional study was performed at Tergooi Medical Center, the Netherlands. CDSS alerts regarding renal dysfunction were collected from the electronic health record (EHR) during a 2-week period and were presented to ChatGPT and an expert panel. Alerts were presented with and without patient variables. To evaluate the performance, suggested medication interventions were compared.
Results: In total, 172 CDDS alerts were generated for 80 patients. Indecisive responses by ChatGPT to alerts were excluded. For alerts presented without patient variables, ChatGPT provided "correct and identical" responses to 19.9%, "correct and different" responses to 26.7%, and "incorrect responses to 53.4% of the alerts. For alerts including patient variables, ChatGPT provided "correct and identical" responses to 16.7%, "correct and different" responses to 16.0%, and "incorrect responses to 67.3% of the alerts. Accuracy was better for newer drugs such as direct oral anticoagulants.
Conclusion: The performance of ChatGPT in clinical rule-guided dose interventions in hospitalized patients with renal dysfunction was poor. Based on these results, we conclude that ChatGPT, in its current state, is not appropriate for automatic integration into our EHR to handle CDSS alerts related to renal dysfunction.
(© 2024. The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.)
Databáze: MEDLINE
Externí odkaz: