Evaluating the accuracy of ChatGPT-4 in predicting ASA scores: A prospective multicentric study ChatGPT-4 in ASA score prediction.

Autor: Turan Eİ; Department of Anesthesiology, Istanbul Health Science University Kanuni Sultan Süleyman Education and Training Hospital, Istanbul, Turkey. Electronic address: enginihsan@hotmail.com., Baydemir AE; Department of Anesthesiology, Basaksehir Cam ve Sakura City Hospital, Istanbul, Turkey., Özcan FG; Department of Anesthesiology, Basaksehir Cam ve Sakura City Hospital, Istanbul, Turkey., Şahin AS; Department of Anesthesiology, Istanbul Health Science University Kanuni Sultan Süleyman Education and Training Hospital, Istanbul, Turkey.
Jazyk: angličtina
Zdroj: Journal of clinical anesthesia [J Clin Anesth] 2024 Sep; Vol. 96, pp. 111475. Date of Electronic Publication: 2024 Apr 23.
DOI: 10.1016/j.jclinane.2024.111475
Abstrakt: Background: This study investigates the potential of ChatGPT-4, developed by OpenAI, in enhancing medical decision-making processes, particularly in preoperative assessments using the American Society of Anesthesiologists (ASA) scoring system. The ASA score, a critical tool in evaluating patients' health status and anesthesia risks before surgery, categorizes patients from I to VI based on their overall health and risk factors. Despite its widespread use, determining accurate ASA scores remains a subjective process that may benefit from AI-supported assessments. This research aims to evaluate ChatGPT-4's capability to predict ASA scores accurately compared to expert anesthesiologists' assessments.
Methods: In this prospective multicentric study, ethical board approval was obtained, and the study was registered with clinicaltrials.gov (NCT06321445). We included 2851 patients from anesthesiology outpatient clinics, spanning neonates to all age groups and genders, with ASA scores between I-IV. Exclusion criteria were set for ASA V and VI scores, emergency operations, and insufficient information for ASA score determination. Data on patients' demographics, health conditions, and ASA scores by anesthesiologists were collected and anonymized. ChatGPT-4 was then tasked with assigning ASA scores based on the standardized patient data.
Results: Our results indicate a high level of concordance between ChatGPT-4 predictions and anesthesiologists' evaluations, with Cohen's kappa analysis showing a kappa value of 0.858 (p = 0.000). While the model demonstrated over 90% accuracy in predicting ASA scores I to III, it showed a notable variance in ASA IV scores, suggesting a potential limitation in assessing patients with more complex health conditions.
Discussion: The findings suggest that ChatGPT-4 can significantly contribute to the medical field by supporting anesthesiologists in preoperative assessments. This study not only demonstrates ChatGPT-4's efficacy in medical data analysis and decision-making but also opens new avenues for AI applications in healthcare, particularly in enhancing patient safety and optimizing surgical outcomes. Further research is needed to refine AI models for complex case assessments and integrate them seamlessly into clinical workflows.
Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
(Copyright © 2024 Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: