ChatGPT-4 Surpasses Residents: A Study of Artificial Intelligence Competency in Plastic Surgery In-service Examinations and Its Advancements from ChatGPT-3.5.

Autor:	Hubany SS; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Scala FD; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Hashemi K; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Kapoor S; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Fedorova JR; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Vaccaro MJ; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Ridout RP; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Hedman CC; From the University of Central Florida College of Medicine, Orlando, Fla.; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Kellogg BC; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla., Leto Barone AA; Division of Craniofacial and Pediatric Plastic Surgery, Nemours Children's Hospital, Orlando, Fla.
Jazyk:	angličtina
Zdroj:	Plastic and reconstructive surgery. Global open [Plast Reconstr Surg Glob Open] 2024 Sep 05; Vol. 12 (9), pp. e6136. Date of Electronic Publication: 2024 Sep 05 (Print Publication: 2024).
DOI:	10.1097/GOX.0000000000006136
Abstrakt:	Background: ChatGPT, launched in 2022 and updated to Generative Pre-trained Transformer 4 (GPT-4) in 2023, is a large language model trained on extensive data, including medical information. This study compares ChatGPT's performance on Plastic Surgery In-Service Examinations with medical residents nationally as well as its earlier version, ChatGPT-3.5. Methods: This study reviewed 1500 questions from the Plastic Surgery In-service Examinations from 2018 to 2023. After excluding image-based, unscored, and inconclusive questions, 1292 were analyzed. The question stem and each multiple-choice answer was inputted verbatim into ChatGPT-4. Results: ChatGPT-4 correctly answered 961 (74.4%) of the included questions. Best performance by section was in core surgical principles (79.1% correct) and lowest in craniomaxillofacial (69.1%). ChatGPT-4 ranked between the 61st and 97th percentiles compared with all residents. Comparatively, ChatGPT-4 significantly outperformed ChatGPT-3.5 in 2018-2022 examinations ( P < 0.001). Although ChatGPT-3.5 averaged 55.5% correctness, ChatGPT-4 averaged 74%, a mean difference of 18.54%. In 2021, ChatGPT-3.5 ranked in the 23rd percentile of all residents, whereas ChatGPT-4 ranked in the 97th percentile. ChatGPT-4 outperformed 80.7% of residents on average and scored above the 97th percentile among first-year residents. Its performance was comparable with sixth-year integrated residents, ranking in the 55.7th percentile, on average. These results show significant improvements in ChatGPT-4's application of medical knowledge within six months of ChatGPT-3.5's release. Conclusion: This study reveals ChatGPT-4's rapid developments, advancing from a first-year medical resident's level to surpassing independent residents and matching a sixth-year resident's proficiency. Competing Interests: Dr. Leto Barone is the founder and chief medical officer of ReconstratA, LLC, and the Founder and President of Reconstruct Together, Corp. All the other authors have no financial interest to declare in relation to the content of this article. This study was supported by The University of Central Florida College of Medicine through funds designated for the publication costs. (Copyright © 2024 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of The American Society of Plastic Surgeons.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu