Beyond human in neurosurgical exams: ChatGPT's success in the Turkish neurosurgical society proficiency board exams.

Autor: Sahin MC; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey. Electronic address: mcaglarsahin@gazi.edu.tr., Sozer A; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey. Electronic address: alperen.sozer@gazi.edu.tr., Kuzucu P; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey. Electronic address: drpelinkuzucu@gmail.com., Turkmen T; Ministry of Health Dortyol State Hospital, Department of Neurosurgery, Hatay, Turkey. Electronic address: tlgturkmen@gmail.com., Sahin MB; Ministry of Health Etimesgut District Health Directorate, Department of Public Health, Ankara, Turkey. Electronic address: merve.buke@hacettepe.edu.tr., Sozer E; Gazi University, Directorate of Health Culture and Sports, Ankara, Turkey. Electronic address: ekin.aktas@gazi.edu.tr., Tufek OY; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey. Electronic address: ozanyavuztufek@gazi.edu.tr., Nernekli K; Stanford University Medical School, Department of Radiology, Stanford, CA, USA. Electronic address: kerem.nernekli@stanford.edu.tr., Emmez H; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey. Electronic address: hemmez@gazi.edu.tr., Celtikci E; Gazi University Faculty of Medicine, Department of Neurosurgery, Ankara, Turkey; Gazi University Artificial Intelligence Center, Ankara, Turkey. Electronic address: emrahceltikci@gazi.edu.tr.
Jazyk: angličtina
Zdroj: Computers in biology and medicine [Comput Biol Med] 2024 Feb; Vol. 169, pp. 107807. Date of Electronic Publication: 2023 Dec 10.
DOI: 10.1016/j.compbiomed.2023.107807
Abstrakt: Chat Generative Pre-Trained Transformer (ChatGPT) is a sophisticated natural language model that employs advanced deep learning techniques and is trained on extensive datasets to produce responses akin to human conversation for user inputs. In this study, ChatGPT's success in the Turkish Neurosurgical Society Proficiency Board Exams (TNSPBE) is compared to the actual candidates who took the exam, along with identifying the types of questions it answered incorrectly, assessing the quality of its responses, and evaluating its performance based on the difficulty level of the questions. Scores of all 260 candidates were recalculated according to the exams they took and included questions in those exams for ranking purposes of this study. The average score of the candidates for a total of 523 questions is 62.02 ± 0.61 compared to ChatGPT, which was 78.77. We have concluded that in addition to ChatGPT's higher response rate, there was also a correlation with the increase in clarity regardless of the difficulty level of the questions with Clarity 1.5, 2.0, 2.5, and 3.0. In the participants, however, there is no such increase in parallel with the increase in clarity.
Competing Interests: Declaration of competing interest This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. The authors of this submission declare no conflict of interest.
(Copyright © 2023 Elsevier Ltd. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: