Does using artificial intelligence take the person out of personal statements? We can't tell.

Autor: Whitrock JN; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH. Electronic address: whitroja@ucmail.uc.edu., Pratt CG; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Carter MM; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Chae RC; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Price AD; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Justiniano CF; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH; Division of Colon and Rectal Surgery, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Van Haren RM; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH; Division of Cardiothoracic Surgery, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Silski LS; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH; Division of Transplantation, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Quillin RC 3rd; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH; Division of Transplantation, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH., Shah SA; Cincinnati Research in Outcomes and Safety in Surgery (CROSS) Research Group, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH; Division of Transplantation, Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH.
Jazyk: angličtina
Zdroj: Surgery [Surgery] 2024 Dec; Vol. 176 (6), pp. 1610-1616. Date of Electronic Publication: 2024 Sep 19.
DOI: 10.1016/j.surg.2024.08.018
Abstrakt: Background: Use of artificial intelligence to generate personal statements for residency is currently not permitted but is difficult to monitor. This study sought to evaluate the ability of surgical residency application reviewers to identify artificial intelligence-generated personal statements and to understand perceptions of this practice.
Methods: Three personal statements were generated using ChatGPT, and 3 were written by medical students who previously matched into surgery residency. Blinded participants at a single institution were instructed to read all personal statements and identify which were generated by artificial intelligence; they then completed a survey exploring their opinions regarding artificial intelligence use.
Results: Of the 30 participants, 50% were faculty (n = 15) and 50% were residents (n = 15). Overall, experience ranged from 0 to 20 years (median, 2 years; interquartile range, 1-6.25 years). Artificial intelligence-derived personal statements were identified correctly only 59% of the time, with 3 (10%) participants identifying all the artificial intelligence-derived personal statements correctly. Artificial intelligence-generated personal statements were labeled as the best 60% of the time and the worst 43.3% of the time. When asked whether artificial intelligence use should be allowed in personal statements writing, 66.7% (n = 20) said no and 30% (n = 9) said yes. When asked if the use of artificial intelligence would impact their opinion of an applicant, 80% (n = 24) said yes, and 20% (n = 6) said no. When survey questions and ability to identify artificial intelligence-generated personal statements were evaluated by faculty/resident status and experience, no differences were noted (P > .05).
Conclusion: This study shows that surgical faculty and residents cannot reliably identify artificial intelligence-generated personal statements and that concerns exist regarding the impact of artificial intelligence on the application process.
Competing Interests: Conflict of Interest/Disclosure The authors have no relevant financial disclosures.
(Copyright © 2024 Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: