Concordance Between Expert and Nonexpert Ratings of Condensed Video-Based Trainee Operative Performance Assessment.

Autor: Scully RE; Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts., Deal SB; Department of Surgery, Virginia Mason Medical Center, Seattle, Washington., Clark MJ; Consulting for Statistics, Computing, and Analytics, University of Michigan, Ann Arbor, Michigan., Yang K; Consulting for Statistics, Computing, and Analytics, University of Michigan, Ann Arbor, Michigan., Wnuk G; Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan., Smink DS; Department of Surgery, Brigham and Women's Hospital, Boston, Massachusetts., Fryer JP; Department of Surgery, Northwestern Memorial Hospital, Chicago, Illinois., Bohnen JD; Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts., Teitelbaum EN; Department of Surgery, Northwestern Memorial Hospital, Chicago, Illinois., Meyerson SL; Department of Surgery, University of Kentucky Medical Center, Lexington, Kentucky., Meier AH; Department of Surgery, SUNY Upstate University Hospital, Syracuse, New York., Gauger PG; Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan., Reddy RM; Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan., Kendrick DE; University Hospitals Case Western Reserve, Cleveland Ohio; Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan., Stern M; Department of Surgery, University of Michigan, Ann Arbor, Michigan., Hughes DT; Department of Surgery, University of Michigan, Ann Arbor, Michigan., Chipman JG; Department of Surgery, University of Minnesota, Minneapolis, Minnesota., Patel JA; Department of Surgery, University of Kentucky Medical Center, Lexington, Kentucky., Alseidi A; Department of Surgery, Virginia Mason Medical Center, Seattle, Washington., George BC; Center for Surgical Training and Research, Department of Surgery, University of Michigan, Ann Arbor, Michigan.
Jazyk: angličtina
Zdroj: Journal of surgical education [J Surg Educ] 2020 May - Jun; Vol. 77 (3), pp. 627-634. Date of Electronic Publication: 2020 Mar 20.
DOI: 10.1016/j.jsurg.2019.12.016
Abstrakt: Objective: We examined the impact of video editing and rater expertise in surgical resident evaluation on operative performance ratings of surgical trainees.
Design: Randomized independent review of intraoperative video.
Setting: Operative video was captured at a single, tertiary hospital in Boston, MA.
Participants: Six common general surgery procedures were video recorded of 6 attending-trainee dyads. Full-length and condensed versions (n = 12 videos) were then reviewed by 13 independent surgeon raters (5 evaluation experts, 8 nonexperts) using a crossed design. Trainee performance was rated using the Operative Performance Rating Scale, System for Improving and Measuring Procedural Learning (SIMPL) Performance scale, the Zwisch scale, and ten Cate scale. These ratings were then standardized before being compared using Bayesian mixed models with raters and videos treated as random effects.
Results: Editing had no effect on the Operative Performance Rating Scale Overall Performance (-0.10, p = 0.30), SIMPL Performance (0.13, p = 0.71), Zwisch (-0.12, p = 0.27), and ten Cate scale (-0.13, p = 0.29). Additionally, rater expertise (evaluation expert vs. nonexpert) had no effect on the same scales (-0.16 (p = 0.32), 0.18 (p = 0.74), 0.25 (p = 0.81), and 0.25 (p = 0.17).
Conclusions: There is little difference in operative performance assessment scores when raters use condensed videos or when raters who are not experts in surgical resident evaluation are used. Future validation studies of operative performance assessment scales may be facilitated by using nonexpert surgeon raters viewing videos condensed using a standardized protocol.
(Copyright © 2020 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: