Establishing Validity Evidence for an Operative Performance Rating System for Plastic Surgery Residents

Autor: Nicole Z. Sommer, Kim A Bjorklund, Michael W. Neumeister, Steven J. Kasten
Rok vydání: 2019
Předmět:
Zdroj: Journal of Surgical Education. 76:529-539
ISSN: 1931-7204
DOI: 10.1016/j.jsurg.2018.08.016
Popis: Objective The aim of this study was to describe an operative performance rating system for plastic surgery residents and provide validity evidence for the instrument. Methods Three plastic surgery residents (PGY levels 1, 5, and 6) from Southern Illinois University School of Medicine (SIUSOM) performed a carpal tunnel release with audio video recording. The 3 videos were reviewed by 8 expert hand surgeons and 3 SIUSOM faculty using the operative performance rating system instrument to assess resident operative performance. Validity evidence including content, internal structure, and relationship to other variables was collected. Results Inter-rater reliability was consistently fair to moderate (weighted Cohen's Kappa 0.44-0.84 for experts, 0.24-0.55 for SIUSOM raters), and all assessment items were highly correlated (Cronbach's alpha of 0.9867). Local SIUSOM faculty routinely demonstrated higher overall scores for PGY 1 and PGY 6 residents compared to expert raters. Conclusions Although limited by small numbers, this pilot study suggests that potential bias based upon PGY year, identity, and performance history may exist and independent assessment by unbiased raters or comparison to national operative norms may be valuable. Our study provides baseline validity evidence for a resident operative performance assessment tool that can be integrated into practice in plastic surgery training programs.
Databáze: OpenAIRE
Externí odkaz: