Comparison of Global Evaluative Assessment of Robotic Surgery with objective performance indicators for the assessment of skill during robotic-assisted thoracic surgery.

Autor: Oh DS; University of Southern California, Keck School of Medicine, Los Angeles, CA; Data and Analytics, Intuitive Surgical, Sunnyvale, CA. Electronic address: daniel.oh@med.usc.edu., Ershad M; Data and Analytics, Intuitive Surgical, Sunnyvale, CA., Wee JO; Brigham and Women's Hospital, Harvard Medical School, Boston, MA., Sancheti MS; Emory University School of Medicine, Atlanta, GA., D'Souza DM; Ohio State University College of Medicine, Columbus, OH., Herrera LJ; Orlando Health, Orlando, FL., Schumacher LY; Massachusetts General Hospital, Harvard Medical School, Boston, MA., Shields M; Data and Analytics, Intuitive Surgical, Sunnyvale, CA., Brown K; Data and Analytics, Intuitive Surgical, Sunnyvale, CA., Yousaf S; Data and Analytics, Intuitive Surgical, Sunnyvale, CA., Lazar JF; Medstar Washington Hospital, Georgetown University, Washington, DC.
Jazyk: angličtina
Zdroj: Surgery [Surgery] 2023 Dec; Vol. 174 (6), pp. 1349-1355. Date of Electronic Publication: 2023 Sep 16.
DOI: 10.1016/j.surg.2023.08.008
Abstrakt: Background: The Global Evaluative Assessment of Robotic Skills is a popular but ultimately subjective assessment tool in robotic-assisted surgery. An alternative approach is to record system or console events or calculate instrument kinematics to derive objective performance indicators. The aim of this study was to compare these 2 approaches and correlate the Global Evaluative Assessment of Robotic Skills with different types of objective performance indicators during robotic-assisted lobectomy.
Methods: Video, system event, and kinematic data were recorded from the robotic surgical system during left upper lobectomy on a standardized perfused and pulsatile ex vivo porcine heart-lung model. Videos were segmented into steps, and the superior vein dissection was graded independently by 2 blinded expert surgeons with Global Evaluative Assessment of Robotic Skills. Objective performance indicators representing categories for energy use, event data, movement, smoothness, time, and wrist articulation were calculated for the same task and compared to Global Evaluative Assessment of Robotic Skills scores.
Results: Video and data from 51 cases were analyzed (44 fellows, 7 attendings). Global Evaluative Assessment of Robotic Skills scores were significantly higher for attendings (P < .05), but there was a significant difference in raters' scores of 31.4% (defined as >20% difference in total score). The interclass correlation was 0.44 for 1 rater and 0.61 for 2 raters. Objective performance indicators correlated with Global Evaluative Assessment of Robotic Skills to varying degrees. The most highly correlated Global Evaluative Assessment of Robotic Skills domain was efficiency. Instrument movement and smoothness were highly correlated among objective performance indicator categories. Of individual objective performance indicators, right-hand median jerk, an objective performance indicator of change of acceleration, had the highest correlation coefficient (0.55).
Conclusion: There was a relatively poor overall correlation between the Global Evaluative Assessment of Robotic Skills and objective performance indicators. However, both appear strongly correlated for certain metrics such as efficiency and smoothness. Objective performance indicators may be a potentially more quantitative and granular approach to assessing skill, given that they can be calculated mathematically and automatically without subjective interpretation.
(Copyright © 2023 Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: