Evaluation of a Simpler Tool to Assess Nontechnical Skills During Simulated Critical Events
Autor: | Matthew D. McEvoy, John R. Boulet, David A. Roberts, Scott C. Watkins, Matthew B. Weinger |
---|---|
Rok vydání: | 2017 |
Předmět: |
Engineering
Educational measurement Epidemiology media_common.quotation_subject Applied psychology Decision Making Medicine (miscellaneous) Education Formative assessment 03 medical and health sciences symbols.namesake 0302 clinical medicine 030202 anesthesiology Anesthesiology Humans Simulation Training media_common Nurse Anesthetists Observer Variation Patient Care Team Teamwork business.industry Critical event Behaviorally anchored rating scales food and beverages Internship and Residency Reproducibility of Results 030208 emergency & critical care medicine Intra-rater reliability Pearson product-moment correlation coefficient Reliability engineering Group Processes Inter-rater reliability Modeling and Simulation symbols Clinical Competence Educational Measurement business |
Zdroj: | Simulation in healthcare : journal of the Society for Simulation in Healthcare. 12(2) |
ISSN: | 1559-713X |
Popis: | Introduction Management of critical events requires teams to employ nontechnical skills (NTS), such as teamwork, communication, decision making, and vigilance. We sought to estimate the reliability and provide evidence for the validity of the ratings gathered using a new tool for assessing the NTS of anesthesia providers, the behaviorally anchored rating scale (BARS), and compare its scores with those of an established NTS tool, the Anaesthetists' Nontechnical Skills (ANTS) scale. Methods Six previously trained raters (4 novices and 2 experts) reviewed and scored 18 recorded simulated pediatric crisis management scenarios using a modified ANTS and a BARS tool. Pearson correlation coefficients were calculated separately for the novice and expert raters, by scenario, and overall. Results The intrarater reliability of the ANTS total score was 0.73 (expert, 0.57; novice, 0.84); for the BARS tool, it was 0.80 (expert, 0.79; novice, 0.81). The average interrater reliability of BARS scores (0.58) was better than ANTS scores (0.37), and the interrater reliabilities of scores from novices (0.69 BARS and 0.52 ANTS) were better than those obtained from experts (0.47 BARS and 0.21 ANTS) for both scoring instruments. The Pearson correlation between the ANTS and BARS total scores was 0.74. Conclusions Overall, reliability estimates were better for the BARS scores than the ANTS scores. For both measures, the intrarater and interrater reliability was better for novices compared with domain experts, suggesting that properly trained novices can reliably assess the NTS of anesthesia providers managing a simulated critical event. There was substantial correlation between the 2 scoring instruments, suggesting that the tools measured similar constructs. The BARS tool can be an alternative to the ANTS scale for the formative assessment of NTS of anesthesia providers. |
Databáze: | OpenAIRE |
Externí odkaz: |