Reliability of assessment of medical students’ non-technical skills using a behavioural marker system: does clinical experience matter?
| Autor: | Joanne Kerins, Samantha E Smith, Victoria R Tallentire, Benjamin Clarke, Ailsa Hamilton, Emma Claire Phillips |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
medicine.medical_specialty
education.field_of_study Intraclass correlation health care facilities manpower and services education Population Health Informatics behavioral disciplines and activities Patient care Education Modeling and Simulation Physical therapy medicine Technical skills Psychology Reliability (statistics) Original Research |
| Zdroj: | BMJ Simul Technol Enhanc Learn |
| ISSN: | 2056-6697 |
| DOI: | 10.1136/bmjstel-2020-000705 |
| Popis: | IntroductionNon-technical skills are recognised to play an integral part in safe and effective patient care. Medi-StuNTS (Medical Students’ Non-Technical Skills) is a behavioural marker system developed to enable assessment of medical students’ non-technical skills. This study aimed to assess whether newly trained raters with high levels of clinical experience could achieve reliability coefficients of >0.7 and to compare differences in inter-rater reliability of raters with varying clinical experience.MethodsForty-four raters attended a workshop on Medi-StuNTS before independently rating three videos of medical students participating in immersive simulation scenarios. Data were grouped by raters’ levels of clinical experience. Inter-rater reliability was assessed by calculating intraclass correlation coefficients (ICC).ResultsEleven raters with more than 10 years of clinical experience achieved single-measure ICC of 0.37 and average-measures ICC of 0.87. Fourteen raters with more than or equal to 5 years and less than 10 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC of 0.59. Nineteen raters with less than 5 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC 0.65.ConclusionsUsing 11 newly trained raters with high levels of clinical experience produced highly reliable ratings that surpassed the prespecified inter-rater reliability standard; however, a single rater from this group would not achieve sufficiently reliable ratings. This is consistent with previous studies using other medical behavioural marker systems. This study demonstrated a decrease in inter-rater reliability of raters with lower levels of clinical experience, suggesting caution when using this population as raters for assessment of non-technical skills. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|