Is the assumption of equal distances between global assessment categories used in borderline regression valid?

Autor: Patrick J. McGown, Celia A. Brown, Ann Sebastian, Ricardo Le, Anjali Amin, Andrew Greenland, Amir H. Sam
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Zdroj: BMC Medical Education, Vol 22, Iss 1, Pp 1-10 (2022)
Druh dokumentu: article
ISSN: 1472-6920
DOI: 10.1186/s12909-022-03753-5
Popis: Abstract Background Standard setting for clinical examinations typically uses the borderline regression method to set the pass mark. An assumption made in using this method is that there are equal intervals between global ratings (GR) (e.g. Fail, Borderline Pass, Clear Pass, Good and Excellent). However, this assumption has never been tested in the medical literature to the best of our knowledge. We examine if the assumption of equal intervals between GR is met, and the potential implications for student outcomes. Methods Clinical finals examiners were recruited across two institutions to place the typical ‘Borderline Pass’, ‘Clear Pass’ and ‘Good’ candidate on a continuous slider scale between a typical ‘Fail’ candidate at point 0 and a typical ‘Excellent’ candidate at point 1. Results were analysed using one-sample t-testing of each interval to an equal interval size of 0.25. Secondary data analysis was performed on summative assessment scores for 94 clinical stations and 1191 medical student examination outcomes in the final 2 years of study at a single centre. Results On a scale from 0.00 (Fail) to 1.00 (Excellent), mean examiner GRs for ‘Borderline Pass’, ‘Clear Pass’ and ‘Good’ were 0.33, 0.55 and 0.77 respectively. All of the four intervals between GRs (Fail-Borderline Pass, Borderline Pass-Clear Pass, Clear Pass-Good, Good-Excellent) were statistically significantly different to the expected value of 0.25 (all p-values
Databáze: Directory of Open Access Journals