The impact of systematically repairing multiple choice questions with low discrimination on assessment reliability: an interrupted time series analysis.

Autor:	Desy J; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Harvey A; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Weeks S; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Busche KD; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Martin K; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Paget M; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., Naugler C; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada., McLaughlin K; Office of Undergraduate Medical Education, Cummings School of Medicine, University of Calgary, Alberta, Canada.
Jazyk:	angličtina
Zdroj:	Canadian medical education journal [Can Med Educ J] 2024 Jul 12; Vol. 15 (3), pp. 52-56. Date of Electronic Publication: 2024 Jul 12 (Print Publication: 2024).
DOI:	10.36834/cmej.77596
Abstrakt:	At our centre, we introduced a continuous quality improvement (CQI) initiative during academic year 2018-19 targeting for repair multiple choice question (MCQ) items with discrimination index ( D ) < 0.1. The purpose of this study was to assess the impact of this initiative on reliability/internal consistency of our assessments. Our participants were medical students during academic years 2015-16 to 2020-21 and our data were summative MCQ assessments during this time. Since the goal was to systematically review and improve summative assessments in our undergraduate program on an ongoing basis, we used interrupted time series analysis to assess the impact on reliability. Between 2015-16 and 2017-18 there was a significant negative trend in the mean alpha coefficient for MCQ exams (regression coefficient -0.027 [-0.008, -0.047], p = 0.024). In the academic year following the introduction of our initiative (2018-19) there was a significant increase in the mean alpha coefficient (regression coefficient 0.113 [0.063, 0.163], p = 0.010) which was then followed by a significant positive post-intervention trend (regression coefficient 0.056 [0.037, 0.075], p = 0.006). In conclusion, our CQI intervention resulted in an immediate and progressive improvement reliability of our MCQ assessments. Competing Interests: None of the authors have conflict of interest to declare regarding the work presented in this manuscript. (© 2024 Desy, Harvey, Weeks, Busche, Martin, Paget, Naugler, McLaughlin; licensee Synergies Partners.)
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu