| Popis: |
Assessment continues to be a major driver and measure of students’ learning & competence. Assessments, therefore, should be designed in a manner as to yield information that accurately and completely describes the degree to which the students have gained requisite knowledge. At the same time, the assessment should be constructed and presented to the students in a manner that is not overly complex. Written assessments, in the context of medical students, are used to test medical knowledge,of all the types of written assessments in current practice, the Multiple-Choice Question (MCQ) is the mainstay written assessment technique. MCQs that are not appropriately constructed or are ‘flawed’ items, may affect student performance, making the test either relatively difficult or easy. Method. In this non-experimental observational study, four hundred multiple choice questions from 8 end of clerkship Ophthalmology written assessments were independently scrutinized for item writing flaws by a panel of four expert judges, who noted the flaws on a prescribed form. Based on their findings two optical mark reader keys were generated, one consisting of all questions; the flawed scale, and the other excluding flawed questions; the standard scale. The student optical mark reader sheets were compared against both to generate a standard and flawed scale test data consisting of student pass rate, mean score, mean discrimination & mean difficulty index and test reliability using Laboratory of educational research test analysis package. Results Sixty item writing flaws were identified. All eight tests showed that item writing flaws affected student performance as well as test psychometrics. Mean scores increased in all standard scales as compared to flawed scales, while pass rate increased in all but four tests; it remained the same in two and was lower in the last two. Mean Discrimination index and reliability also increased in standard scale across all tests. Mean difficulty index increased in all tests but two, it was lower in one, while no change was observed in the other. Conclusion: This study shows that Item writing flaws affect test validity and reliability. Moreover, they also distort the pass rates, penalizing students most of the time. |
