| Popis: |
Lens models have been used extensively for the examination of human judgments. Applications of lens model have occurred in a variety of contexts including judgments of reading achievement, clinical diagnosis, and personality characteristics. Rater-mediated writing assessments involve human judgments toward student essays. The judgmental process determines the quality of ratings, and this directly affects validity and fairness of ratings. Lens models provide a theoretical framework to evaluate rater judgment in rater-mediated assessments. Rasch measurement theory offers an alternative methodological approach for studying human judgments, and we propose combining Rasch measurement theory with a multifocal lens model. In order to illustrate our approach, the ratings of three expert raters and 20 operational raters from a state writing assessment program in the United States are examined in this study. There are only 35% of the students with comparable writing proficiency measures based on a comparison of ratings from operational raters and expert raters. The combination of a multifocal lens model with Rasch measurement theory offers a new pathway for understanding the quality of ratings for rater-mediated assessments. |
