Weight-Based Classification of Raters and Rater Cognition in an EFL Speaking Test
| Autor: | Hongwen Cai |
|---|---|
| Rok vydání: | 2015 |
| Předmět: | |
| Zdroj: | Language Assessment Quarterly. 12:262-282 |
| ISSN: | 1543-4311 1543-4303 |
| DOI: | 10.1080/15434303.2015.1053134 |
| Popis: | This study is an attempt to classify raters according to their weighting patterns and explore systematic differences between rater types in the rating process. In the context of an EFL speaking test, 126 raters were classified into three types—form-oriented, balanced, and content-oriented—through cluster analyses of their weighting patterns derived from a holistic judgment task. In subsequent verbal protocols, a smaller sample of raters from each type rated the performance of real test takers and justified their ratings. The procedure for classifying raters yielded reliable results that agreed broadly with self-perceived and actual weights of the raters in the rating process. Verbal protocol analyses found systematic differences across rater types in the distinction between criteria and conformity to rubric-implied criteria. The form-oriented raters were found to overweigh form-related criteria and underweigh content-related criteria, which suggested unbalanced construct representation and led to more hal... |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|