| Popis: |
The COVID-19 pandemic influenced entrance testing to the master program of psychology at the Masaryk University in 2020. An administration of the standard paper-and-pencil knowledge test was not possible; therefore, we chose the bachelor theses rating. This paper is “psychometrical casuistic”, which covers criteria development regarding content validity, design selection, and results. Two randomly selected raters rated each thesis, and we equated their severity using the logistic linear test model (LLTM) under the Item Response Theory (IRT) paradigm. This procedure resulted in the unidimensional and unbiased scores equated across 18 judges and 2 terms (n1 = 82, n2 = 48). The reliability was comparable to the standard tests, r = 0,869, and judges’ severity or criteria difficulty did not differ across them. The resulting rating seems to be valid and not less fair compared to the written exam. The proposed method can serve other departments and other goals, not only as an entrance test. We share an analytical script and all the necessary materials to enhance using the method. |
