Kappa Coefficients for Missing Data
| Autor: | Alexandra de Raadt, Henk A.L. Kiers, Matthijs J. Warrens, Roel Bosker |
|---|---|
| Přispěvatelé: | Research and Evaluation of Educational Effectiveness, Psychometrics and Statistics |
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
Mean squared error
Cohen’s kappa Article Education nominal ratings missing data Cohen's kappa 0504 sociology inter-rater reliability Statistics Developmental and Educational Psychology COHENS KAPPA Applied Psychology Reliability (statistics) Mathematics listwise deletion Applied Mathematics Listwise deletion 05 social sciences 050401 social sciences methods 050301 education Missing data Inter-rater reliability Level of measurement AGREEMENT RELIABILITY Gwet’s kappa 0503 education Kappa |
| Zdroj: | Educational and Psychological Measurement Educational and Psychological Measurement, 79(3), 558-576. SAGE Publications Inc. |
| ISSN: | 0013-1644 |
| DOI: | 10.1177/0013164418823249 |
| Popis: | Cohen’s kappa coefficient is commonly used for assessing agreement between classifications of two raters on a nominal scale. Three variants of Cohen’s kappa that can handle missing data are presented. Data are considered missing if one or both ratings of a unit are missing. We study how well the variants estimate the kappa value for complete data under two missing data mechanisms—namely, missingness completely at random and a form of missingness not at random. The kappa coefficient considered in Gwet ( Handbook of Inter-rater Reliability, 4th ed.) and the kappa coefficient based on listwise deletion of units with missing ratings were found to have virtually no bias and mean squared error if missingness is completely at random, and small bias and mean squared error if missingness is not at random. Furthermore, the kappa coefficient that treats missing ratings as a regular category appears to be rather heavily biased and has a substantial mean squared error in many of the simulations. Because it performs well and is easy to compute, we recommend to use the kappa coefficient that is based on listwise deletion of missing ratings if it can be assumed that missingness is completely at random or not at random. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|