New improved gamma: Enhancing the accuracy of Goodman–Kruskal’s gamma using ROC curves
| Autor: | D. Paul Higham, Philip A. Higham |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Signal Detection
Psychological Concordance Experimental and Cognitive Psychology Scale (descriptive set theory) ROC curves Article 050105 experimental psychology 03 medical and health sciences Discrimination Psychological 0302 clinical medicine Arts and Humanities (miscellaneous) Kruskal's algorithm Statistics Developmental and Educational Psychology Humans 0501 psychology and cognitive sciences Detection theory Gamma General Psychology Mathematics Parametric statistics Models Statistical Receiver operating characteristic Trapezoidal rule 05 social sciences Probabilistic logic Trapezoidal rule (differential equations) ROC Curve Psychology (miscellaneous) Resolution Metacognition 030217 neurology & neurosurgery |
| Zdroj: | Behavior Research Methods |
| ISSN: | 1554-3528 |
| DOI: | 10.3758/s13428-018-1125-5 |
| Popis: | For decades, researchers have debated the relative merits of different measures of people’s ability to discriminate the correctness of their own responses (resolution). The probabilistic approach, primarily led by Nelson, has advocated the Goodman–Kruskal gamma coefficient, an ordinal measure of association. The signal detection approach has advocated parametric measures of distance between the evidence distributions or the area under the receiver operating characteristic (ROC) curve. Here we provide mathematical proof that the indices associated with the two approaches are far more similar than has previously been thought: The true value of gamma is equal to twice the true area under the ROC curve minus one. Using this insight, we report 36 simulations involving 3,600,000 virtual participants that pitted gamma estimated with the original concordance/discordance formula against gamma estimated via ROC curves and the trapezoidal rule. In all but five of our simulations—which systematically varied resolution, the number of points on the metacognitive scale, and response bias—the ROC-based gamma estimate deviated less from the true value of gamma than did the traditional estimate. Consequently, we recommend using ROC curves to estimate gamma in the future. Electronic supplementary material The online version of this article (10.3758/s13428-018-1125-5) contains supplementary material, which is available to authorized users. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink