Sequential Detection of Compromised Items Using Response Times in Computerized Adaptive Testing
Autor: | Jinming Zhang, Hua Hua Chang, Edison M. Choe |
---|---|
Rok vydání: | 2017 |
Předmět: |
Psychometrics
Computer science Machine learning computer.software_genre behavioral disciplines and activities 01 natural sciences 010104 statistics & probability 0504 sociology Moving average Reaction Time False positive paradox Humans Computer Simulation 0101 mathematics General Psychology Item pool Computers business.industry Applied Mathematics 05 social sciences 050401 social sciences methods Response time Computerized adaptive testing Artificial intelligence business computer Algorithms Change detection |
Zdroj: | Psychometrika. 83:650-673 |
ISSN: | 1860-0980 0033-3123 |
Popis: | Item compromise persists in undermining the integrity of testing, even secure administrations of computerized adaptive testing (CAT) with sophisticated item exposure controls. In ongoing efforts to tackle this perennial security issue in CAT, a couple of recent studies investigated sequential procedures for detecting compromised items, in which a significant increase in the proportion of correct responses for each item in the pool is monitored in real time using moving averages. In addition to actual responses, response times are valuable information with tremendous potential to reveal items that may have been leaked. Specifically, examinees that have preknowledge of an item would likely respond more quickly to it than those who do not. Therefore, the current study proposes several augmented methods for the detection of compromised items, all involving simultaneous monitoring of changes in both the proportion correct and average response time for every item using various moving average strategies. Simulation results with an operational item pool indicate that, compared to the analysis of responses alone, utilizing response times can afford marked improvements in detection power with fewer false positives. |
Databáze: | OpenAIRE |
Externí odkaz: |