Hybrid Threshold-Based Sequential Procedures for Detecting Compromised Items in a Computerized Adaptive Testing Licensure Exam
| Autor: | Hong Qian, Chansoon Lee |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
Licensure
Item pool Computer science business.industry Applied Mathematics 05 social sciences 050401 social sciences methods 050301 education Machine learning computer.software_genre Education Classical test theory 0504 sociology Item response theory Developmental and Educational Psychology Computerized adaptive testing Artificial intelligence business 0503 education computer Applied Psychology |
| Zdroj: | Educational and Psychological Measurement. 82:782-810 |
| ISSN: | 1552-3888 0013-1644 |
| Popis: | Using classical test theory and item response theory, this study applied sequential procedures to a real operational item pool in a variable-length computerized adaptive testing (CAT) to detect items whose security may be compromised. Moreover, this study proposed a hybrid threshold approach to improve the detection power of the sequential procedure while controlling the Type I error rate. The hybrid threshold approach uses a local threshold for each item in an early stage of the CAT administration, and then it uses the global threshold in the decision-making stage. Applying various simulation factors, a series of simulation studies examined which factors contribute significantly to the power rate and lag time of the procedure. In addition to the simulation study, a case study investigated whether the procedures are applicable to the real item pool administered in CAT and can identify potentially compromised items in the pool. This research found that the increment of probability of a correct answer ( p-increment) was the simulation factor most important to the sequential procedures’ ability to detect compromised items. This study also found that the local threshold approach improved power rates and shortened lag times when the p-increment was small. The findings of this study could help practitioners implement the sequential procedures using the hybrid threshold approach in real-time CAT administration. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|