Evaluating Stan’s Variational Bayes Algorithm for Estimating Multidimensional IRT Models

Autor: Esther Ulitzsch, Steffen Nestler
Jazyk: angličtina
Rok vydání: 2022
Předmět:
Zdroj: Psych, Vol 4, Iss 1, Pp 73-88 (2022)
Druh dokumentu: article
ISSN: 2624-8611
DOI: 10.3390/psych4010007
Popis: Bayesian estimation of multidimensional item response theory (IRT) models in large data sets may come with impractical computational burdens when general-purpose Markov chain Monte Carlo (MCMC) samplers are employed. Variational Bayes (VB)—a method for approximating the posterior distribution—poses a potential remedy. Stan’s general-purpose VB algorithms have drastically improved the accessibility of VB methods for a wide psychometric audience. Using marginal maximum likelihood (MML) and MCMC as benchmarks, the present simulation study investigates the utility of Stan’s built-in VB function for estimating multidimensional IRT models with between-item dimensionality. VB yielded a marked speed-up in comparison to MCMC, but did not generally outperform MML in terms of run time. VB estimates were trustworthy only for item difficulties, while bias in item discriminations depended on the model’s dimensionality. Under realistic conditions of non-zero correlations between dimensions, VB correlation estimates were subject to severe bias. The practical relevance of performance differences is illustrated with data from PISA 2018. We conclude that in its current form, Stan’s built-in VB algorithm does not pose a viable alternative for estimating multidimensional IRT models.
Databáze: Directory of Open Access Journals