Gradient-flow adaptive importance sampling for Bayesian leave one out cross-validation with application to sigmoidal classification models.

Autor:	Chang JC; NIH Clinical Center, Rehabilitation Medicine, Epidemiology and Biostatistics Section, Bethesda MD, USA., Li X; UCLA Department of Computational Medicine, Los Angeles CA, USA., Xu S; Data Science Research Center, Duke Kunshan University, Kunshan, Jiangsu, China., Yao HR; NIH Clinical Center, Rehabilitation Medicine, Epidemiology and Biostatistics Section, Bethesda MD, USA., Porcino J; NIH Clinical Center, Rehabilitation Medicine, Epidemiology and Biostatistics Section, Bethesda MD, USA., Chow CC; NIH NIDDK, Laboratory of Biological Modeling, Bethesda MD, USA.
Jazyk:	angličtina
Zdroj:	ArXiv [ArXiv] 2024 Oct 20. Date of Electronic Publication: 2024 Oct 20.
Abstrakt:	We introduce gradient-flow-guided adaptive importance sampling (IS) transformations for stabilizing Monte-Carlo approximations of leave-one-out (LOO) cross-validated predictions for Bayesian models. After defining two variational problems, we derive corresponding simple nonlinear transformations that utilize gradient information to shift a model's pre-trained full-data posterior closer to the target LOO posterior predictive distributions. In doing so, the transformations stabilize importance weights. The resulting Monte Carlo integrals depend on Jacobian determinants with respect to the model Hessian. We derive closed-form exact formulae for these Jacobian determinants in the cases of logistic regression and shallow ReLU-activated artificial neural networks, and provide a simple approximation that sidesteps the need to compute full Hessian matrices and their spectra. We test the methodology on an n ≪ p dataset that is known to produce unstable LOO IS weights.
Databáze:	MEDLINE
Externí odkaz:	Zobrazit plný text záznamu