PAC$^m$-Bayes: Narrowing the Empirical Risk Gap in the Misspecified Bayesian Regime

Autor:	Morningstar, Warren R., Alemi, Alexander A., Dillon, Joshua V.
Rok vydání:	2020
Předmět:	Computer Science - Machine Learning Statistics - Machine Learning
Zdroj:	International Conference on Artificial Intelligence and Statistics, 8270-8298, (2022)
Druh dokumentu:	Working Paper
Popis:	The Bayesian posterior minimizes the "inferential risk" which itself bounds the "predictive risk". This bound is tight when the likelihood and prior are well-specified. However since misspecification induces a gap, the Bayesian posterior predictive distribution may have poor generalization performance. This work develops a multi-sample loss (PAC$^m$) which can close the gap by spanning a trade-off between the two risks. The loss is computationally favorable and offers PAC generalization guarantees. Empirical study demonstrates improvement to the predictive distribution. Comment: Accepted at AISTATS2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2010.09629 Zobrazit plný text záznamu View this record from Arxiv