Promises and Pitfalls of the Linearized Laplace in Bayesian Optimization

Autor:	Kristiadi, Agustinus, Immer, Alexander, Eschenhagen, Runa, Fortuin, Vincent
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	The linearized-Laplace approximation (LLA) has been shown to be effective and efficient in constructing Bayesian neural networks. It is theoretically compelling since it can be seen as a Gaussian process posterior with the mean function given by the neural network's maximum-a-posteriori predictive function and the covariance function induced by the empirical neural tangent kernel. However, while its efficacy has been studied in large-scale tasks like image classification, it has not been studied in sequential decision-making problems like Bayesian optimization where Gaussian processes -- with simple mean functions and kernels such as the radial basis function -- are the de-facto surrogate models. In this work, we study the usefulness of the LLA in Bayesian optimization and highlight its strong performance and flexibility. However, we also present some pitfalls that might arise and a potential problem with the LLA when the search space is unbounded. Comment: AABI 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2304.08309 Zobrazit plný text záznamu View this record from Arxiv