Optimization of Annealed Importance Sampling Hyperparameters
| Autor: | Shirin Goshtasbpour, Fernando Perez-Cruz |
|---|---|
| Přispěvatelé: | Amini, Massih-Reza, Canu, Stéphane, Fischer, Asja, Guns, Tias, Kralj Novak, Petra, Tsoumakas, Grigorios |
| Rok vydání: | 2023 |
| Předmět: | |
| Zdroj: | Machine Learning and Knowledge Discovery in Databases ISBN: 9783031264184 Lecture Notes in Computer Science, 13717 Machine Learning and Knowledge Discovery in Databases |
| ISSN: | 0302-9743 1611-3349 |
| DOI: | 10.1007/978-3-031-26419-1_11 |
| Popis: | Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. Although AIS is guaranteed to provide unbiased estimate for any set of hyperparameters, the common implementations rely on simple heuristics such as the geometric average bridging distributions between initial and the target distribution which affect the estimation performance when the computation budget is limited. In order to reduce the number of sampling iterations, we present a parameteric AIS process with flexible intermediary distributions defined by a residual density with respect to the geometric mean path. Our method allows parameter sharing between annealing distributions, the use of fix linear schedule for discretization and amortization of hyperparameter selection in latent variable models. We assess the performance of Optimized-Path AIS for marginal likelihood estimation of deep generative models and compare it to compare it to more computationally intensive AIS. Lecture Notes in Computer Science, 13717 ISSN:0302-9743 ISSN:1611-3349 Machine Learning and Knowledge Discovery in Databases ISBN:978-3-031-26419-1 ISBN:978-3-031-26418-4 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|