Principled Pruning of Bayesian Neural Networks through Variational Free Energy Minimization

Autor:	Jim Beckers, Bart van Erp, Ziyue Zhao, Kirill Kondrashov, Bert de Vries
Přispěvatelé:	Electrical Engineering, Bayesian Intelligent Autonomous Systems, Signal Processing Systems, EAISI High Tech Systems, EAISI Health, EAISI Foundational
Jazyk:	angličtina
Rok vydání:	2022
Předmět:	Signal Processing (eess.SP) FOS: Computer and information sciences Computer Science - Machine Learning FOS: Electrical engineering electronic engineering information engineering Electrical Engineering and Systems Science - Signal Processing Machine Learning (cs.LG)
Zdroj:	arXiv, 2022:2210.09134. Cornell University Library Pure TUe
Popis:	Bayesian model reduction provides an efficient approach for comparing the performance of all nested sub-models of a model, without re-evaluating any of these sub-models. Until now, Bayesian model reduction has been applied mainly in the computational neuroscience community. In this paper, we formulate and apply Bayesian model reduction to perform principled pruning of Bayesian neural networks, based on variational free energy minimization. This novel parameter pruning scheme solves the shortcomings of many current state-of-the-art pruning methods that are used by the signal processing community. The proposed approach has a clear stopping criterion and minimizes the same objective that is used during training. Next to these theoretical benefits, our experiments indicate better model performance in comparison to state-of-the-art pruning schemes.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_dedup___::f850c9f4f498d1ff8629fb07b27b9141 https://arxiv.org/abs/2210.09134 Zobrazit plný text záznamu