Understanding disentangling in $\beta$-VAE

Autor:	Burgess, Christopher P., Higgins, Irina, Pal, Arka, Matthey, Loic, Watters, Nick, Desjardins, Guillaume, Lerchner, Alexander
Rok vydání:	2018
Předmět:	Statistics - Machine Learning Computer Science - Artificial Intelligence Computer Science - Learning
Druh dokumentu:	Working Paper
Popis:	We present new intuitions and theoretical assessments of the emergence of disentangled representation in variational autoencoders. Taking a rate-distortion theory perspective, we show the circumstances under which representations aligned with the underlying generative factors of variation of data emerge when optimising the modified ELBO bound in $\beta$-VAE, as training progresses. From these insights, we propose a modification to the training regime of $\beta$-VAE, that progressively increases the information capacity of the latent code during training. This modification facilitates the robust learning of disentangled representations in $\beta$-VAE, without the previous trade-off in reconstruction accuracy. Comment: Presented at the 2017 NIPS Workshop on Learning Disentangled Representations
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1804.03599 Zobrazit plný text záznamu View this record from Arxiv