Learning Wake-Sleep Recurrent Attention Models

Autor:	Ba, Jimmy, Grosse, Roger, Salakhutdinov, Ruslan, Frey, Brendan
Rok vydání:	2015
Předmět:	Computer Science - Learning
Druh dokumentu:	Working Paper
Popis:	Despite their success, convolutional neural networks are computationally expensive because they must examine all image locations. Stochastic attention-based models have been shown to improve computational efficiency at test time, but they remain difficult to train because of intractable posterior inference and high variance in the stochastic gradient estimates. Borrowing techniques from the literature on training deep generative models, we present the Wake-Sleep Recurrent Attention Model, a method for training stochastic attention networks which improves posterior inference and which reduces the variability in the stochastic gradients. We show that our method can greatly speed up the training time for stochastic attention networks in the domains of image classification and caption generation. Comment: To appear in NIPS 2015
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1509.06812 Zobrazit plný text záznamu View this record from Arxiv