STDP-Compatible Approximation of Backpropagation in an Energy-Based Model
| Autor: | Yoshua Bengio, Yuhuai Wu, Thomas Mesnard, Saizheng Zhang, Asja Fischer |
|---|---|
| Rok vydání: | 2017 |
| Předmět: |
0301 basic medicine
Cognitive Neuroscience Inference Latent variable Machine learning computer.software_genre 03 medical and health sciences symbols.namesake 0302 clinical medicine Models of neural computation Arts and Humanities (miscellaneous) Mathematics Quantitative Biology::Neurons and Cognition business.industry Markov chain Monte Carlo Stationary point Backpropagation Approximate inference 030104 developmental biology symbols Artificial intelligence business computer Algorithm 030217 neurology & neurosurgery Energy (signal processing) |
| Zdroj: | Neural Computation. 29:555-577 |
| ISSN: | 1530-888X 0899-7667 |
| DOI: | 10.1162/neco_a_00934 |
| Popis: | We show that Langevin Markov chain Monte Carlo inference in an energy-based model with latent variables has the property that the early steps of inference, starting from a stationary point, correspond to propagating error gradients into internal layers, similar to backpropagation. The backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point. Backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units. These lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate. Simulations and a theoretical argument suggest that this rate-based update rule is consistent with those associated with spike-timing-dependent plasticity. The ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does, with neural computation corresponding to both approximate inference in continuous-valued latent variables and error backpropagation, at the same time. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|