{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T12:20:23Z","timestamp":1774527623525,"version":"3.50.1"},"reference-count":33,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2017,3]]},"abstract":"<jats:p> We show that Langevin Markov chain Monte Carlo inference in an energy-based model with latent variables has the property that the early steps of inference, starting from a stationary point, correspond to propagating error gradients into internal layers, similar to backpropagation. The backpropagated error is with respect to output units that have received an outside driving force pushing them away from the stationary point. Backpropagated error gradients correspond to temporal derivatives with respect to the activation of hidden units. These lead to a weight update proportional to the product of the presynaptic firing rate and the temporal rate of change of the postsynaptic firing rate. Simulations and a theoretical argument suggest that this rate-based update rule is consistent with those associated with spike-timing-dependent plasticity. The ideas presented in this article could be an element of a theory for explaining how brains perform credit assignment in deep hierarchies as efficiently as backpropagation does, with neural computation corresponding to both approximate inference in continuous-valued latent variables and error backpropagation, at the same time. <\/jats:p>","DOI":"10.1162\/neco_a_00934","type":"journal-article","created":{"date-parts":[[2017,1,17]],"date-time":"2017-01-17T21:43:31Z","timestamp":1484689411000},"page":"555-577","source":"Crossref","is-referenced-by-count":63,"title":["STDP-Compatible Approximation of Backpropagation in an Energy-Based Model"],"prefix":"10.1162","volume":"29","author":[{"given":"Yoshua","family":"Bengio","sequence":"first","affiliation":[{"name":"Montreal Institute for Learning Algorithms, University of Montreal, Montreal H3T 1J4, Quebec, Canada, and Canadian Institute for Advanced Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Mesnard","sequence":"additional","affiliation":[{"name":"Computer Science Department, \u00c9cole Normale Sup\u00e9rieure, Paris 75005, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Asja","family":"Fischer","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Bonn, Bonn 53117, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saizheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Montreal Institute for Learning Algorithms, University of Montreal, Montreal H3T 1J4, Quebec, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuhuai","family":"Wu","sequence":"additional","affiliation":[{"name":"University of Toronto, Toronto, Ontario, M5S 1A1, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","author":"Alain G.","year":"2013","journal-title":"Proceedings of the International Conference on Learning Representations"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020281327116"},{"key":"B3","author":"Arora S.","year":"2015","journal-title":"Why are deep nets reversible? A simple theory, with implications for training"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1126\/science.1195870"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.18-24-10464.1998"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.24.1.139"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.02-01-00032.1982"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2012.08.001"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.97.048104"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1007\/s11229-007-9237-y"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1038\/416433a"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1038\/383076a0"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1105933108"},{"key":"B14","author":"Hinton G. E.","year":"2007","journal-title":"How to do backpropagation in a brain"},{"key":"B15","author":"Hinton G. E.","year":"2016","journal-title":"Can the brain do back-propagation?"},{"key":"B16","first-page":"358","volume-title":"Neural information processing systems","author":"Hinton G. E.","year":"1988"},{"key":"B17","first-page":"282","volume-title":"Parallel distributed processing: Explorations in the microstructure of cognition. Vol. 1: Foundations","author":"Hinton G. E.","year":"1986"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.81.10.3088"},{"key":"B19","first-page":"4107","volume-title":"Advances in neural information processing systems","volume":"29","author":"Hubara I.","year":"2016"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80003-6"},{"key":"B21","author":"Lillicrap T. P.","year":"2014","journal-title":"Random feedback weights support learning in deep neural networks"},{"issue":"2","key":"B22","first-page":"8","volume":"4","author":"Markram H.","year":"2012","journal-title":"Frontiers in Synaptic Plasticity"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5297.213"},{"key":"B24","first-page":"2007","volume":"21","author":"Markram H.","year":"1995","journal-title":"Soc. Neurosci. Abs."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-008-0233-1"},{"issue":"38","key":"B26","volume":"8","author":"Rezende D. J.","year":"2014","journal-title":"Frontiers in Computational Neuroscience"},{"key":"B27","author":"Scellier B.","year":"2016","journal-title":"Equilibrium propagation: Bridging the gap between energy-based models and backpropagation"},{"key":"B28","author":"Shepherd G. M.","year":"2003","journal-title":"The synaptic organization of the brain"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.20-23-08812.2000"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00142"},{"key":"B31","first-page":"3371","volume":"11","author":"Vincent P.","year":"2010","journal-title":"J. Machine Learning Res."},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"B33","first-page":"199","volume-title":"Advances in neural information processing systems","volume":"12","author":"Xie X.","year":"2000"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00934","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:41:44Z","timestamp":1615585304000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/29\/3\/555-577\/8247"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,3]]}},"alternative-id":["10.1162\/NECO_a_00934"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00934","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3]]}}}