{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,18]],"date-time":"2025-10-18T10:27:14Z","timestamp":1760783234165,"version":"3.37.3"},"reference-count":66,"publisher":"MIT Press","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2010,6]]},"abstract":"<jats:p>We introduce a framework for decision making in which the learning of decision making is reduced to its simplest and biologically most plausible form: Hebbian learning on a linear neuron. We cast our Bayesian-Hebb learning rule as reinforcement learning in which certain decisions are rewarded and prove that each synaptic weight will on average converge exponentially fast to the log-odd of receiving a reward when its pre- and postsynaptic neurons are active. In our simple architecture, a particular action is selected from the set of candidate actions by a winner-take-all operation. The global reward assigned to this action then modulates the update of each synapse. Apart from this global reward signal, our reward-modulated Bayesian Hebb rule is a pure Hebb update that depends only on the coactivation of the pre- and postsynaptic neurons, not on the weighted sum of all presynaptic inputs to the postsynaptic neuron as in the perceptron learning rule or the Rescorla-Wagner rule. This simple approach to action-selection learning requires that information about sensory inputs be presented to the Bayesian decision stage in a suitably preprocessed form resulting from other adaptive processes (acting on a larger timescale) that detect salient dependencies among input features. Hence our proposed framework for fast learning of decisions also provides interesting new hypotheses regarding neural nodes and computational goals of cortical areas that provide input to the final decision stage.<\/jats:p>","DOI":"10.1162\/neco.2010.03-09-980","type":"journal-article","created":{"date-parts":[[2010,2,8]],"date-time":"2010-02-08T21:44:20Z","timestamp":1265665460000},"page":"1399-1444","source":"Crossref","is-referenced-by-count":30,"title":["Reward-Modulated Hebbian Learning of Decision Making"],"prefix":"10.1162","volume":"22","author":[{"given":"Michael","family":"Pfeiffer","sequence":"first","affiliation":[{"name":"Institute for Theoretical Computer Science, Graz University of Technology, A-8010 Graz, Austria"}]},{"given":"Bernhard","family":"Nessler","sequence":"additional","affiliation":[{"name":"Institute for Theoretical Computer Science, Graz University of Technology, A-8010 Graz, Austria"}]},{"given":"Rodney J.","family":"Douglas","sequence":"additional","affiliation":[{"name":"Institute of Neuroinformatics, University of Z\u00fcrich and ETH Z\u00fcrich, CH-8057 Z\u00fcrich, Switzerland"}]},{"given":"Wolfgang","family":"Maass","sequence":"additional","affiliation":[{"name":"Institute for Theoretical Computer Science, Graz University of Technology, A-8010 Graz, Austria"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1038\/81453"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2356"},{"volume-title":"Proc. of the 9th Int. Workshop on Artificial Intelligence and Statistics","year":"2003","author":"Attias H.","key":"B3"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-75225-7_15"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013689704352"},{"key":"B6","first-page":"89","volume-title":"Advances in neural information processing systems","volume":"21","author":"Auer P.","year":"2009"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1038\/35036191"},{"volume-title":"Neuro-dynamic programming","year":"1996","author":"Bertsekas D. P.","key":"B8"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-45528-0"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.31.060407.125639"},{"volume-title":"Theoretical neuroscience: Computational and mathematical modeling of neural systems","year":"2001","author":"Dayan P.","key":"B11"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.8.4.429"},{"key":"B13","first-page":"451","volume-title":"Advances in neural information processing systems","volume":"13","author":"Dayan P.","year":"2001"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1080\/03772063.2003.11416335"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2008.20.1.91"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007413511361"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.27.070203.144152"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00044-8"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/18.796383"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00364.2007"},{"key":"B21","first-page":"515","volume-title":"The handbook of brain theory and neural networks","author":"Fr\u00e9gnac Y.","year":"2003"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1126\/science.3749885"},{"key":"B23","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1111\/j.2517-6161.1979.tb01068.x","volume":"41","author":"Gittins J.","year":"1979","journal-title":"Journal of the Royal Statistical Society"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.29.051605.113038"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogpsych.2005.05.004"},{"key":"B26","first-page":"442","volume":"19","author":"Gurney K.","year":"2006","journal-title":"Neural Computation"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1038\/35016072"},{"volume-title":"The organization of behavior","year":"1949","author":"Hebb D. O.","key":"B28"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36127-8_35"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-68282-2"},{"key":"B31","first-page":"260","volume-title":"Proc. of the 15th International Conference on Machine Learning (ICML)","author":"Kearns M.","year":"1998"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/BF00200801"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/18.910572"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1016\/0196-8858(85)90002-8"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065789000499"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065796000816"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000180"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1016\/S0005-1098(98)00019-3"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300014827"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1038\/377725a0"},{"volume-title":"Learning Bayesian networks","year":"2004","author":"Neapolitan R.","key":"B41"},{"key":"B42","volume-title":"Advances in neural information processing systems","volume":"20","author":"Neftci E.","year":"2008"},{"key":"B43","volume-title":"Advances in neural information processing systems","volume":"21","author":"Nessler B.","year":"2009"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.1998.0287"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1038\/381607a0"},{"key":"B46","first-page":"893","volume-title":"The handbook of brain theory and neural networks","author":"Pouget A.","year":"2002","edition":"2"},{"key":"B47","first-page":"239","volume-title":"Bayesian brain","author":"Rao R. P. N.","year":"2007"},{"key":"B48","first-page":"64","volume-title":"A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement","author":"Rescorla R. A.","year":"1972"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1038\/35092560"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1038\/14819"},{"key":"B51","first-page":"898","volume-title":"Proc. of the International Joint Conference on Artificial Intelligence","author":"Roth D.","year":"1999"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2006.05.034"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1080\/net.13.2.179.194"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2009.08-08-837"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1126\/science.1094765"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1038\/nrn1666"},{"key":"B58","first-page":"161","volume-title":"Proceedings of the 7th Yale Workshop on Adaptive and Learning Systems","author":"Sutton R. S.","year":"1992"},{"volume-title":"Reinforcement learning: An introduction","year":"1998","author":"Sutton R. S.","key":"B59"},{"key":"B60","first-page":"1393","volume-title":"Advances in neural information processing systems","volume":"18","author":"Verma D.","year":"2006"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730197"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)01092-9"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1038\/nature05852"},{"key":"B64","first-page":"157","volume-title":"Advances in neural information processing systems","volume":"15","author":"Yu A.","year":"2003"},{"key":"B65","first-page":"1561","volume-title":"Advances in neural information processing systems","volume":"18","author":"Yuille A. L.","year":"2006"},{"key":"B66","first-page":"1228","volume-title":"The handbook of brain theory and neural networks","author":"Yuille A. L.","year":"2003"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco.2010.03-09-980","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T02:18:25Z","timestamp":1739845105000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/22\/6\/1399-1444\/7555"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,6]]},"references-count":66,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2010,6]]}},"alternative-id":["10.1162\/neco.2010.03-09-980"],"URL":"https:\/\/doi.org\/10.1162\/neco.2010.03-09-980","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"type":"print","value":"0899-7667"},{"type":"electronic","value":"1530-888X"}],"subject":[],"published":{"date-parts":[[2010,6]]}}}