{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,7,18]],"date-time":"2023-07-18T16:17:41Z","timestamp":1689697061248},"reference-count":77,"publisher":"MIT Press - Journals","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2015,8]]},"abstract":"<jats:p> Rate distortion theory describes how to communicate relevant information most efficiently over a channel with limited capacity. One of the many applications of rate distortion theory is bounded rational decision making, where decision makers are modeled as information channels that transform sensory input into motor output under the constraint that their channel capacity is limited. Such a bounded rational decision maker can be thought to optimize an objective function that trades off the decision maker\u2019s utility or cumulative reward against the information processing cost measured by the mutual information between sensory input and motor output. In this study, we interpret a spiking neuron as a bounded rational decision maker that aims to maximize its expected reward under the computational constraint that the mutual information between the neuron\u2019s input and output is upper bounded. This abstract computational constraint translates into a penalization of the deviation between the neuron\u2019s instantaneous and average firing behavior. We derive a synaptic weight update rule for such a rate distortion optimizing neuron and show in simulations that the neuron efficiently extracts reward-relevant information from the input by trading off its synaptic strengths against the collected reward. <\/jats:p>","DOI":"10.1162\/neco_a_00758","type":"journal-article","created":{"date-parts":[[2015,6,16]],"date-time":"2015-06-16T19:20:37Z","timestamp":1434482437000},"page":"1686-1720","source":"Crossref","is-referenced-by-count":8,"title":["A Reward-Maximizing Spiking Neuron as a Bounded Rational Decision Maker"],"prefix":"10.1162","volume":"27","author":[{"given":"Felix","family":"Leibfried","sequence":"first","affiliation":[{"name":"Max Planck Institute for Biological Cybernetics, Max Planck Institute for Intelligent Systems, Graduate Training Centre of Neuroscience, D-72076 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel A.","family":"Braun","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Biological Cybernetics, Max Planck Institute for Intelligent Systems, D-72076 T\u00fcbingen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1038\/81453"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1007\/s11460-010-0101-3"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1972.1054753"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2007.19.8.2245"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1995.7.6.1129"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.02-01-00032.1982"},{"key":"B7","volume-title":"Pattern Recognition and Machine Learning.","author":"Bishop C. M.","year":"2006"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1972.1054855"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.3390\/e16084662"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1109\/ADPRL.2011.5967366"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2010.08-09-1084"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.70.10.2974"},{"issue":"5588","key":"B13","first-page":"705","volume":"264","author":"Changeux J. P.","year":"1976","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1162\/089976603321891774"},{"key":"B15","volume-title":"Elements of information theory.","author":"Cover T. M.","year":"2006"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1974.1055146"},{"key":"B17","author":"Daniel C.","year":"2012","journal-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2013.6706759"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2006.03.006"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.8.4.429"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2008.08.003"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1038\/nn2077"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2007.19.6.1468"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2787"},{"key":"B26","volume-title":"Bounded rationality: The adaptive toolbox.","author":"Gigerenzer G.","year":"2001"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/ISIT.2007.4557285"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1007\/BF01106578"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhl152"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1007\/BF02418571"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2008.01.003"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-012-5278-7"},{"key":"B33","first-page":"713","volume-title":"Advances in Neural Information Processing Systems","author":"Klampfl S.","year":"2006"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2008.01-07-432"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-neuro-062111-150512"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000180"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1989.1.3.402"},{"key":"B38","volume-title":"Information theory, inference, and learning algorithms","author":"MacKay D. J. C.","year":"2003"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1016\/S0899-8256(02)00014-3"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1006\/game.1995.1023"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1056562461"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1111\/j.1460-9568.2012.08074.x"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1196\/annals.1390.022"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1007\/BF00275687"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.2991\/agi.2010.10"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22887-2_28"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2012.0683"},{"issue":"2","key":"B50","first-page":"269","volume":"2","author":"Ortega P. A.","year":"2014","journal-title":"Complex Adaptive Systems Modeling"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907488"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2009.04-06-184"},{"key":"B53","volume-title":"Proceedings of the National Conference on Articial Intelligence.","author":"Peters J.","year":"2010"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2008.02.003"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.6.1318"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2012.VIII.045"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24647-0_3"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1038\/nn2066"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87536-9_40"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.12.004"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"B62","first-page":"142","volume":"7","author":"Shannon C. E.","year":"1959","journal-title":"Institute of Radio Engineers, International Convention Record"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1037\/h0042769"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1016\/S0304-3932(03)00029-1"},{"key":"B65","volume-title":"Proceedings of the Deutsche Bundesbank Spring Conference.","author":"Sims C. A.","year":"2005"},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.1257\/000282806777212431"},{"key":"B67","volume-title":"Handbook of monetary economics","author":"Sims C. A.","year":"2011"},{"key":"B68","first-page":"1741","volume-title":"Advances in Neural Information Processing Systems","author":"Sprekeler H.","year":"2009"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/85\/28005"},{"key":"B70","first-page":"17","volume-title":"Proceeedings of the Workshop on Information Theoretic Methods in Science and Engineering","author":"Still S.","year":"2014"},{"key":"B71","volume-title":"Reinforcement learning: An introduction.","author":"Sutton R. S.","year":"1998"},{"key":"B73","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-1452-1_19"},{"key":"B74","first-page":"1369","volume-title":"Advances in Neural Information Processing Systems, 19","author":"Todorov E.","year":"2006"},{"key":"B75","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0710743106"},{"key":"B76","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0500495102"},{"key":"B77","doi-asserted-by":"publisher","DOI":"10.1038\/nrn1327"},{"key":"B78","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992696"},{"key":"B79","volume-title":"Complex engineered systems.","author":"Wolpert D. H.","year":"2004"},{"key":"B80","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.041909"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00758","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:40:55Z","timestamp":1615585255000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/27\/8\/1686-1720\/8114"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,8]]},"references-count":77,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2015,8]]}},"alternative-id":["10.1162\/NECO_a_00758"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00758","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,8]]}}}