{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T14:38:38Z","timestamp":1778337518961,"version":"3.51.4"},"reference-count":99,"publisher":"Hindawi Limited","license":[{"start":{"date-parts":[[2012,1,1]],"date-time":"2012-01-01T00:00:00Z","timestamp":1325376000000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational and Mathematical Methods in Medicine"],"published-print":{"date-parts":[[2012]]},"abstract":"<jats:p>It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.<\/jats:p>","DOI":"10.1155\/2012\/937860","type":"journal-article","created":{"date-parts":[[2011,12,21]],"date-time":"2011-12-21T21:01:11Z","timestamp":1324501271000},"page":"1-27","source":"Crossref","is-referenced-by-count":103,"title":["Free Energy, Value, and Attractors"],"prefix":"10.1155","volume":"2012","author":[{"given":"Karl","family":"Friston","sequence":"first","affiliation":[{"name":"The Wellcome Trust Centre for Neuroimaging, UCL, Institute of Neurology, 12 Queen Square, London WC1N 3BG, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ping","family":"Ao","sequence":"additional","affiliation":[{"name":"Shanghai Center for Systems Biomedicine, Key Laboratory of Systems Biomedicine of Ministry of Education, Shanghai Jiao Tong University, Shanghai 200240, China"},{"name":"Departments of Mechanical Engineering and Physics, University of Washington, Seattle, WA 98195, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"98","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jphysparis.2006.10.001"},{"key":"2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.88.2.135"},{"key":"3","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2006.03.006"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.8.4.429"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2008.03.006"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-010-0364-z"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0006421"},{"key":"8","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1073\/pnas.36.1.48","volume":"36","year":"1950","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1016\/S1364-6613(03)00094-9"},{"issue":"5","key":"10","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1162\/neco.1995.7.5.889","volume":"7","year":"1995","journal-title":"Neural computation"},{"issue":"7","key":"11","doi-asserted-by":"crossref","first-page":"1434","DOI":"10.1364\/JOSAA.20.001434","volume":"20","year":"2003","journal-title":"Journal of the Optical Society of America A"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1038\/4580"},{"key":"13","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2005.1622"},{"key":"14","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1073\/pnas.38.8.716","volume":"38","year":"1952","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"15","doi-asserted-by":"publisher","DOI":"10.1006\/jtbi.2000.2106"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.74.011901"},{"key":"17","year":"2005"},{"key":"18","doi-asserted-by":"publisher","DOI":"10.1006\/game.1995.1023"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1007\/BF01193705"},{"key":"20","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1073\/pnas.17.2.656","volume":"17","year":"1931","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"21","first-page":"1064","volume":"20","year":"1950","journal-title":"Zhurnal Eksperimental'noi i Teoreticheskoi Fiziki"},{"key":"22","year":"1983"},{"key":"23","year":"1981"},{"issue":"5","key":"24","first-page":"054102\/1","volume":"90","year":"2003","journal-title":"Physical Review Letters"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2003.07.009"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1021\/jp055592s"},{"key":"27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000092"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1063\/1.3309017"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(99)00107-0"},{"key":"30","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1080\/00221309.1947.9918144","volume":"37","year":"1947","journal-title":"The Journal of General Psychology"},{"key":"31","year":"1977"},{"key":"32","doi-asserted-by":"crossref","first-page":"154","DOI":"10.2307\/2373052","volume":"88","year":"1966","journal-title":"American Journal of Mathematics"},{"key":"33","series-title":"Springer Series in Synergetics","year":"2004"},{"key":"34","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000211"},{"key":"35","doi-asserted-by":"publisher","DOI":"10.1016\/j.bpc.2004.12.001"},{"key":"36","doi-asserted-by":"publisher","DOI":"10.1016\/S0076-6879(09)67005-1"},{"key":"37","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0506347102"},{"key":"38","doi-asserted-by":"publisher","DOI":"10.1016\/S1673-8527(08)60093-4"},{"key":"39","first-page":"57","volume":"4","year":"1999","journal-title":"Annali di Matematica Pura ed Applicata"},{"issue":"10","key":"40","doi-asserted-by":"crossref","first-page":"1551","DOI":"10.1080\/0026897031000085173","volume":"101","year":"2003","journal-title":"Molecular Physics"},{"issue":"4","key":"41","first-page":"1285","volume":"36","year":"2006","journal-title":"Brazilian Journal of Physics"},{"key":"42","volume-title":"Autopoiesis: the organization of the living","year":"1980"},{"key":"43","year":"1972"},{"key":"45","doi-asserted-by":"publisher","DOI":"10.1049\/el:19950331"},{"key":"47","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","volume":"22","year":"1951","journal-title":"Annals of Mathematical Statistics"},{"key":"48","volume-title":"Concerning the perceptions in general","volume":"3","year":"1866\/1962"},{"issue":"24","key":"49","first-page":"279","volume":"171","year":"1968","journal-title":"Proceedings of the Royal Society of London. Series B"},{"issue":"1038","key":"50","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1098\/rstb.1980.0090","volume":"290","year":"1980","journal-title":"Philosophical transactions of the Royal Society of London Series B"},{"key":"51","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.55.090902.142005"},{"key":"52","doi-asserted-by":"publisher","DOI":"10.1016\/j.tins.2004.10.007"},{"key":"53","doi-asserted-by":"publisher","DOI":"10.1155\/2010\/621670"},{"key":"54","doi-asserted-by":"publisher","DOI":"10.1007\/BF00198477"},{"key":"55","first-page":"717","volume":"407","year":"1989","journal-title":"Journal of the American Statistical Association"},{"key":"56","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/37\/3\/L01"},{"key":"57"},{"key":"58","doi-asserted-by":"publisher","DOI":"10.1088\/0253-6102\/49\/5\/01"},{"key":"59","doi-asserted-by":"publisher","DOI":"10.1257\/aer.98.1.180"},{"key":"60","year":"2003"},{"issue":"6363","key":"61","doi-asserted-by":"crossref","first-page":"772","DOI":"10.1038\/355772a0","volume":"355","year":"1992","journal-title":"Nature"},{"key":"62","first-page":"203","volume":"213","year":"1998","journal-title":"Novartis Foundation Symposium"},{"key":"63","volume-title":"Change of genetic environment and evolution","year":"1954"},{"key":"64","volume-title":"Punctuated equilibria: an alternative to phyletic gradualism","year":"1972"},{"key":"65","year":"1975"},{"key":"66","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","year":"1951","journal-title":"Annals of Mathematical Statistics"},{"key":"67","year":"1990"},{"key":"68","volume-title":"A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement","year":"1972"},{"key":"69","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"issue":"6551","key":"70","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1038\/377725a0","volume":"377","year":"1995","journal-title":"Nature"},{"key":"71","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"72","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4522(94)90592-4"},{"key":"73","first-page":"1388","volume":"22","year":"1996","journal-title":"Society for Neuroscience"},{"key":"74","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2007.2098"},{"key":"75","doi-asserted-by":"publisher","DOI":"10.1016\/0149-7634(85)90033-8"},{"issue":"4","key":"76","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1037\/0033-295X.100.4.640","volume":"100","year":"1993","journal-title":"Psychological Review"},{"key":"77","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainresrev.2007.07.014"},{"key":"78","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00056-4"},{"key":"79","doi-asserted-by":"publisher","DOI":"10.1007\/BF02691333"},{"issue":"5","key":"80","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1017\/S0140525X01000097","volume":"24","year":"2001","journal-title":"Behavioral and Brain Sciences"},{"key":"81","doi-asserted-by":"publisher","DOI":"10.1007\/BF00198909"},{"key":"82","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroscience.2008.03.050"},{"key":"83","doi-asserted-by":"publisher","DOI":"10.1126\/science.1155564"},{"key":"84","doi-asserted-by":"publisher","DOI":"10.1007\/BF00877522"},{"key":"85","doi-asserted-by":"publisher","DOI":"10.1016\/j.mehy.2007.03.043"},{"key":"86","doi-asserted-by":"publisher","DOI":"10.1016\/j.newideapsych.2006.09.002"},{"key":"87","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-008-9000-9"},{"key":"88","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2005.1643"},{"key":"89","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijpsycho.2005.12.008"},{"key":"90","doi-asserted-by":"publisher","DOI":"10.1016\/j.jphysparis.2007.12.001"},{"key":"91","doi-asserted-by":"publisher","DOI":"10.1162\/089976603322362428"},{"key":"92","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2009.01.014"},{"key":"93","doi-asserted-by":"publisher","DOI":"10.1007\/s10339-007-0193-8"},{"key":"94","doi-asserted-by":"publisher","DOI":"10.1063\/1.1604251"},{"key":"95","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2004.05.007"},{"key":"96","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008913712526"},{"key":"97","doi-asserted-by":"crossref","first-page":"319","DOI":"10.2307\/1906982","volume":"7","year":"1939","journal-title":"Econometrica"},{"key":"98","year":"1979"},{"key":"99","year":"1995"},{"key":"100","volume-title":"Linearly-solvable Markov decision problems","volume":"19","year":"2006"},{"key":"102","first-page":"1063","volume":"5","year":"2004","journal-title":"Journal of Machine Learning Research"}],"container-title":["Computational and Mathematical Methods in Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2012\/937860.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2012\/937860.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2012\/937860.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2017,6,20]],"date-time":"2017-06-20T11:40:04Z","timestamp":1497958804000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.hindawi.com\/journals\/cmmm\/2012\/937860\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012]]},"references-count":99,"alternative-id":["937860","937860"],"URL":"https:\/\/doi.org\/10.1155\/2012\/937860","relation":{},"ISSN":["1748-670X","1748-6718"],"issn-type":[{"value":"1748-670X","type":"print"},{"value":"1748-6718","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012]]}}}