{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T04:03:19Z","timestamp":1777521799567,"version":"3.51.4"},"reference-count":48,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2005,6,1]],"date-time":"2005-06-01T00:00:00Z","timestamp":1117584000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Adaptive Behavior"],"published-print":{"date-parts":[[2005,6]]},"abstract":"<jats:p>Since 1995, numerous Actor\u2013Critic architectures for reinforcement learning have been proposed as models of dopamine-like reinforcement learning mechanisms in the rat\u2019s basal ganglia. However, these models were usually tested in different tasks, and it is then difficult to compare their efficiency for an autonomous animat. We present here the comparison of four architectures in an animat as it per forms the same reward-seeking task. This will illustrate the consequences of different hypotheses about the management of different Actor sub-modules and Critic units, and their more or less autono mously determined coordination. We show that the classical method of coordination of modules by mixture of experts, depending on each module\u2019s performance, did not allow solving our task. Then we address the question of which principle should be applied efficiently to combine these units. Improve ments for Critic modeling and accuracy of Actor\u2013Critic models for a natural task are finally discussed in the perspective of our Psikharpax project\u2014an artificial rat having to survive autonomously in unpre dictable environments.<\/jats:p>","DOI":"10.1177\/105971230501300205","type":"journal-article","created":{"date-parts":[[2005,5,25]],"date-time":"2005-05-25T07:57:47Z","timestamp":1117007867000},"page":"131-148","source":"Crossref","is-referenced-by-count":58,"title":["Actor\u2013Critic Models of Reinforcement Learning in the Basal Ganglia:                 From Natural to Artificial Rats"],"prefix":"10.1177","volume":"13","author":[{"given":"Mehdi","family":"Khamassi","sequence":"first","affiliation":[{"name":"AnimatLab, LIP6, Paris, France; LPPA, CNRS\u2013Coll\u00e8ge                         de France, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lo\u00efc","family":"Lach\u00e8ze","sequence":"additional","affiliation":[{"name":"AnimatLab, LIP6, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Beno\u00eet","family":"Girard","sequence":"additional","affiliation":[{"name":"AnimatLab, LIP6, Paris, France; LPPA, CNRS\u2013Coll\u00e8ge                         de France, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alain","family":"Berthoz","sequence":"additional","affiliation":[{"name":"LPPA, CNRS\u2013Coll\u00e8ge de France, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Agn\u00e8s","family":"Guillot","sequence":"additional","affiliation":[{"name":"AnimatLab, LIP6, Paris, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2005,6,1]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1038\/72929"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1016\/S0166-4328(00)00303-X"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1016\/0166-2236(89)90074-X"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1007\/s004220000171"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1016\/S1389-0417(01)00039-0"},{"key":"atypb6","first-page":"131","volume-title":"From animals to animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior","author":"Baldassarre, G.","year":"2000"},{"key":"atypb7","first-page":"309","volume":"28","author":"Brown, J.","year":"1999","journal-title":"Brain Research Reviews"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1016\/0006-8993(95)00457-2"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1002\/syn.890090202"},{"key":"atypb10","first-page":"3","volume-title":"The hippocampal and parietal foundations of spatial cognition","author":"Burgess, N.","year":"1999"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1016\/S0149-7634(02)00007-6"},{"key":"atypb12","first-page":"11","volume-title":"Proceedings of NIPS 14","author":"Dayan., P.","year":"2001"},{"key":"atypb13","unstructured":"Daw, N. D. (2003).\n                      Reinforcement learning models of the dopamine system                     and their behavioral implications\n                      . Ph.D. thesis, Carnegie Mellon                 University, Pittsburgh, PA."},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015961"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1162\/089976602753712972"},{"key":"atypb16","first-page":"2","volume-title":"From animals to animats 8: Proceedings of the Eighth International Conference on Simulation of Adaptive Behavior","author":"Filliat, D.","year":"2004"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1098-1063(2000)10:1<1::AID-HIPO1>3.0.CO;2-1"},{"key":"atypb18","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.1.2.137"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.07-12-03915.1987"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1142\/S0219635203000299"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1177\/105971230501300204"},{"key":"atypb22","doi-asserted-by":"publisher","DOI":"10.1007\/PL00007984"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1007\/PL00007985"},{"key":"atypb24","volume-title":"Models of information processing in the basal ganglia","author":"Houk, J. C.","year":"1995"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1016\/S0165-0173(99)00023-5"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.1.79"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00047-3"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4522(99)00575-8"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/13.4.400"},{"key":"atypb30","first-page":"142","volume-title":"Proceedings of the International Conference on Control Automation and Systems, ICCAS 2003","author":"Lee, J. K."},{"key":"atypb31","first-page":"285","volume-title":"Neural Connections, Mental Computations","author":"McNaughton, B. L.","year":"1989"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996"},{"key":"atypb33","doi-asserted-by":"crossref","first-page":"157","DOI":"10.7551\/mitpress\/3120.003.0018","volume-title":"From animals to animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior","author":"Montes-Gonzalez, F.","year":"2000"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1126\/science.1094285"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1016\/0165-0173(95)00014-3"},{"key":"atypb36","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44597-8_37"},{"key":"atypb37","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1998.80.1.1"},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.13-03-00900.1993"},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"atypb40","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00083-7"},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00062-X"},{"key":"atypb42","unstructured":"Str\u00f6sslin, T. (2004).\n                      A connectionist model of spatial learning                     in the rat.\n                      Ph.D thesis, EPFL, Swiss Federal Institute of Technology."},{"key":"atypb43","doi-asserted-by":"publisher","DOI":"10.1162\/089976601300014376"},{"key":"atypb44","volume-title":"Reinforcement learning: An introduction","author":"Sutton, R. S.","year":"1998"},{"key":"atypb45","first-page":"227","volume-title":"IEEE\/INNS International Joint Conference on Neural Networks","author":"Tang, B."},{"key":"atypb46","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(99)00060-X"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1002\/1098-1063(2000)10:4<411::AID-HIPO7>3.0.CO;2-A"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1111\/j.1460-9568.2004.03095.x"}],"container-title":["Adaptive Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/105971230501300205","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/105971230501300205","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T16:15:27Z","timestamp":1777392927000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/105971230501300205"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2005,6]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2005,6]]}},"alternative-id":["10.1177\/105971230501300205"],"URL":"https:\/\/doi.org\/10.1177\/105971230501300205","relation":{},"ISSN":["1059-7123","1741-2633"],"issn-type":[{"value":"1059-7123","type":"print"},{"value":"1741-2633","type":"electronic"}],"subject":[],"published":{"date-parts":[[2005,6]]}}}