{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,6]],"date-time":"2024-10-06T00:43:05Z","timestamp":1728175385842},"reference-count":46,"publisher":"World Scientific Pub Co Pte Ltd","issue":"02n03","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Advs. Complex Syst."],"published-print":{"date-parts":[[2013,5]]},"abstract":"<jats:p>Classical conditioning (conventionally modeled as correlation-based learning) and operant conditioning (conventionally modeled as reinforcement learning or reward-based learning) have been found in biological systems. Evidence shows that these two mechanisms strongly involve learning about associations. Based on these biological findings, we propose a new learning model to achieve successful control policies for artificial systems. This model combines correlation-based learning using input correlation learning (ICO learning) and reward-based learning using continuous actor\u2013critic reinforcement learning (RL), thereby working as a dual learner system. The model performance is evaluated by simulations of a cart-pole system as a dynamic motion control problem and a mobile robot system as a goal-directed behavior control problem. Results show that the model can strongly improve pole balancing control policy, i.e., it allows the controller to learn stabilizing the pole in the largest domain of initial conditions compared to the results obtained when using a single learning mechanism. This model can also find a successful control policy for goal-directed behavior, i.e., the robot can effectively learn to approach a given goal compared to its individual components. Thus, the study pursued here sharpens our understanding of how two different learning mechanisms can be combined and complement each other for solving complex tasks.<\/jats:p>","DOI":"10.1142\/s021952591350015x","type":"journal-article","created":{"date-parts":[[2013,4,29]],"date-time":"2013-04-29T06:38:25Z","timestamp":1367217505000},"page":"1350015","source":"Crossref","is-referenced-by-count":10,"title":["COMBINING CORRELATION-BASED AND REWARD-BASED LEARNING IN NEURAL CONTROL FOR POLICY IMPROVEMENT"],"prefix":"10.1142","volume":"16","author":[{"given":"PORAMATE","family":"MANOONPONG","sequence":"first","affiliation":[{"name":"Bernstein Center for Computational Neuroscience, The Third Institute of Physics, University of G\u00f6ttingen, G\u00f6ttingen 37077, Germany"},{"name":"ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai Seika-cho, Soraku-gun, Kyoto 619-0288, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"CHRISTOPH","family":"KOLODZIEJSKI","sequence":"additional","affiliation":[{"name":"Bernstein Center for Computational Neuroscience, The Third Institute of Physics, University of G\u00f6ttingen, G\u00f6ttingen 37077, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"FLORENTIN","family":"W\u00d6RG\u00d6TTER","sequence":"additional","affiliation":[{"name":"Bernstein Center for Computational Neuroscience, The Third Institute of Physics, University of G\u00f6ttingen, G\u00f6ttingen 37077, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"JUN","family":"MORIMOTO","sequence":"additional","affiliation":[{"name":"Bernstein Center for Computational Neuroscience, The Third Institute of Physics, University of G\u00f6ttingen, G\u00f6ttingen 37077, Germany"},{"name":"ATR Computational Neuroscience Laboratories, 2-2-2 Hikaridai Seika-cho, Soraku-gun, Kyoto 619-0288, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2013,7,16]]},"reference":[{"key":"rf4","doi-asserted-by":"publisher","DOI":"10.1142\/S0219525911002937"},{"key":"rf5","volume-title":"Animal Behavior: Mechanism, Development, Function, and Evolution","author":"Barnard C.","year":"2004"},{"key":"rf6","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022140919877"},{"key":"rf7","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1983.6313077"},{"key":"rf8","volume-title":"Autonomous Robots From Biological Inspiration to Implementation and Control","author":"Bekey G.","year":"2005"},{"key":"rf9","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2008.08.011"},{"key":"rf11","doi-asserted-by":"publisher","DOI":"10.1101\/lm.7.2.104"},{"key":"rf13","doi-asserted-by":"publisher","DOI":"10.1142\/S0219525998000065"},{"key":"rf15","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)00963-7"},{"key":"rf16","doi-asserted-by":"publisher","DOI":"10.1142\/S0219525911002998"},{"key":"rf17","doi-asserted-by":"publisher","DOI":"10.1613\/jair.639"},{"key":"rf18","first-page":"1012","author":"Doya K.","year":"1997","journal-title":"Advances in Neural Information Processing Systems"},{"key":"rf19","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015961"},{"key":"rf20","doi-asserted-by":"publisher","DOI":"10.1177\/0278364907084980"},{"key":"rf21","volume-title":"A Modulatory Learning Rule for Neural Learning and Metalearning in Real World Robots with Many Degees of Freedom","author":"Fischer J.","year":"2003"},{"key":"rf22","first-page":"937","volume":"9","author":"Gomez F.","year":"2008","journal-title":"J. Mach. Lear. Res."},{"key":"rf23","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(09)80004-X"},{"key":"rf24","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(90)90056-Q"},{"key":"rf25","first-page":"17","volume":"3","author":"Howery L. D.","year":"2007","journal-title":"Backyards and Beyond: Rural Living in Arizona"},{"key":"rf26","volume-title":"A Behavior System: An Introduction to Behavior Theory Concerning the Individual Organism","author":"Hull C. L.","year":"1952"},{"key":"rf27","doi-asserted-by":"publisher","DOI":"10.1038\/nature02194"},{"key":"rf31","doi-asserted-by":"publisher","DOI":"10.20965\/jrm.2010.p0542"},{"key":"rf32","doi-asserted-by":"crossref","first-page":"85","DOI":"10.3758\/BF03333113","volume":"16","author":"Klopf A. H.","year":"1988","journal-title":"Psychobiology"},{"key":"rf35","volume-title":"Integrative Activity of the Brain","author":"Konorski J.","year":"1967"},{"key":"rf39","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.9.3.225"},{"key":"rf40","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.0030134"},{"key":"rf41","doi-asserted-by":"publisher","DOI":"10.3389\/fncir.2013.00012"},{"key":"rf44","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30132-5_112"},{"key":"rf46","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(01)00113-0"},{"key":"rf47","doi-asserted-by":"publisher","DOI":"10.1037\/10802-000"},{"key":"rf49","doi-asserted-by":"publisher","DOI":"10.1088\/0954-898X_9_4_006"},{"key":"rf50","volume-title":"Conditioned Reflexes","author":"Pavlov I.","year":"1927"},{"key":"rf52","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.6.1380"},{"key":"rf53","doi-asserted-by":"publisher","DOI":"10.1016\/j.biosystems.2006.04.026"},{"key":"rf54","doi-asserted-by":"publisher","DOI":"10.1613\/jair.898"},{"key":"rf55","doi-asserted-by":"publisher","DOI":"10.1037\/h0024475"},{"key":"rf56","unstructured":"R.\u00a0Rescorla and A.\u00a0Wagner, Classical Conditioning II: Current Research and Theory (1972)\u00a0pp. 64\u201399."},{"key":"rf58","doi-asserted-by":"publisher","DOI":"10.1002\/scj.10207"},{"key":"rf59","volume-title":"The Behavior of Organisms: An Experimental Analysis","author":"Skinner B.","year":"1938"},{"key":"rf61","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4522(00)00554-6"},{"key":"rf62","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R.","year":"1998"},{"key":"rf63","first-page":"68","volume":"8","author":"Thorndike E.","year":"1898","journal-title":"Psychol. Rev. Monogr. Suppl."},{"key":"rf65","doi-asserted-by":"publisher","DOI":"10.1177\/105971239700500302"},{"key":"rf68","doi-asserted-by":"publisher","DOI":"10.1901\/jeab.1969.12-511"},{"key":"rf69","unstructured":"B. G.\u00a0Woolley and K. O.\u00a0Stanley, Parallel Problem Solving from Nature, Lecture Notes in Computer Science (2010)\u00a0pp. 270\u2013279."},{"key":"rf70","doi-asserted-by":"publisher","DOI":"10.1162\/0899766053011555"}],"container-title":["Advances in Complex Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S021952591350015X","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,14]],"date-time":"2022-02-14T23:20:24Z","timestamp":1644880824000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S021952591350015X"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,5]]},"references-count":46,"journal-issue":{"issue":"02n03","published-online":{"date-parts":[[2013,7,16]]},"published-print":{"date-parts":[[2013,5]]}},"alternative-id":["10.1142\/S021952591350015X"],"URL":"https:\/\/doi.org\/10.1142\/s021952591350015x","relation":{},"ISSN":["0219-5259","1793-6802"],"issn-type":[{"value":"0219-5259","type":"print"},{"value":"1793-6802","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,5]]}}}