{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T12:15:50Z","timestamp":1782476150221,"version":"3.54.5"},"reference-count":75,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:p> Sensorimotor tasks that humans perform are often affected by different sources of uncertainty. Nevertheless, the central nervous system (CNS) can gracefully coordinate our movements. Most learning frameworks rely on the internal model principle, which requires a precise internal representation in the CNS to predict the outcomes of our motor commands. However, learning a perfect internal model in a complex environment over a short period of time is a nontrivial problem. Indeed, achieving proficient motor skills may require years of training for some difficult tasks. Internal models alone may not be adequate to explain the motor adaptation behavior during the early phase of learning. Recent studies investigating the active regulation of motor variability, the presence of suboptimal inference, and model-free learning have challenged some of the traditional viewpoints on the sensorimotor learning mechanism. As a result, it may be necessary to develop a computational framework that can account for these new phenomena. Here, we develop a novel theory of motor learning, based on model-free adaptive optimal control, which can bypass some of the difficulties in existing theories. This new theory is based on our recently developed adaptive dynamic programming (ADP) and robust ADP (RADP) methods and is especially useful for accounting for motor learning behavior when an internal model is inaccurate or unavailable. Our preliminary computational results are in line with experimental observations reported in the literature and can account for some phenomena that are inexplicable using existing models. <\/jats:p>","DOI":"10.1162\/neco_a_01260","type":"journal-article","created":{"date-parts":[[2020,1,17]],"date-time":"2020-01-17T22:33:43Z","timestamp":1579300423000},"page":"562-595","source":"Crossref","is-referenced-by-count":33,"title":["Model-Free Robust Optimal Feedback Mechanisms of Biological Motor Control"],"prefix":"10.1162","volume":"32","author":[{"given":"Tao","family":"Bian","sequence":"first","affiliation":[{"name":"Control and Networks Lab, Department of Electrical and Computer Engineering, Tandon School of Engineering, New York University, Brooklyn, NY 11201, U.S.A."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel M.","family":"Wolpert","sequence":"additional","affiliation":[{"name":"Zuckerman Mind Brain Behavior Institute, Department of Neuroscience, Columbia University, New York, NY 10027, U.S.A., and Department of Engineering, University of Cambridge, Cambridge CB2 1PZ, U.K."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhong-Ping","family":"Jiang","sequence":"additional","affiliation":[{"name":"Control and Networks Lab, Department of Electrical and Computer Engineering, Tandon School of Engineering, New York University, Brooklyn, NY 11201, U.S.A."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003661"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0159-16.2016"},{"key":"B3","volume-title":"Stochastic differential equations: Theory and applications","author":"Arnold L.","year":"1974"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1038\/nrn3289"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2012.03.016"},{"key":"B6","volume-title":"Dynamic programming","author":"Bellman R. E.","year":"1957"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2752"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2015.2503980"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1007\/s004220050543"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2016.05.003"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysconle.2018.03.001"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2019.1911390"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2014.08.023"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2016.2550518"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1038\/35106566"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-005-0025-9"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00901.2014"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3001-13.2014"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0770-19.2019"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(02)00963-7"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015961"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1162\/089976602753712972"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2258"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.05-07-01688.1985"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/s00221-003-1443-3"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1014269108"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-5465-6_1"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1038\/29528"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00404.2005"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2018.2840727"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1126\/science.1253138"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2011.04.012"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2015.03.003"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5359-07.2008"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1007\/s00422-014-0613-7"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1007\/s11424-015-3310-2"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1002\/9781119132677"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejcon.2013.05.017"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2018.09.001"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1016\/S0959-4388(99)00028-8"},{"key":"B41","volume-title":"Stochastic stability and control","author":"Kushner H. J.","year":"1967"},{"key":"B42","volume-title":"Reinforcement learning and approximate dynamic programming for feedback control","author":"Lewis F. L.","year":"2013"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/MCS.2012.2214134"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2015.03.008"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.1110-06.2007"},{"key":"B46","volume-title":"Nonlinear control of dynamic networks","author":"Liu T.","year":"2014"},{"key":"B47","unstructured":"Minsky, M. L. (1954). Theory of neural-analog reinforcement systems and its application to the brain model problem. PhD diss. Princeton University."},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1007\/BF00236911"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3244-14.2015"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1016\/0301-0082(89)90004-X"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00410"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2357"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2014.02.013"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2017.01.014"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.14-05-03208.1994"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262016964.001.0001"},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.0040179"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115009"},{"key":"B59","volume-title":"Reinforcement learning: An introduction","author":"Sutton R. S.","year":"2018","edition":"2"},{"key":"B60","first-page":"399","volume-title":"Proceedings of the 40th IEEE Conference on Decision and Control","author":"Tang C.","year":"2001"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-444-63356-9.00009-1"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1038\/nn1309"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1162\/0899766053491887"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1038\/nn963"},{"key":"B65","doi-asserted-by":"publisher","DOI":"10.1007\/BF00204593"},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2311-07.2007"},{"key":"B67","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2002.1101"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2656-14.2015"},{"key":"B69","volume-title":"Optimal adaptive control and differential games by reinforcement learning principles","author":"Vrabie D.","year":"2013"},{"key":"B70","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2017.2712188"},{"key":"B71","doi-asserted-by":"publisher","DOI":"10.1038\/81497"},{"key":"B72","doi-asserted-by":"publisher","DOI":"10.1126\/science.7569931"},{"key":"B73","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3616"},{"key":"B74","doi-asserted-by":"publisher","DOI":"10.1152\/jn.90529.2008"},{"key":"B75","doi-asserted-by":"publisher","DOI":"10.1080\/23307706.2015.1129295"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01260","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:43:31Z","timestamp":1615585411000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/32\/3\/562-595\/95582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3]]},"references-count":75,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,3]]}},"alternative-id":["10.1162\/neco_a_01260"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01260","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3]]}}}