{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T17:32:37Z","timestamp":1740159157474,"version":"3.37.3"},"reference-count":15,"publisher":"Walter de Gruyter GmbH","issue":"4","license":[{"start":{"date-parts":[[2012,1,1]],"date-time":"2012-01-01T00:00:00Z","timestamp":1325376000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2012,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The implementation of sequence learning in robotic platforms offers several challenges. Deciding when to stop one action and continue to the next requires a balance between stability of sensory information and, of course, the knowledge about what action is required next. The work presented here proposes a starting point for the successful execution and learning of dynamic sequences. Making use of the NAO humanoid platform we propose a mathematical model based on dynamic field theory and reinforcement learning methods for obtaining and performing a sequence of elementary motor behaviors. Results from the comparison of two reinforcement learning methods applied to sequence generation, for both simulation and implementation, are provided.<\/jats:p>","DOI":"10.2478\/s13230-013-0109-5","type":"journal-article","created":{"date-parts":[[2013,5,1]],"date-time":"2013-05-01T18:35:12Z","timestamp":1367433312000},"source":"Crossref","is-referenced-by-count":1,"title":["Learning a DFT-based sequence with reinforcement learning: a NAO implementation"],"prefix":"10.2478","volume":"3","author":[{"given":"Boris","family":"Dur\u00e1n","sequence":"first","affiliation":[]},{"given":"Gauss","family":"Lee","sequence":"additional","affiliation":[]},{"given":"Robert","family":"Lowe","sequence":"additional","affiliation":[]}],"member":"374","reference":[{"key":"109_CR1","first-page":"77","volume":"27","author":"S Amari","year":"1977","unstructured":"S. Amari, \u201cDynamics of pattern formation in lateral-inhibition type neural fields,\u201d BiologicalCybernetics, vol. 27, pp. 77\u201387, 1977.","journal-title":"BiologicalCybernetics"},{"key":"109_CR2","first-page":"101","volume-title":"Cambridge Handbok of Computational CognitiveModeling","author":"G Sch\u00f6ner","year":"2008","unstructured":"G. Sch\u00f6ner, Cambridge Handbok of Computational CognitiveModeling. R. Sun, UK: Cambridge University Press, 2008, ch. Dynamical systems approaches to cognition, pp. 101\u2013126."},{"issue":"5","key":"109_CR3","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1177\/02783640022066950","volume":"19","author":"E Bicho","year":"2000","unstructured":"E. Bicho, P. Mallet, and G. Schoner, \u201cTarget representation on an autonomous vehicle with low-level sensors.\u201d The International Journal of Robotics Research, vol. 19, no. 5, pp. 424\u2013447, May 2000. [Online]. Available: http:\/\/dx.doi.org\/10.1177\/02783640022066950","journal-title":"The International Journal of Robotics Research"},{"key":"109_CR4","doi-asserted-by":"crossref","DOI":"10.1109\/DEVLRN.2007.4354022","volume-title":"On the development of intention understanding for joint action tasks","author":"W Erlhagen","year":"2007","unstructured":"W. Erlhagen, A. Mukovskiy, F. Chersi, and E. Bicho, \u201cOn the development of intention understanding for joint action tasks,\u201d 2007."},{"issue":"10","key":"109_CR5","doi-asserted-by":"crossref","first-page":"1164","DOI":"10.1016\/j.neunet.2010.07.012","volume":"23","author":"Y Sandamirskaya","year":"2010","unstructured":"Y. Sandamirskaya and G. Sch\u00f6ner, \u201cAn embodied account of serial order: How instabilities drive sequence generation,\u201d Neural Networks, vol. 23, no. 10, pp. 1164\u20131179, December 2010.","journal-title":"Neural Networks"},{"key":"109_CR6","volume-title":"Front. Comput. Neurosci.: Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meting,BC11, no. 0","author":"Y Sandamirskaya","year":"2011","unstructured":"Y. Sandamirskaya, M. Richter, and G. Sch\u00f6ner, \u201cNeural dynamics of sequence generation and behavioral organization,\u201d in Front. Comput. Neurosci.: Computational Neuroscience & Neurotechnology Bernstein Conference & Neurex Annual Meting,BC11, no. 0, 2011."},{"key":"109_CR7","volume-title":"Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning)","author":"R S Sutton","year":"1998","unstructured":"R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). The MIT Press, Mar. 1998. [Online]. Available: http:\/\/www.amazon.com\/exec\/obidos\/redirect?tag=citeulike07-20&path=ASIN\/0262193981"},{"key":"109_CR8","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1007\/s002210050467","volume":"121","author":"R E Suri","year":"1998","unstructured":"R. E. Suri and W. Schultz, \u201cLearning of sequential movements by neural network model with dopamine-like reinforcement signal,\u201d Experimental Brain Research, vol. 121, pp. 350\u2013354, 1998, 10.1007\/s002210050467. [Online]. Available: http:\/\/dx.doi.org\/10.1007\/s002210050467","journal-title":"Experimental Brain Research"},{"key":"109_CR9","volume-title":"CoRR","author":"J Modayil","year":"2011","unstructured":"J. Modayil, A. White, and R. S. Sutton, \u201cMulti-timescale nexting in a reinforcement learning robot,\u201d CoRR, vol. abs\/1112.1133, 2011."},{"key":"109_CR10","volume-title":"Development and Learning, 2010. ICDL 2010. 9th IEEE InternationalConferenceon","author":"Y Sandamirskaya","year":"2010","unstructured":"Y. Sandamirskaya and G. Sch\u00f6ner, \u201cSerial order in an acting system: a multidimensional dynamic neural fields implementation,\u201d in Development and Learning, 2010. ICDL 2010. 9th IEEE InternationalConferenceon, 2010."},{"issue":"3","key":"109_CR11","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.jmp.2008.12.005","volume":"53","author":"Y Niv","year":"2009","unstructured":"Y. Niv, \u201cReinforcement learning in the brain,\u201d Journal of Mathematical Psychology, vol. 53, no. 3, pp. 139\u2013154, 2009. [Online]. Available: http:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0022249608001181","journal-title":"Journal of Mathematical Psychology"},{"key":"109_CR12","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-27645-3","volume-title":"Reinforcement Learning: State-Of-the-Art, ser. Adaptation, Learning, and Optimization. Springer","author":"M Wiering","year":"2012","unstructured":"M. Wiering and M. van Otterlo, Reinforcement Learning: State-Of-the-Art, ser. Adaptation, Learning, and Optimization. Springer, 2012. [Online]. Available: http:\/\/books.google.com\/books?id=YPjNuvrJR0MC"},{"key":"109_CR13","volume-title":"Dynamic Systems Approach to the Develop","author":"E Thelen","year":"1996","unstructured":"E. Thelen and L. Smith, Dynamic Systems Approach to the Develop, ser. The MIT Press\/Bradford Books series in cognitive psychology. Mit Press, 1996. [Online]. Available: http:\/\/books.google.com\/books?id=kBslxoe0TekC"},{"issue":"5","key":"109_CR14","volume":"24","year":"2001","unstructured":"J. K. O\u2019Regan and A. No\u00eb, \u201cA sensorimotor account of vision and visual consciousness.\u201d The Behavioral and brain sciences, vol. 24, no. 5, Oct. 2001. [Online]. Available: http:\/\/view.ncbi.nlm.nih.gov\/pubmed\/12239892","journal-title":"The Behavioral and brain sciences"},{"key":"109_CR15","volume-title":"CoRR","author":"S Kazerounian","year":"2012","unstructured":"S. Kazerounian, M. D. Luciw, M. Richter, and Y. Sandamirskaya, \u201cAutonomous reinforcement of behavioral sequences in neural dynamics,\u201d CoRR, vol. abs\/1210.3569, 2012."}],"container-title":["Paladyn, Journal of Behavioral Robotics"],"original-title":[],"link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.2478\/s13230-013-0109-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.2478\/s13230-013-0109-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.2478\/s13230-013-0109-5","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,28]],"date-time":"2021-02-28T16:13:51Z","timestamp":1614528831000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.degruyter.com\/document\/doi\/10.2478\/s13230-013-0109-5\/html"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1,1]]},"references-count":15,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.2478\/s13230-013-0109-5","relation":{},"ISSN":["2081-4836"],"issn-type":[{"type":"electronic","value":"2081-4836"}],"subject":[],"published":{"date-parts":[[2012,1,1]]}}}