{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T07:21:57Z","timestamp":1772781717108,"version":"3.50.1"},"reference-count":56,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2020,9,11]],"date-time":"2020-09-11T00:00:00Z","timestamp":1599782400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ICA"],"published-print":{"date-parts":[[2020,9,11]]},"abstract":"<jats:p>Designing robots has usually implied knowing beforehand the tasks to be carried out and in what domains. However, in the case of fully autonomous robots this is not possible. Autonomous robots need to operate in an open-ended manner, that is, deciding on the most interesting goals to achieve in domains that are not known at design time. This obviously poses a challenge from the point of view of designing the robot control structure. In particular, the main question that arises is how to endow the robot with a designer defined purpose and with means to translate that purpose into operational decisions without any knowledge of what situations the robot will find itself in. In this paper, we provide a formalization of motivation from an engineering perspective that allows for the structured design of purposeful robots. This formalization is based on a definition of the concepts of robot needs and drives, which are related through experience to the appropriate goals in specific domains. To illustrate the process, a motivational system to guide the operation of a real robot is constructed using this approach. A series of experiments carried out over it are discussed providing some insights on the design of purposeful motivated operation.<\/jats:p>","DOI":"10.3233\/ica-200633","type":"journal-article","created":{"date-parts":[[2020,6,2]],"date-time":"2020-06-02T16:58:19Z","timestamp":1591117099000},"page":"353-372","source":"Crossref","is-referenced-by-count":18,"title":["Motivation as a tool for designing lifelong learning robots"],"prefix":"10.1177","volume":"27","author":[{"given":"Alejandro","family":"Romero","sequence":"first","affiliation":[]},{"given":"Francisco","family":"Bellas","sequence":"additional","affiliation":[]},{"given":"Jos\u00e9 A.","family":"Becerra","sequence":"additional","affiliation":[]},{"given":"Richard J.","family":"Duro","sequence":"additional","affiliation":[]}],"member":"179","reference":[{"issue":"8","key":"10.3233\/ICA-200633_ref1","doi-asserted-by":"crossref","first-page":"1850018","DOI":"10.1142\/S0129065718500181","article-title":"Control of a 7-DOF Robotic Arm System with an SSVEP-Based BCI","volume":"28","author":"Chen","year":"2018","journal-title":"Int J Neural Syst."},{"key":"10.3233\/ICA-200633_ref2","doi-asserted-by":"crossref","first-page":"15","DOI":"10.3233\/ICA-150503","article-title":"Design and implementation of membrane controllers for trajectory tracking of nonholonomic wheeled mobile robots","volume":"23","author":"Wang","year":"2016","journal-title":"Integr Comput Aided Eng."},{"issue":"3","key":"10.3233\/ICA-200633_ref3","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/S0921-8890(97)00081-X","article-title":"A rapidly deployable manipulator system","volume":"21","author":"Paredis","year":"1997","journal-title":"Rob Auton Syst."},{"issue":"1","key":"10.3233\/ICA-200633_ref4","doi-asserted-by":"crossref","first-page":"97","DOI":"10.3233\/ICA-180585","article-title":"Speech gestural interpretation by applying word representations in robotics","volume":"26","author":"Almagro-Cadiz","year":"2019","journal-title":"Integr Comput Aided Eng."},{"key":"10.3233\/ICA-200633_ref5","doi-asserted-by":"crossref","first-page":"81","DOI":"10.3233\/ICA-170558","article-title":"Estimation of robot execution time for close proximity human-robot collaboration","volume":"25","author":"Pellegrinelli","year":"2018","journal-title":"Integr Comput Aided Eng."},{"key":"10.3233\/ICA-200633_ref6","doi-asserted-by":"crossref","unstructured":"Caruana R, Niculescu-Mizil A. An empirical comparison of supervised learning algorithms. In: Proceedings of the 23rd international conference on Machine learning. 2006, pp. 161-8.","DOI":"10.1145\/1143844.1143865"},{"key":"10.3233\/ICA-200633_ref7","unstructured":"Sutton RS, Barto AG. Introduction to Reinforcement Learning. Learning [Internet]. 1998; 4(1996): 1-5. Available from: lhttp\/\/dl.acm.org\/citation.cfm?id=551283."},{"issue":"1","key":"10.3233\/ICA-200633_ref8","doi-asserted-by":"crossref","first-page":"15","DOI":"10.3233\/ICA-170554","article-title":"A distributed approach to the control of multi-robot systems using XP colonies","volume":"25","author":"Florea","year":"2018","journal-title":"Integr Comput Aided Eng."},{"issue":"2","key":"10.3233\/ICA-200633_ref9","doi-asserted-by":"crossref","first-page":"173","DOI":"10.3233\/ICA-180569","article-title":"Ontology Based Design, Control and Programming of Modular Robots","volume":"25","author":"Ramos","year":"2018","journal-title":"Integr Comput Aided Eng."},{"key":"10.3233\/ICA-200633_ref10","doi-asserted-by":"crossref","unstructured":"Doncieux S, Filliat D, Diaz-Rodriguez N, Hospedales T, Duro R, Coninx A, et al. Open-ended learning: A conceptual framework based on representational redescription. Front Neurorobot. 2018; 12.","DOI":"10.3389\/fnbot.2018.00059"},{"issue":"1-2","key":"10.3233\/ICA-200633_ref11","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/0921-8890(95)00004-Y","article-title":"Lifelong robot learning","volume":"15","author":"Thrun","year":"1995","journal-title":"Rob Auton Syst."},{"key":"10.3233\/ICA-200633_ref12","first-page":"599","article-title":"Autonomous mental development in robots and animals","volume":"2001","author":"Weng","journal-title":"Science (80-)"},{"issue":"1","key":"10.3233\/ICA-200633_ref13","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/TAMD.2009.2021702","article-title":"Cognitive Developmental Robotics: A Survey","volume":"1","author":"Asada","year":"2009","journal-title":"IEEE Trans Auton Ment Dev [Internet]."},{"key":"10.3233\/ICA-200633_ref14","first-page":"185","article-title":"Cognitive developmental robotics as a new paradigm for the design of humanoid robots","volume":"37","author":"Asada","year":"2001","journal-title":"Robot [Internet]."},{"key":"10.3233\/ICA-200633_ref15","doi-asserted-by":"crossref","unstructured":"Piaget J, Cook M. The origins of intelligence in children. New York: New York: International Universities Press, 1952.","DOI":"10.1037\/11494-000"},{"issue":"4","key":"10.3233\/ICA-200633_ref16","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1080\/00222895.1983.10735305","article-title":"The organization of spontaneous leg movements in newborn infants","volume":"15","author":"Thelen","year":"1983","journal-title":"J Mot Behav."},{"key":"10.3233\/ICA-200633_ref17","unstructured":"Hull CL. Principles of Behavior. New York: Appleton-Century-Crofts, 1943."},{"issue":"13","key":"10.3233\/ICA-200633_ref18","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1037\/h0054346","article-title":"A theory of human motivation","volume":"50","author":"Maslow","year":"1943","journal-title":"Psychol Rev."},{"key":"10.3233\/ICA-200633_ref19","first-page":"13","article-title":"Goal setting theory","author":"Locke","year":"1994","journal-title":"Motiv Theory Res."},{"issue":"5","key":"10.3233\/ICA-200633_ref20","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1037\/h0040934","article-title":"Motivation reconsidered: the concept of competence","volume":"66","author":"White","year":"1959","journal-title":"Psychol Rev."},{"key":"10.3233\/ICA-200633_ref21","first-page":"1","article-title":"Intrinsically motivated learning in natural and artificial systems","author":"Baldassarre","year":"2013","journal-title":"Intrinsically Motivated Learning Systems: an Overview [Internet]"},{"key":"10.3233\/ICA-200633_ref22","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/978-3-642-32375-1_13","article-title":"Intrinsically Motivated Learning of Real-World Sensorimotor Skills with Developmental Constraints","author":"Oudeyer","year":"2013","journal-title":"Intrinsically Motiv Learn Nat Artif Syst [Internet]."},{"issue":"2","key":"10.3233\/ICA-200633_ref23","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1109\/TAMD.2010.2051031","article-title":"Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective","volume":"2","author":"Singh","year":"2010","journal-title":"IEEE Trans Auton Ment Dev [Internet]."},{"key":"10.3233\/ICA-200633_ref24","unstructured":"Oudeyer P, Hafner VV, Whyte A. The Playground Experiment: Task-Independent Development of a Curious Robot. In: Proc AAAI Spring Symposium on Developmental Robotics. 2005, pp. 42-7."},{"key":"10.3233\/ICA-200633_ref25","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.neunet.2012.12.012","article-title":"Phasic dopamine as a prediction error of intrinsic and extrinsic reinforcements driving both action acquisition and reward maximization: A simulated robotic study","volume":"39","author":"Mirolli","year":"2013","journal-title":"Neural Networks."},{"issue":"1","key":"10.3233\/ICA-200633_ref26","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1006\/ceps.1999.1015","article-title":"Expectancy-value theory of achivement motivation","volume":"25","author":"Wigfield","year":"2000","journal-title":"Contemp Educ Psychol [Internet]."},{"issue":"1","key":"10.3233\/ICA-200633_ref27","doi-asserted-by":"crossref","first-page":"3","DOI":"10.3233\/ICA-180578","article-title":"Introducing Separable Utility Regions in a Motivational Engine for Cognitive Developmental Robotics","volume":"26","author":"Prieto","year":"2019","journal-title":"Integr Comput Aided Eng."},{"key":"10.3233\/ICA-200633_ref28","unstructured":"Romero A, Bellas F, Becerra JA, Duro RJ. Bootstrapping Autonomous Skill Learning in the MDB Cognitive Architecture. In: Ferrandez Vicente JM, Alvarez Sanchez JR, De la Paz Lopez F, Toledo Moreo J, Adeli H, editors. Ferrandez Vicente JM, Alvarez-Sanchez JR, de la Paz Lopez F, Toledo Moreo J, Adeli H, Eds, Understanding the Brain Function and Emotions, Proceedings of the 8th International Work-Conference on the Interplay Between Natural and Artificial. Springer, 2019, pp. 120-9."},{"key":"10.3233\/ICA-200633_ref29","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/978-3-642-32375-1_2","article-title":"Intrinsic motivation and reinforcement learning","author":"Barto","year":"2013","journal-title":"Intrinsically Motivated Learning in Natural and Artificial Systems"},{"key":"10.3233\/ICA-200633_ref30","unstructured":"Huang X, Weng J. Value system development for a robot. IEEE Int Conf Neural Networks \u2013 Conf Proc. 2004; 4: 2883-8."},{"issue":"2","key":"10.3233\/ICA-200633_ref31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TAMD.2010.2051435","article-title":"A comparative study of value systems for self-motivated exploration and learning by robots","volume":"2","author":"Merrick","year":"2010","journal-title":"IEEE Trans Auton Ment Dev."},{"key":"10.3233\/ICA-200633_ref32","doi-asserted-by":"crossref","first-page":"265","DOI":"10.4018\/978-1-60960-551-3.ch011","article-title":"Motivated Learning for Computational Intelligence","author":"Starzyk","year":"2011","journal-title":"Computational Modeling and Simulation of Intellect: Current State and Future Perspectives [Internet]"},{"issue":"5-6","key":"10.3233\/ICA-200633_ref33","doi-asserted-by":"crossref","first-page":"1020","DOI":"10.1016\/j.artint.2011.02.002","article-title":"A survey of motivation frameworks for intelligent systems","volume":"175","author":"Hawes","year":"2011","journal-title":"Artif Intell."},{"key":"10.3233\/ICA-200633_ref34","doi-asserted-by":"crossref","unstructured":"Rolf M, Asada M. \u201cWhat are goals? And if so, how many\u201d, In: Proc Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob). 2015, pp. 332-9.","DOI":"10.1109\/DEVLRN.2015.7346167"},{"issue":"6","key":"10.3233\/ICA-200633_ref35","doi-asserted-by":"crossref","first-page":"1850053","DOI":"10.1142\/S0129065718500533","article-title":"Perceptual generalization and context in a network memory inspired long term memory for artificial cognition","volume":"29","author":"Duro","year":"2019","journal-title":"Int J Neural Syst."},{"key":"10.3233\/ICA-200633_ref36","first-page":"1633","article-title":"Transfer learning for reinforcement learning domains: A Survey","volume":"10","author":"Taylor","year":"2009","journal-title":"J Mach Learn Res [Internet]."},{"issue":"1","key":"10.3233\/ICA-200633_ref37","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1006\/ceps.1999.1020","article-title":"Intrinsic and extrinsic motivations: Classic definitions and new directions","volume":"25","author":"Ryan","year":"2000","journal-title":"Contemp Educ Psychol [Internet]."},{"key":"10.3233\/ICA-200633_ref38","doi-asserted-by":"crossref","first-page":"273","DOI":"10.3389\/fpsyg.2014.00273","article-title":"The role of intrinsic motivations in attention allocation and shifting","volume":"5","author":"Nocera","year":"2014","journal-title":"Front Psychol."},{"key":"10.3233\/ICA-200633_ref39","doi-asserted-by":"crossref","unstructured":"Ryan R, Deci E. Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. Contemp Educ Psychol [Internet]. 2000; 25(1): 54-67. Available from: http\/\/www.ncbi.nlm.nih.gov\/pubmed\/10620381.","DOI":"10.1006\/ceps.1999.1020"},{"issue":"4","key":"10.3233\/ICA-200633_ref40","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1037\/h0058114","article-title":"Learning and satiation of response in intrinsically motivated complex puzzle performance by monkeys","volume":"43","author":"Harlow","year":"1950","journal-title":"J Comp Physiol Psychol."},{"key":"10.3233\/ICA-200633_ref41","doi-asserted-by":"crossref","first-page":"985","DOI":"10.3389\/fpsyg.2014.00985","article-title":"Intrinsic motivations and open-ended development in animals, humans, and robots: An overview","volume":"5","author":"Baldassarre","year":"2014","journal-title":"Front Psychol."},{"key":"10.3233\/ICA-200633_ref42","first-page":"6","article-title":"What is intrinsic motivation? A typology of computational approaches","volume":"1","author":"Oudeyer","year":"2009","journal-title":"Front Neurorobot."},{"key":"10.3233\/ICA-200633_ref43","unstructured":"Huang X, Weng J. Novelty and Reinforcement Learning in the Value System of Developmental Robots. In: Proceedings Second International Workshop on Epigenetic Robotics [Internet]. 2002, pp. 47-55. Available from: http\/\/cogprints.org\/2511\/."},{"key":"10.3233\/ICA-200633_ref44","unstructured":"Burda Y, Edwards H, Pathak D, Storkey A, Darrel T, Efros A. Large-scale study of curiosity-driven learning. ArXiv Prepr. 2018; 1808.04355."},{"key":"10.3233\/ICA-200633_ref45","unstructured":"Forestier S, Mollard Y, Oudeyer P. Intrinsically motivated goal exploration processes with automatic curriculum learning. ArXiv Prepr. 2017; 1708.02190."},{"key":"10.3233\/ICA-200633_ref46","doi-asserted-by":"crossref","first-page":"907","DOI":"10.3389\/fpsyg.2013.00907","article-title":"Novelty or Surprise","volume":"4","author":"Barto","year":"2013","journal-title":"Front Psychol."},{"key":"10.3233\/ICA-200633_ref47","doi-asserted-by":"crossref","unstructured":"Schembri M, Mirolli M, Baldassarre G. Evolving internal reinforcers for an intrinsically motivated reinforcement-learning robot. In: 2007 IEEE 6th International Conference on Development and Learning. IEEE; 2007, pp. 282-7.","DOI":"10.1109\/DEVLRN.2007.4354052"},{"key":"10.3233\/ICA-200633_ref48","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.robot.2015.02.006","article-title":"Intrinsically motivated learning systems based on biologically-inspired novelty detection","volume":"68","author":"Gatsoulis","year":"2015","journal-title":"Rob Auton Syst."},{"key":"10.3233\/ICA-200633_ref49","unstructured":"Berseth G, Geng D, Devin C, Finn C, Jayaraman D, Levine S. SMiRL: Surprise Minimizing RL inDynamic Environment. ArXiv: 191205510v1, 2019."},{"key":"10.3233\/ICA-200633_ref50","doi-asserted-by":"crossref","unstructured":"Menache I, Mannor S, Shimkin N. Q-Cut \u2013 Dynamic Discovery of Sub-Goals in Reinforcement Learning. In: Proceedings of the 13th European Conference on Machine Learning, 2002, pp. 295-306.","DOI":"10.1007\/3-540-36755-1_25"},{"key":"10.3233\/ICA-200633_ref51","unstructured":"Redmon J, Farhadi A. YOLOv3: An Incremental Improvement [Internet]. Pre-trained model available: https:\/\/pjreddie.com\/darknet\/yolo. 2018; Available from: https\/\/pjreddie.com\/media\/files\/papers\/YOLOv3.pdf."},{"key":"10.3233\/ICA-200633_ref52","doi-asserted-by":"crossref","unstructured":"Schroff F, Kalenichenko D, Philbin J. FaceNet: A unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Pre-trained model available at: https\/\/github.com\/davidsandberg\/facenet: Boston, MA; 2015, pp. 815-23.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"10.3233\/ICA-200633_ref53","unstructured":"Borba F. Tutorial: Using Deep Learning and CNNs to make a Hand Gesture recognition model [Internet]. Towards data Science. 2019, Available from: https:\/\/towardsdatascience.com\/tutorial-using-deep-learning-and-cnns-to-make-a-hand-gesture-recognition-model-371770b63a51. Pre-trained model available: https\/\/github.com\/filipefborba\/HandRecognition."},{"key":"10.3233\/ICA-200633_ref54","first-page":"1","article-title":"Motivational engine with autonomous sub-goal identification for the Multilevel Darwinist Brain","volume":"17","author":"Salgado","year":"2016","journal-title":"Biol Inspired Cogn Archit."},{"key":"10.3233\/ICA-200633_ref55","unstructured":"Kingma D, Ba J. ADAM: A method for stochastic optimization. ArXiv14126980; \u2013 Code available https\/\/keras.io\/optimizers\/. 2014."},{"issue":"2","key":"10.3233\/ICA-200633_ref56","doi-asserted-by":"crossref","first-page":"111","DOI":"10.3233\/ICA-180568","article-title":"Robust grasp preimages under unknown mass and friction distributions","volume":"25","author":"Price","year":"2018","journal-title":"Integr Comput Aided Eng."}],"container-title":["Integrated Computer-Aided Engineering"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/ICA-200633","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T19:27:43Z","timestamp":1741634863000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/ICA-200633"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,11]]},"references-count":56,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.3233\/ica-200633","relation":{},"ISSN":["1069-2509","1875-8835"],"issn-type":[{"value":"1069-2509","type":"print"},{"value":"1875-8835","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,11]]}}}