{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T23:22:09Z","timestamp":1773444129165,"version":"3.50.1"},"reference-count":62,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2011,9,29]],"date-time":"2011-09-29T00:00:00Z","timestamp":1317254400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int J of Soc Robotics"],"published-print":{"date-parts":[[2011,11]]},"DOI":"10.1007\/s12369-011-0113-z","type":"journal-article","created":{"date-parts":[[2011,9,28]],"date-time":"2011-09-28T12:37:34Z","timestamp":1317213454000},"page":"427-441","source":"Crossref","is-referenced-by-count":17,"title":["Learning the Selection of Actions for an Autonomous Social Robot by Reinforcement Learning Based on Motivations"],"prefix":"10.1007","volume":"3","author":[{"given":"\u00c1lvaro","family":"Castro-Gonz\u00e1lez","sequence":"first","affiliation":[]},{"given":"Mar\u00eda","family":"Malfaz","sequence":"additional","affiliation":[]},{"given":"Miguel A.","family":"Salichs","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2011,9,29]]},"reference":[{"issue":"4","key":"113_CR1","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1177\/027836499801700402","volume":"17","author":"R Alami","year":"1998","unstructured":"Alami R, Chatila R, Fleury S, Ghallab M, Ingrand F (1998) An architecture for autonomy. Int J Robot Res 17(4):315\u2013337. doi: 10.1177\/027836499801700402","journal-title":"Int J Robot Res"},{"key":"113_CR2","unstructured":"Aldewereld H (2007) Autonomy vs. conformity an institutional perspective on norms and protocols. PhD thesis"},{"key":"113_CR3","volume-title":"Proc 8th intl conference on simulation of adaptive behavior (SAB\u201904)","author":"O \u00c1vila Garc\u00eda","year":"2004","unstructured":"\u00c1vila Garc\u00eda O, Ca\u00f1amero, L (2004) Using hormonal feedback to modulate action selection in a competitive scenario. In: Proc 8th intl conference on simulation of adaptive behavior (SAB\u201904)"},{"key":"113_CR4","volume-title":"The IEEE\/RSJ international conference on intelligent robots and systems, IROS2003","author":"B Bakker","year":"2003","unstructured":"Bakker B, Zhumatiy V, Gruener G, Schmidhuber J (2003) A\u00a0robot that reinforcement-learns to identify and memorize important previous observations. In: The IEEE\/RSJ international conference on intelligent robots and systems, IROS2003"},{"key":"113_CR5","first-page":"8","volume-title":"Proceedings of the autonomy control software workshop at autonomous agents","author":"K Barber","year":"1999","unstructured":"Barber K, Martin C (1999) Agent autonomy: specification, measurement, and dynamic adjustment. In: Proceedings of the autonomy control software workshop at autonomous agents, vol\u00a01999, pp 8\u201315"},{"key":"113_CR6","first-page":"61","volume-title":"International conference on field and service robotics","author":"R Barber","year":"2001","unstructured":"Barber R, Salichs MA (2001) Mobile robot navigation based on event maps. In: International conference on field and service robotics, pp 61\u201366"},{"key":"113_CR7","first-page":"85","volume-title":"Proceedings of the 4th IFAC symposium on intelligent autonomous vehicles","author":"R Barber","year":"2002","unstructured":"Barber R, Salichs M (2002) A\u00a0new human based architecture for intelligent autonomous robots. In: Proceedings of the 4th IFAC symposium on intelligent autonomous vehicles. Elsevier, Amsterdam, pp 85\u201390"},{"issue":"3","key":"113_CR8","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1093\/cercor\/10.3.295","volume":"10","author":"A Bechara","year":"2000","unstructured":"Bechara A, Damasio H, Damasio AR (2000) Emotion decision making and the orbitofrontal cortex. Cereb Cortex (NY 1991) 10(3):295\u2013307","journal-title":"Cereb Cortex (NY 1991)"},{"key":"113_CR9","volume-title":"Autonomous robots: from biological inspiration to implementation and control","author":"G Bekey","year":"2005","unstructured":"Bekey G (2005) Autonomous robots: from biological inspiration to implementation and control. MIT Press, Cambridge"},{"key":"113_CR10","volume-title":"Emotions in humans and artifacts, chap. Emotions: meaningful mappings between the individual and its world","author":"KL Bellman","year":"2003","unstructured":"Bellman KL (2003) Emotions in humans and artifacts, chap. Emotions: meaningful mappings between the individual and its world. MIT Press, Cambridge"},{"key":"113_CR11","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1016\/j.physbeh.2004.02.004","volume":"81","author":"KC Berridge","year":"2004","unstructured":"Berridge KC (2004) Motivation concepts in behavioural neuroscience. Physiol Behav 81:179\u2013209","journal-title":"Physiol Behav"},{"key":"113_CR12","volume-title":"The 5th international conference on developmental learning (ICDL)","author":"A Bonarini","year":"2006","unstructured":"Bonarini A, Lazaric A, Restelli M, Vitali P (2006) Self-development framework for reinforcement learning agents. In: The 5th international conference on developmental learning (ICDL)"},{"issue":"1\u20132","key":"113_CR13","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/S0004-3702(00)00033-3","volume":"121","author":"C Boutilier","year":"2000","unstructured":"Boutilier C, Dearden R, Goldszmidt M (2000) Stochastic dynamic programming with factored representation. Artif Intell 121(1\u20132):49\u2013107","journal-title":"Artif Intell"},{"key":"113_CR14","first-page":"369","volume-title":"Advances in neural information processing systems","author":"J Boyan","year":"1995","unstructured":"Boyan J, Moore A (1995) Generalization in reinforcement learning: Safely approximating the value function. In: Advances in neural information processing systems, vol\u00a07. MIT Press, Cambridge, pp 369\u2013376"},{"key":"113_CR15","unstructured":"Callum A (1995) Reinforcement learning with selective perception and hidden state. Ph.D. thesis, University of Rochester, Rochester, NY"},{"key":"113_CR16","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1145\/267658.267688","volume-title":"First international symposium on autonomous agents (Agents\u201997)","author":"L Ca\u00f1amero","year":"1997","unstructured":"Ca\u00f1amero L (1997) Modeling motivations and emotions as a basis for intelligent behavior. In: First international symposium on autonomous agents (Agents\u201997). ACM Press, New York, pp 148\u2013155"},{"key":"113_CR17","doi-asserted-by":"crossref","unstructured":"Ca\u00f1amero L (2000) Designing emotions for activity selection. Tech. rep., Dept. of Computer Science Technical Report DAIMI PB 545, University of Aarhus, Denmark","DOI":"10.7146\/dpb.v29i545.7079"},{"key":"113_CR18","volume-title":"Emotions in humans and artifacts, chap. Designing emotions for activity selection in autonomous agents","author":"L Ca\u00f1amero","year":"2003","unstructured":"Ca\u00f1amero L (2003) In: Emotions in humans and artifacts, chap. Designing emotions for activity selection in autonomous agents. MIT Press, Cambridge"},{"key":"113_CR19","volume-title":"Proceedings of the international symposium on artificial intelligence, robotics, and automation in space","author":"T Estlin","year":"2001","unstructured":"Estlin T, Volpe R, Nesnas I, Mutz D, Fisher F, Engelhardt B, Chien S (2001) Decision-making in a robotic architecture for autonomy. In: Proceedings of the international symposium on artificial intelligence, robotics, and automation in space"},{"key":"113_CR20","unstructured":"Gadanho S (1999) Reinforcement learning in autonomous robots: an empirical investigation of the role of emotions. PhD thesis, University of Edinburgh"},{"key":"113_CR21","first-page":"385","volume":"4","author":"S Gadanho","year":"2003","unstructured":"Gadanho S (2003) Learning behavior-selection by emotions and cognition in a multi-goal robot task. J Mach Learn Res 4:385\u2013412","journal-title":"J Mach Learn Res"},{"key":"113_CR22","volume-title":"From animals to animats VII, proceedings of the seventh international conference on simulation of adaptive behavior (SAB\u201902)","author":"S Gadanho","year":"2002","unstructured":"Gadanho S, Custodio L (2002) Asynchronous learning by emotions and cognition. In: From animals to animats VII, proceedings of the seventh international conference on simulation of adaptive behavior (SAB\u201902), Edinburgh, UK"},{"key":"113_CR23","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/978-4-431-35873-2_26","volume-title":"Distributed autonomous robotic systems","author":"J Gancet","year":"2007","unstructured":"Gancet J, Lacroix S (2007) Embedding heterogeneous levels of decisional autonomy in multi-robot systems. In: Distributed autonomous robotic systems, vol\u00a06. Springer, Berlin, pp\u00a0263\u2013272. doi: 10.1007\/978-4-431-35873-2"},{"key":"113_CR24","doi-asserted-by":"crossref","first-page":"329","DOI":"10.3233\/ICA-2006-13403","volume":"13","author":"T Geerinck","year":"2006","unstructured":"Geerinck T, Colon E, Berrabah SA, Cauwerts K, Sahli H (2006) Tele-robot with shared autonomy: distributed navigation development framework. Integr Comput-Aided Eng 13:329\u2013345","journal-title":"Integr Comput-Aided Eng"},{"issue":"1\u20132","key":"113_CR25","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/S0004-3702(02)00376-4","volume":"147","author":"R Givan","year":"2003","unstructured":"Givan R, Dean T, Greig M (2003) Equivalence notions and model minimization in Markov decision processes. Artif Intell 147(1\u20132), 163\u2013223","journal-title":"Artif Intell"},{"key":"113_CR26","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1613\/jair.1000","volume":"19","author":"C Guestrin","year":"2003","unstructured":"Guestrin C, Koller D, Parr R, Venkataraman S (2003) Efficient solution algorithms for factored mdps. J Artif Intell Res 19:399\u2013468","journal-title":"J Artif Intell Res"},{"key":"113_CR27","volume-title":"Principles of behavior","author":"CL Hull","year":"1943","unstructured":"Hull CL (1943) Principles of behavior. Appleton Century Crofts, New York"},{"key":"113_CR28","doi-asserted-by":"crossref","unstructured":"Humphrys M (1997) Action selection methods using reinforcement learning. PhD thesis, Trinity Hall, Cambridge","DOI":"10.7551\/mitpress\/3118.003.0018"},{"key":"113_CR29","volume-title":"the fifth international conference on autonomous agents","author":"C Isbell","year":"2001","unstructured":"Isbell C, Shelton CR, Kearns M, Singh S, Stone P (2001) A\u00a0social reinforcement learning agent. In: the fifth international conference on autonomous agents, Montreal, Quebec, Canada"},{"key":"113_CR30","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","volume":"4","author":"LP Kaelbling","year":"1996","unstructured":"Kaelbling LP, Littman LM, Moore AW (1996) Reinforcement learning: a\u00a0survey. J Artif Intell Res 4:237\u2013285","journal-title":"J Artif Intell Res"},{"issue":"5","key":"113_CR31","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1016\/j.robot.2010.02.003","volume":"58","author":"T Kaupp","year":"2010","unstructured":"Kaupp T, Makarenko A, Durrant-Whyte H (2010) Human-robot communication for collaborative decision making\u2014a\u00a0probabilistic approach. Robot Auton Syst 58(5):444\u2013456. doi: 10.1016\/j.robot.2010.02.003","journal-title":"Robot Auton Syst"},{"key":"113_CR32","first-page":"531","volume-title":"Ninth international symposium on artificial intelligence and mathematics","author":"L Li","year":"2006","unstructured":"Li L, Walsh T, Littman M (2006) Towards a unified theory of state abstraction for MDP. In: Ninth international symposium on artificial intelligence and mathematics, pp 531\u2013539"},{"key":"113_CR33","volume-title":"Behind the mirror","author":"K Lorenz","year":"1977","unstructured":"Lorenz K (1977) Behind the mirror. Methuen Young Books, London. ISBN\u00a004 16942709"},{"key":"113_CR34","volume-title":"Motivation of human and animal behaviour; an ethological view","author":"K Lorenz","year":"1973","unstructured":"Lorenz K, Leyhausen P (1973) Motivation of human and animal behaviour; an ethological view, vol\u00a0XIX. Van Nostrand-Reinhold, New York"},{"key":"113_CR35","unstructured":"Malfaz M (2007) Decision making system for autonomous social agents based on emotions and self-learning. PhD thesis, Carlos III University of Madrid (2007)"},{"key":"113_CR36","volume-title":"The 8th international conference on development and learning (ICDL 2009)","author":"M Malfaz","year":"2009","unstructured":"Malfaz M, Salichs M (2009) Learning to deal with objects. In: The 8th international conference on development and learning (ICDL 2009)"},{"key":"113_CR37","volume-title":"Ninth international conference on epigenetic robotics: modeling cognitive development in robotic systems (EpiRob09)","author":"M Malfaz","year":"2009","unstructured":"Malfaz M, Salichs M (2009) The use of emotions in an autonomous agent\u2019s decision making process. In: Ninth international conference on epigenetic robotics: modeling cognitive development in robotic systems (EpiRob09), Venice, Italy"},{"issue":"1","key":"113_CR38","first-page":"21","volume":"2","author":"M Malfaz","year":"2010","unstructured":"Malfaz M, Salichs M (2010) Using muds as an experimental platform for testing a decision making system for self-motivated autonomous agents. AISB J 2(1):21\u201344","journal-title":"AISB J"},{"key":"113_CR39","doi-asserted-by":"crossref","unstructured":"Malfaz M, Castro-Gonzalez A, Barber R, Salichs M (2011) A\u00a0biologically inspired architecture for an autonomous and social robot. IEEE Trans Auton Ment Dev 3(2). doi: 10.1109\/TAMD.2011.2112766","DOI":"10.1109\/TAMD.2011.2112766"},{"key":"113_CR40","volume-title":"The IEEE\/RSJ international conference on intelligent robots and systems (IROS)","author":"E Martinson","year":"2002","unstructured":"Martinson E, Stoytchev A, Arkin R (2002) Robot behavioral selection using q-learning. In: The IEEE\/RSJ international conference on intelligent robots and systems (IROS), EPFL, Switzerland"},{"issue":"3","key":"113_CR41","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/S1364-6613(98)01141-3","volume":"2","author":"M Mataric","year":"1998","unstructured":"Mataric M (1998) Behavior-based robotics as a tool for synthesis of artificial behavior and analysis of natural behavior. Trends Cogn Sci 2(3):82\u201387","journal-title":"Trends Cogn Sci"},{"issue":"1","key":"113_CR42","doi-asserted-by":"crossref","first-page":"57","DOI":"10.2478\/s13230-010-0003-3","volume":"1","author":"F Michaud","year":"2010","unstructured":"Michaud F, Ferland F, L\u00e9tourneau D, Legault MA, Lauria M (2010) Toward autonomous, compliant, omnidirectional humanoid robots for natural interaction in real-life settings. Paladyn 1(1):57\u201365. doi: 10.2478\/s13230-010-0003-3","journal-title":"Paladyn"},{"key":"113_CR43","volume-title":"AAMAS 2002","author":"CHC Ribeiro","year":"2002","unstructured":"Ribeiro CHC, Pegoraro R, RealiCosta AH (2002) Experience generalization for concurrent reinforcement learners: the minimax-qs algorithm. In: AAMAS 2002"},{"key":"113_CR44","volume-title":"6th IFAC symposium on intelligent autonomous vehicles","author":"R Rivas","year":"2007","unstructured":"Rivas R, Corrales A, Barber R, Salichs MA (2007) Robot skill abstraction for ad architecture. In: 6th IFAC symposium on intelligent autonomous vehicles"},{"key":"113_CR45","volume-title":"IEEE international conference on robotics, automation and mechatronics (RAM)","author":"MA Salichs","year":"2006","unstructured":"Salichs MA, Barber R, Khamis MA, Malfaz M, Gorostiza FJ, Pacheco R, Rivas R, Corrales A, Delgado E (2006) Maggie: a\u00a0robotic platform for human-robot social interaction. In: IEEE international conference on robotics, automation and mechatronics (RAM), Bangkok, Thailand"},{"key":"113_CR46","volume-title":"FIRA RoboWorld congress 2009","author":"J Salichs","year":"2009","unstructured":"Salichs J, Castro-Gonzalez A, Salichs MA (2009) Infrared remote control with a social robot. In: FIRA RoboWorld congress 2009, Incheon, Korea. Springer, Berlin"},{"issue":"2","key":"113_CR47","doi-asserted-by":"crossref","first-page":"5","DOI":"10.4995\/RIAI.2010.04.03","volume":"7","author":"MA Salichs","year":"2010","unstructured":"Salichs MA, Malfaz M, Gorostiza JF (2010) Toma de decisiones en robotica. Rev Iberoam Autom Inf Ind 7(2):5\u201316","journal-title":"Rev Iberoam Autom Inf Ind"},{"key":"113_CR48","unstructured":"Santa-Cruz J, Tobal JM, Vindel AC, Fern\u00e1ndez EG (1989) Introducci\u00f3n a la psicolog\u00eda. Facultad de Psicolog\u00eda. Universidad Complutense de Madrid"},{"key":"113_CR49","volume-title":"Procceding of ICMI-MLMI 2009","author":"P Schermerhorn","year":"2009","unstructured":"Schermerhorn P, Scheutz M (2009) Dynamic robot autonomy: investigating the effects of robot decision-making in a human-robot team task. In: Procceding of ICMI-MLMI 2009, Cambridge, MA, USA. doi: 10.1145\/1647314.1647328"},{"key":"113_CR50","doi-asserted-by":"crossref","unstructured":"Scheutz M, Schermerhorn P (2009) Affective goal and task selection for social robots. Handbook of research on synthetic emotions and sociable robotics: new applications in affective computing and artificial intelligence, p\u00a074","DOI":"10.4018\/978-1-60566-354-8.ch005"},{"key":"113_CR51","volume-title":"International conference on robotics and automation (ICRA2002)","author":"WD Smart","year":"2002","unstructured":"Smart WD, Kaelbling LP (2002) Effective reinforcement learning for mobile robots. In: International conference on robotics and automation (ICRA2002)"},{"key":"113_CR52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1566445.1566489","volume-title":"ACM-SE 47: proceedings of the 47th annual southeast regional conference","author":"EB Smith","year":"2009","unstructured":"Smith EB (2009) The motion control of a mobile robot using multiobjective decision making. In: ACM-SE 47: proceedings of the 47th annual southeast regional conference. ACM Press, New York, pp 1\u20136"},{"key":"113_CR53","volume-title":"The 18th international joint conference on artificial intelligence (IJCAI-03)","author":"N Sprague","year":"2003","unstructured":"Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0). In: The 18th international joint conference on artificial intelligence (IJCAI-03), Acapulco, Mexico"},{"key":"113_CR54","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press\/Bradford Book, Cambridge"},{"key":"113_CR55","volume-title":"The 5th international conference on developmental learning (ICDL)","author":"AL Thomaz","year":"2006","unstructured":"Thomaz AL, Breazeal C (2006) Transparency and socially guided machine learning. In: The 5th international conference on developmental learning (ICDL)"},{"key":"113_CR56","first-page":"934","volume-title":"The handbook of brain theory and neural networks","author":"C Touzet","year":"2003","unstructured":"Touzet C (2003) Q-learning for robots. In: The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 934\u2013937"},{"key":"113_CR57","volume-title":"Fourteenth national conf artificial intelligence","author":"J Vel\u00e1squez","year":"1997","unstructured":"Vel\u00e1squez J (1997) Modeling emotions and other motivations in synthetic agents. In: Fourteenth national conf artificial intelligence"},{"key":"113_CR58","volume-title":"1998 AAAI fall symposium emotional and intelligent: the tangled knot of cognition","author":"J Vel\u00e1squez","year":"1998","unstructured":"Vel\u00e1squez J (1998) Modelling emotion-based decision-making. In: 1998 AAAI fall symposium emotional and intelligent: the tangled knot of cognition"},{"key":"113_CR59","volume-title":"Proceedings of AAAI-98","author":"J Vel\u00e1squez","year":"1998","unstructured":"Vel\u00e1squez J (1998) When robots weep: emotional memories and decision making. In: Proceedings of AAAI-98"},{"key":"113_CR60","unstructured":"Verhagen H (2000) Norm autonomous agents. PhD thesis, The Royal Institute of Technology and Stockholm University"},{"issue":"2","key":"113_CR61","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1109\/TAMD.2010.2050205","volume":"2","author":"C Vigorito","year":"2010","unstructured":"Vigorito C, Barto A (2010) Intrinsically motivated hierarchical skill learning in structured environment. IEEE Trans Auton Ment Dev 2(2):132\u2013143. Special Issue on Active Learning and Intrinsically Motivated Exploration in Robots","journal-title":"IEEE Trans Auton Ment Dev"},{"key":"113_CR62","unstructured":"Watkins CJ (1989) Models of delayed reinforcement learning. PhD thesis, Cambridge University, Cambridge, UK"}],"container-title":["International Journal of Social Robotics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-011-0113-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s12369-011-0113-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-011-0113-z","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,13]],"date-time":"2024-04-13T00:26:00Z","timestamp":1712967960000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s12369-011-0113-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,9,29]]},"references-count":62,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2011,11]]}},"alternative-id":["113"],"URL":"https:\/\/doi.org\/10.1007\/s12369-011-0113-z","relation":{},"ISSN":["1875-4791","1875-4805"],"issn-type":[{"value":"1875-4791","type":"print"},{"value":"1875-4805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,9,29]]}}}