{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T07:44:34Z","timestamp":1767339874513,"version":"build-2065373602"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:00:00Z","timestamp":1712620800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universidad Carlos III"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J of Soc Robotics"],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Robots in multi-user environments require adaptation to produce personalized interactions. In these scenarios, the user\u2019s feedback leads the robots to learn from experiences and use this knowledge to generate adapted activities to the user\u2019s preferences. However, preferences are user-specific and may suffer variations, so learning is required to personalize the robot\u2019s actions to each user. Robots can obtain feedback in Human\u2013Robot Interaction by asking users their opinion about the activity (explicit feedback) or estimating it from the interaction (implicit feedback). This paper presents a Reinforcement Learning framework for social robots to personalize activity selection using the preferences and feedback obtained from the users. This paper also studies the role of user feedback in learning, and it asks whether combining explicit and implicit user feedback produces better robot adaptive behavior than considering them separately. We evaluated the system with 24 participants in a long-term experiment where they were divided into three conditions: (i) adapting the activity selection using the explicit feedback that was obtained from asking the user how much they liked the activities; (ii) using the implicit feedback obtained from interaction metrics of each activity generated from the user\u2019s actions; and (iii) combining explicit and implicit feedback. As we hypothesized, the results show that combining both feedback produces better adaptive values when correlating initial and final activity scores, overcoming the use of individual explicit and implicit feedback. We also found that the kind of user feedback does not affect the user\u2019s engagement or the number of activities carried out during the experiment.<\/jats:p>","DOI":"10.1007\/s12369-024-01124-2","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T00:01:47Z","timestamp":1712620907000},"page":"1999-2017","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Personalizing Activity Selection in Assistive Social Robots from Explicit and Implicit User Feedback"],"prefix":"10.1007","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9576-1731","authenticated-orcid":false,"given":"Marcos","family":"Maroto-G\u00f3mez","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2317-3329","authenticated-orcid":false,"given":"Mar\u00eda","family":"Malfaz","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0454-9466","authenticated-orcid":false,"given":"Jos\u00e9 Carlos","family":"Castillo","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5189-0002","authenticated-orcid":false,"given":"\u00c1lvaro","family":"Castro-Gonz\u00e1lez","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0263-6606","authenticated-orcid":false,"given":"Miguel \u00c1ngel","family":"Salichs","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,4,9]]},"reference":[{"issue":"4","key":"1124_CR1","doi-asserted-by":"publisher","first-page":"67","DOI":"10.3390\/mti3040067","volume":"3","author":"N Akalin","year":"2019","unstructured":"Akalin N, Kristoffersson A, Loutfi A (2019) The influence of feedback type in robot-assisted training. Multimodal Technol Interact 3(4):67","journal-title":"Multimodal Technol Interact"},{"issue":"4","key":"1124_CR2","doi-asserted-by":"publisher","first-page":"1292","DOI":"10.3390\/s21041292","volume":"21","author":"N Akalin","year":"2021","unstructured":"Akalin N, Loutfi A (2021) Reinforcement learning approaches in social robotics. Sensors 21(4):1292","journal-title":"Sensors"},{"key":"1124_CR3","doi-asserted-by":"publisher","first-page":"100683","DOI":"10.1016\/j.websem.2021.100683","volume":"72","author":"L Asprino","year":"2022","unstructured":"Asprino L, Ciancarini P, Nuzzolese AG, Presutti V, Russo A (2022) A reference architecture for social robots. J Web Semant 72:100683","journal-title":"J Web Semant"},{"key":"1124_CR4","doi-asserted-by":"crossref","unstructured":"Baraka K, Veloso M (2015) Adaptive interaction of persistent robots to user temporal preferences. In: International conference on social robotics. Springer, pp 61\u201371","DOI":"10.1007\/978-3-319-25554-5_7"},{"key":"1124_CR5","unstructured":"Boggess K, Chen S, Feng L, (2020) Towards personalized explanation of robot path planning via user feedback. arXiv:2011.00524"},{"issue":"3","key":"1124_CR6","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1080\/01972243.2018.1444255","volume":"34","author":"P Caleb-Solly","year":"2018","unstructured":"Caleb-Solly P, Dogramadzi S, Huijnen CA, Heuvel HVD (2018) Exploiting ability for human adaptation to facilitate improved human-robot interaction and acceptance. Inf Soc 34(3):153\u2013165","journal-title":"Inf Soc"},{"issue":"3","key":"1124_CR7","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1007\/s12369-021-00820-7","volume":"14","author":"J Ceha","year":"2022","unstructured":"Ceha J, Law E, Kuli\u0107 D, Oudeyer P-Y, Roy D (2022) Identifying functions and behaviours of social robots for in-class learning activities: Teachers\u2019 perspective. Int J Soc Robot 14(3):747\u2013761","journal-title":"Int J Soc Robot"},{"issue":"3","key":"1124_CR8","doi-asserted-by":"publisher","first-page":"692","DOI":"10.1109\/TRO.2020.2964824","volume":"36","author":"Y Che","year":"2020","unstructured":"Che Y, Okamura AM, Sadigh D (2020) Efficient and trustworthy social navigation via explicit and implicit robot\u2013human communication. IEEE Trans Robot 36(3):692\u2013707","journal-title":"IEEE Trans Robot"},{"key":"1124_CR9","doi-asserted-by":"crossref","unstructured":"Cross ES, Hortensius R, Wykowska A (2019) From social brains to social robots: applying neurocognitive insights to human\u2013robot interaction","DOI":"10.1098\/rstb.2018.0024"},{"key":"1124_CR10","doi-asserted-by":"crossref","unstructured":"Cruz F, W\u00fcppen P, Fazrie A, Weber C, Wermter S (2018) Action selection methods in a robotic reinforcement learning scenario. In: 2018 IEEE Latin American conference on computational intelligence (LA-CCI). IEEE, pp 1\u20136","DOI":"10.1109\/LA-CCI.2018.8625243"},{"issue":"12","key":"1124_CR11","doi-asserted-by":"publisher","first-page":"3440","DOI":"10.3390\/s20123440","volume":"20","author":"E Fern\u00e1ndez-Rodicio","year":"2020","unstructured":"Fern\u00e1ndez-Rodicio E, Castro-Gonz\u00e1lez \u00c1, Alonso-Mart\u00edn F, Maroto-G\u00f3mez M, Salichs M\u00c1 (2020) Modelling multimodal dialogues for social robots using communicative acts. Sensors 20(12):3440","journal-title":"Sensors"},{"issue":"5","key":"1124_CR12","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1089\/cyber.2020.0181","volume":"24","author":"J Fox","year":"2021","unstructured":"Fox J, Gambino A (2021) Relationship development with humanoid social robots: applying interpersonal theories to human\u2013robot interaction. Cyberpsychol Behav Soc Network 24(5):294\u2013299","journal-title":"Cyberpsychol Behav Soc Network"},{"key":"1124_CR13","doi-asserted-by":"crossref","unstructured":"Haas Md, Baxter P, deJong C, Krahmer E, Vogt P (2017) Exploring different types of feedback in preschooler and robot interaction. In: Proceedings of the companion of the 2017 ACM\/IEEE international conference on human\u2013robot interaction, pp 127\u2013128","DOI":"10.1145\/3029798.3038433"},{"key":"1124_CR14","doi-asserted-by":"crossref","unstructured":"Hemminahaus J, Kopp S (2017) Towards adaptive social behavior generation for assistive robots using reinforcement learning. In: 2017 12th ACM\/IEEE international conference on human\u2013robot interaction (HRI). IEEE, pp 332\u2013340","DOI":"10.1145\/2909824.3020217"},{"key":"1124_CR15","doi-asserted-by":"crossref","unstructured":"Holtz J, Biswas J (2022) Socialgym: a framework for benchmarking social robot navigation. In: 2022 IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 11246\u201311252","DOI":"10.1109\/IROS47612.2022.9982021"},{"issue":"8","key":"1124_CR16","doi-asserted-by":"publisher","first-page":"2691","DOI":"10.3390\/s18082691","volume":"18","author":"M Maroto-G\u00f3mez","year":"2018","unstructured":"Maroto-G\u00f3mez M, Castro-Gonz\u00e1lez \u00c1, Castillo JC, Malfaz M, Salichs MA (2018) A bio-inspired motivational decision making system for social robots based on the perception of the user. Sensors 18(8):2691","journal-title":"Sensors"},{"issue":"2","key":"1124_CR17","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1007\/s11257-022-09321-2","volume":"33","author":"M Maroto-G\u00f3mez","year":"2022","unstructured":"Maroto-G\u00f3mez M, Castro-Gonz\u00e1lez \u00c1, Castillo JC, Malfaz M, Salichs M\u00c1 (2022) An adaptive decision-making system supported on user preference predictions for human-robot interactive communication. User Model User-Adapted Interact 33(2):359\u2013403","journal-title":"User Model User-Adapted Interact"},{"issue":"6","key":"1124_CR18","doi-asserted-by":"publisher","first-page":"6661","DOI":"10.1007\/s40747-023-01077-5","volume":"9","author":"M Maroto-G\u00f3mez","year":"2023","unstructured":"Maroto-G\u00f3mez M, Castro-Gonz\u00e1lez \u00c1, Malfaz M, Salichs M\u00c1 (2023) A biologically inspired decision-making system for the autonomous adaptive behavior of social robots. Complex Intell Syst 9(6):6661\u20136679","journal-title":"Complex Intell Syst"},{"issue":"2","key":"1124_CR19","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1007\/s12293-023-00390-3","volume":"15","author":"M Maroto-G\u00f3mez","year":"2023","unstructured":"Maroto-G\u00f3mez M, Malfaz M, Castro-Gonz\u00e1lez \u00c1, Salichs M\u00c1 (2023) A motivational model based on artificial biological functions for the intelligent decision-making of social robots. Memetic Comput 15(2):237\u2013257","journal-title":"Memetic Comput"},{"key":"1124_CR20","doi-asserted-by":"crossref","unstructured":"Maroto-G\u00f3mez M, Villarroya SM, Malfaz M, Castro-Gonz\u00e1lez \u00c1, Castillo JC, Salichs M\u00c1 (2022) A preference learning system for the autonomous selection and personalization of entertainment activities during human\u2013robot interaction. In: 2022 IEEE international conference on development and learning (ICDL). IEEE, pp 343\u2013348","DOI":"10.1109\/ICDL53763.2022.9962204"},{"issue":"2","key":"1124_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3277903","volume":"7","author":"C Moro","year":"2018","unstructured":"Moro C, Nejat G, Mihailidis A (2018) Learning and personalizing socially assistive robot behaviors to aid with activities of daily living. ACM Trans Hum Robot Interact (THRI) 7(2):1\u201325","journal-title":"ACM Trans Hum Robot Interact (THRI)"},{"issue":"1","key":"1124_CR22","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s12369-021-00766-w","volume":"14","author":"J Nasir","year":"2022","unstructured":"Nasir J, Bruno B, Chetouani M, Dillenbourg P (2022) What if social robots look for productive engagement? Int J Soc Robot 14(1):55\u201371","journal-title":"Int J Soc Robot"},{"issue":"1","key":"1124_CR23","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1515\/pjbr-2020-0007","volume":"11","author":"S Olatunji","year":"2020","unstructured":"Olatunji S, Oron-Gilad T, Sarne-Fleischmann V, Edan Y (2020) User-centered feedback design in person-following robots for older adults. Paladyn J Behav Robot 11(1):86\u2013103","journal-title":"Paladyn J Behav Robot"},{"key":"1124_CR24","first-page":"687","volume":"33","author":"HW Park","year":"2019","unstructured":"Park HW, Grover I, Spaulding S, Gomez L, Breazeal C (2019) A model-free affective reinforcement learning approach to personalization of an autonomous social robot companion for early literacy education. Proc AAAI Conf Artif Intell 33:687\u2013694","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"1124_CR25","doi-asserted-by":"crossref","unstructured":"Ritschel H, Andr\u00e9 E (2017) Real-time robot personality adaptation based on reinforcement learning and social signals. In: Proceedings of the companion of the 2017 ACM\/IEEE international conference on human\u2013robot interaction. pp 265\u2013266","DOI":"10.1145\/3029798.3038381"},{"key":"1124_CR26","doi-asserted-by":"crossref","unstructured":"Ritschel H, Baur T, Andr\u00e9 E (2017) Adapting a robot\u2019s linguistic style based on socially-aware reinforcement learning. In: 2017 26th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 378\u2013384","DOI":"10.1109\/ROMAN.2017.8172330"},{"key":"1124_CR27","doi-asserted-by":"crossref","unstructured":"Ritschel H, Seiderer A, Janowski K, Wagner S, Andr\u00e9 E (2019) Adaptive linguistic style for an assistive robotic health companion based on explicit human feedback. In: Proceedings of the 12th ACM international conference on PErvasive technologies related to assistive environments, pp 247\u2013255","DOI":"10.1145\/3316782.3316791"},{"key":"1124_CR28","doi-asserted-by":"publisher","first-page":"713964","DOI":"10.3389\/frobt.2022.713964","volume":"9","author":"I Salhi","year":"2022","unstructured":"Salhi I, Qbadou M, Gouraguine S, Mansouri K, Lytridis C, Kaburlasos V (2022) Towards robot-assisted therapy for children with autism\u2014the ontological knowledge models and reinforcement learning-based algorithms. Front Robot AI 9:713964","journal-title":"Front Robot AI"},{"key":"1124_CR29","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1007\/s12369-020-00687-0","volume":"12","author":"MA Salichs","year":"2020","unstructured":"Salichs MA, Castro-Gonz\u00e1lez \u00c1, Salichs E, Fern\u00e1ndez-Rodicio E, Maroto-G\u00f3mez M, Gamboa-Montero JJ, Marques-Villarroya S, Castillo JC, Alonso-Mart\u00edn F, Malfaz M (2020) Mini: a new social robot for the elderly. Int J Soc Robot 12:1231\u20131249","journal-title":"Int J Soc Robot"},{"key":"1124_CR30","doi-asserted-by":"crossref","unstructured":"Schneider S, Kummert F (2017) Exploring embodiment and dueling bandit learning for preference adaptation in human\u2013robot interaction. In: 2017 26th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 1325\u20131331","DOI":"10.1109\/ROMAN.2017.8172476"},{"issue":"5","key":"1124_CR31","doi-asserted-by":"publisher","first-page":"1763","DOI":"10.1213\/ANE.0000000000002864","volume":"126","author":"P Schober","year":"2018","unstructured":"Schober P, Boer C, Schwarte LA (2018) Correlation coefficients: appropriate use and interpretation. Anesth Analg 126(5):1763\u20131768","journal-title":"Anesth Analg"},{"issue":"4","key":"1124_CR32","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1177\/0018720816644364","volume":"58","author":"TB Sheridan","year":"2016","unstructured":"Sheridan TB (2016) Human-robot interaction: status and challenges. Hum Factors 58(4):525\u2013532","journal-title":"Hum Factors"},{"issue":"4","key":"1124_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3526111","volume":"11","author":"Z Shi","year":"2022","unstructured":"Shi Z, Groechel TR, Jain S, Chima K, Rudovic O, Matari\u0107 MJ (2022) Toward personalized affect-aware socially assistive robot tutors for long-term interventions with children with autism. ACM Trans Hum Robot Interact (THRI) 11(4):1\u201328","journal-title":"ACM Trans Hum Robot Interact (THRI)"},{"key":"1124_CR34","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"issue":"2","key":"1124_CR35","doi-asserted-by":"publisher","first-page":"49","DOI":"10.3390\/technologies6020049","volume":"6","author":"K Tsiakas","year":"2018","unstructured":"Tsiakas K, Abujelala M, Makedon F (2018) Task engagement as personalization feedback for socially-assistive robots and cognitive training. Technologies 6(2):49","journal-title":"Technologies"},{"key":"1124_CR36","doi-asserted-by":"crossref","unstructured":"Van\u00a0Otterlo M, Wiering M (2012) Reinforcement learning and Markov decision processes. In: Reinforcement learning. Springer, pp 3\u201342","DOI":"10.1007\/978-3-642-27645-3_1"},{"key":"1124_CR37","doi-asserted-by":"crossref","unstructured":"Wakayama S, Ahmed N (2023) Active inference for autonomous decision-making with contextual multi-armed bandits. In: 2023 IEEE international conference on robotics and automation (ICRA). IEEE, pp 7916\u20137922","DOI":"10.1109\/ICRA48891.2023.10160593"},{"issue":"1","key":"1124_CR38","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1075\/is.18042.wan","volume":"20","author":"N Wang","year":"2019","unstructured":"Wang N, Di Nuovo A, Cangelosi A, Jones R (2019) Temporal patterns in multi-modal social interaction between elderly users and service robot. Interact Stud 20(1):4\u201324","journal-title":"Interact Stud"},{"key":"1124_CR39","doi-asserted-by":"crossref","unstructured":"Whitney D, Rosen E, MacGlashan J, Wong LL, Tellex S (2017) Reducing errors in object-fetching interactions through social feedback. In: 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1006\u20131013","DOI":"10.1109\/ICRA.2017.7989121"},{"key":"1124_CR40","doi-asserted-by":"crossref","unstructured":"Wirth C, Akrour R, Neumann G, F\u00fcrnkranz J et al (2017) A survey of preference-based reinforcement learning methods. J Mach Learn Res 18(136):1\u201346","DOI":"10.1609\/aaai.v30i1.10269"}],"container-title":["International Journal of Social Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-024-01124-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12369-024-01124-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-024-01124-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T14:54:41Z","timestamp":1761663281000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12369-024-01124-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,9]]},"references-count":40,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["1124"],"URL":"https:\/\/doi.org\/10.1007\/s12369-024-01124-2","relation":{},"ISSN":["1875-4791","1875-4805"],"issn-type":[{"type":"print","value":"1875-4791"},{"type":"electronic","value":"1875-4805"}],"subject":[],"published":{"date-parts":[[2024,4,9]]},"assertion":[{"value":"11 March 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 April 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}},{"value":"The University Carlos III of Madrid approved the Human\u2013Robot Interaction experiment, and all participants gave their consent and approved their participation","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Human Participants Disclosure"}}]}}