{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T04:30:12Z","timestamp":1780633812908,"version":"3.54.1"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T00:00:00Z","timestamp":1694044800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T00:00:00Z","timestamp":1694044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007195","name":"Universit\u00e0 degli Studi di Napoli Federico II","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007195","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J of Soc Robotics"],"published-print":{"date-parts":[[2025,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>A robot intended to monitor human behavior must account for the user\u2019s reactions to minimize his\/her perceived discomfort. The possibility of learning user interaction preferences and changing the robot\u2019s behavior accordingly may positively impact the perceived quality of the interaction with the robot. The robot should approach the user without causing any discomfort or interference. In this work, we contribute and implement a novel Reinforcement Learning (RL) approach for robot navigation toward a human user. Our implementation is a proof-of-concept that uses data gathered from real-world experiments to show that our algorithm works on the kind of data that it would run on in a realistic scenario. To the best of our knowledge, our work is one of the first attempts to provide an adaptive navigation algorithm that uses RL to account for non-deterministic phenomena.<\/jats:p>","DOI":"10.1007\/s12369-023-01044-7","type":"journal-article","created":{"date-parts":[[2023,9,7]],"date-time":"2023-09-07T12:04:58Z","timestamp":1694088298000},"page":"903-915","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep Reinforcement Learning for Robotic Approaching Behavior Influenced by User Activity and Disengagement"],"prefix":"10.1007","volume":"17","author":[{"given":"Luca","family":"Raggioli","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio Aurelio","family":"D\u2019Asaro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3379-1756","authenticated-orcid":false,"given":"Silvia","family":"Rossi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,9,7]]},"reference":[{"key":"1044_CR1","doi-asserted-by":"crossref","unstructured":"Akalin N, Kiselev A, Kristoffersson A, Loutfi A (2018) Enhancing social human\u2013robot interaction with deep reinforcement learning. In: FAIM\/ISCA workshop on artificial intelligence for multimodal human robot interaction (AI-MHRI). MHRI, pp 48\u201350","DOI":"10.21437\/AI-MHRI.2018-12"},{"key":"1044_CR2","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s12369-009-0030-6","volume":"1","author":"E Broadbent","year":"2009","unstructured":"Broadbent E, Stafford R, MacDonald B (2009) Acceptance of healthcare robots for the older population: review and future directions. Int J Soc Robot 1:319\u2013330","journal-title":"Int J Soc Robot"},{"issue":"3","key":"1044_CR3","doi-asserted-by":"publisher","first-page":"172988141877618","DOI":"10.1177\/1729881418776183","volume":"15","author":"W Chen","year":"2018","unstructured":"Chen W, Zhang T, Zou Y (2018) Mobile robot path planning based on social interaction space in social environment. Int J Adv Robot Syst 15(3):1729881418776183","journal-title":"Int J Adv Robot Syst"},{"key":"1044_CR4","doi-asserted-by":"crossref","unstructured":"Dautenhahn K, Walters M, Woods S, Koay KL, Nehaniv CL, Sisbot A, Alami R, Sim\u00e9on T (2006) How may i serve you? A robot companion approaching a seated person in a helping context. In: Proceedings of the 1st HRI, pp 172\u2013179","DOI":"10.1145\/1121241.1121272"},{"key":"1044_CR5","doi-asserted-by":"crossref","unstructured":"Di\u00a0Napoli C, Ercolano G, Rossi S (2022) Personalized home-care support for the elderly: a field experience with a social robot at home. User Model User Adapt Interact 1\u201336","DOI":"10.1007\/s11257-022-09333-y"},{"key":"1044_CR6","doi-asserted-by":"crossref","unstructured":"Ercolano G, Raggioli L, Leone E, Ruocco M, Savino E, Ross S (2018) Seeking and approaching users in domestic environments: testing a reactive approach on two commercial robots. In: 27th IEEE RO-MAN. IEEE, pp 808\u2013813","DOI":"10.1109\/ROMAN.2018.8525593"},{"key":"1044_CR7","doi-asserted-by":"crossref","unstructured":"Hayashi K, Shiomi M, Kanda T, Hagita N (2012) Friendly patrolling: a model of natural encounters. In: Proceedings of the RSS. p 121","DOI":"10.7551\/mitpress\/9481.003.0021"},{"issue":"8","key":"1044_CR8","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780","journal-title":"Neural Comput"},{"key":"1044_CR9","doi-asserted-by":"crossref","unstructured":"H\u00fcttenrauch H, Eklundh KS, Green A, Topp EA (2006) Investigating spatial relationships in human\u2013robot interaction. In: 2006 IEEE\/RSJ IROS. IEEE, pp 5052\u20135059","DOI":"10.1109\/IROS.2006.282535"},{"issue":"6","key":"1044_CR10","doi-asserted-by":"publisher","first-page":"1382","DOI":"10.1109\/TRO.2009.2032969","volume":"25","author":"T Kanda","year":"2009","unstructured":"Kanda T, Glas DF, Shiomi M, Hagita N (2009) Abstracting people\u2019s trajectories for social robots to proactively approach customers. IEEE Trans Robot 25(6):1382\u20131396","journal-title":"IEEE Trans Robot"},{"key":"1044_CR11","doi-asserted-by":"crossref","unstructured":"Kato Y, Kanda T, Ishiguro H (2015) May i help you? Design of human-like polite approaching behavior. In: 10th ACM\/IEEE HRI. IEEE, pp 35\u201342","DOI":"10.1145\/2696454.2696463"},{"issue":"4","key":"1044_CR12","doi-asserted-by":"publisher","first-page":"881","DOI":"10.1109\/TCDS.2018.2843122","volume":"10","author":"M Khamassi","year":"2018","unstructured":"Khamassi M, Velentzas G, Tsitsimis T, Tzafestas C (2018) Robot fast adaptation to changes in human engagement during simulated dynamic social interaction with active exploration in parameterized reinforcement learning. IEEE Trans Cognit Develop Syst 10(4):881\u2013893","journal-title":"IEEE Trans Cognit Develop Syst"},{"key":"1044_CR13","unstructured":"Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980"},{"issue":"12","key":"1044_CR14","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.1016\/j.robot.2013.05.007","volume":"61","author":"T Kruse","year":"2013","unstructured":"Kruse T, Pandey AK, Alami R, Kirsch A (2013) Human-aware robot navigation: a survey. Robot Auton Syst 61(12):1726\u20131743","journal-title":"Robot Auton Syst"},{"key":"1044_CR15","doi-asserted-by":"crossref","unstructured":"McDuff D, Mahmoud A, Mavadati M, Amr M, Turcot J, El Kaliouby R (2016) Affdex SDK: a cross-platform real-time multi-face expression recognition toolkit. In: Proceedings of the 2016 CHI conference extended abstracts on human factors in computing systems. ACM, pp 3723\u20133726","DOI":"10.1145\/2851581.2890247"},{"key":"1044_CR16","first-page":"100","volume":"100","author":"R Mead","year":"2015","unstructured":"Mead R, Mataric MJ (2015) Robots have needs too: people adapt their proxemic preferences to improve autonomous robot recognition of human social signals. New Front Hum Robot Interact 100:100\u2013107","journal-title":"New Front Hum Robot Interact"},{"key":"1044_CR17","doi-asserted-by":"crossref","unstructured":"Michalowski MP, Sabanovic S, Simmons R (2006) A spatial model of engagement for a social robot. In: 9th IEEE international workshop on advanced motion control. IEEE, pp 762\u2013767","DOI":"10.1109\/AMC.2006.1631755"},{"key":"1044_CR18","doi-asserted-by":"crossref","unstructured":"Mumm J, Mutlu B (2011) Human\u2013robot proxemics: physical and psychological distancing in human\u2013robot interaction. In: Proceedings of HRI, pp 331\u2013338","DOI":"10.1145\/1957656.1957786"},{"key":"1044_CR19","doi-asserted-by":"publisher","first-page":"141","DOI":"10.3389\/fnagi.2015.00141","volume":"7","author":"M Pino","year":"2015","unstructured":"Pino M, Boulay M, Jouen F, Rigaud A-S (2015) \u201cAre we ready for robots that care for us?\u2019\u2019 attitudes and opinions of older adults toward socially assistive robots. Front Aging Neurosci 7:141","journal-title":"Front Aging Neurosci"},{"key":"1044_CR20","doi-asserted-by":"crossref","unstructured":"Qureshi AH, Nakamura Y, Yoshikawa Y, Ishiguro H (2016) Robot gains social intelligence through multimodal deep reinforcement learning. In: 2016 IEEE-RAS 16th International Conference on Humanoid robots (humanoids). IEEE, pp 745\u2013751","DOI":"10.1109\/HUMANOIDS.2016.7803357"},{"key":"1044_CR21","doi-asserted-by":"crossref","unstructured":"Raggioli L, Rossi S (2019) A reinforcement-learning approach for adaptive and comfortable assistive robot monitoring behavior. In: 2019 28th IEEE international conference on robot and human interactive communication (RO-MAN). IEEE, pp 1\u20136","DOI":"10.1109\/RO-MAN46459.2019.8956259"},{"issue":"2","key":"1044_CR22","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1007\/s12369-014-0251-1","volume":"7","author":"J Rios-Martinez","year":"2015","unstructured":"Rios-Martinez J, Spalanzani A, Laugier C (2015) From proxemics theory to socially-aware navigation: a survey. Int J Soc Robot 7(2):137\u2013153","journal-title":"Int J Soc Robot"},{"issue":"6","key":"1044_CR23","doi-asserted-by":"publisher","first-page":"1265","DOI":"10.1007\/s12369-020-00650-z","volume":"12","author":"S Rossi","year":"2020","unstructured":"Rossi S, Rossi A, Dautenhahn K (2020) The secret life of robots: perspectives and challenges for robot\u2019s behaviours during non-interactive tasks. Int J Soc Robot 12(6):1265\u20131278","journal-title":"Int J Soc Robot"},{"key":"1044_CR24","doi-asserted-by":"crossref","unstructured":"Rossi S, Ercolano G, Raggioli L, Savino E, Ruocco M (2018) The disappearing robot: an analysis of disengagement and distraction during non-interactive tasks. In: 27th IEEE RO-MAN. IEEE, pp 522\u2013527","DOI":"10.1109\/ROMAN.2018.8525514"},{"key":"1044_CR25","doi-asserted-by":"crossref","unstructured":"Rossi S, Staffa M, Bove L, Capasso R, Ercolano G (2017) User\u2019s personality and activity influence on HRI comfortable distances. In: International conference on social robotics. Springer, pp 167\u2013177","DOI":"10.1007\/978-3-319-70022-9_17"},{"key":"1044_CR26","doi-asserted-by":"crossref","unstructured":"Satake S, Kanda T, Glas DF, Imai M, Ishiguro H, Hagita N (2009) How to approach humans? Strategies for social robots to initiate interaction. In: Proceedings of the 4th ACM\/IEEE international conference on human robot interaction, pp 109\u2013116","DOI":"10.1145\/1514095.1514117"},{"issue":"5","key":"1044_CR27","doi-asserted-by":"publisher","first-page":"874","DOI":"10.1109\/TRO.2007.904911","volume":"23","author":"EA Sisbot","year":"2007","unstructured":"Sisbot EA, Marin-Urias LF, Alami R, Simeon T (2007) A human aware mobile robot motion planner. IEEE Trans Robot 23(5):874\u2013883","journal-title":"IEEE Trans Robot"},{"key":"1044_CR28","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"key":"1044_CR29","unstructured":"Tsitsimis T, Velentzas G, Khamassi M, Tzafestas C (2017) Online adaptation to human engagement perturbations in simulated human-robot interaction using hybrid reinforcement learning. MultiLearn workshop at the 25th European Signal Processing Conference (EUSIPCO 2017)"},{"issue":"3\u20134","key":"1044_CR30","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3\u20134):229\u2013256","journal-title":"Mach Learn"},{"issue":"1","key":"1044_CR31","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0733464813515092","volume":"35","author":"W Ya-Huei","year":"2016","unstructured":"Ya-Huei W, Cristancho-Lacroix V, Fassert C, Faucounau V, de Rotrou J, Rigaud A-S (2016) The attitudes and perceptions of older adults with mild cognitive impairment toward an assistive robot. J Appl Gerontol 35(1):3\u201317","journal-title":"J Appl Gerontol"}],"container-title":["International Journal of Social Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-023-01044-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12369-023-01044-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-023-01044-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,28]],"date-time":"2025-05-28T15:41:18Z","timestamp":1748446878000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12369-023-01044-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,7]]},"references-count":31,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,5]]}},"alternative-id":["1044"],"URL":"https:\/\/doi.org\/10.1007\/s12369-023-01044-7","relation":{},"ISSN":["1875-4791","1875-4805"],"issn-type":[{"value":"1875-4791","type":"print"},{"value":"1875-4805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,7]]},"assertion":[{"value":"15 August 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 September 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}