{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T11:20:13Z","timestamp":1773141613457,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T00:00:00Z","timestamp":1667865600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T00:00:00Z","timestamp":1667865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation, Korea","doi-asserted-by":"crossref","award":["NRF- 2022R1G1A1012746"],"award-info":[{"award-number":["NRF- 2022R1G1A1012746"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100014188","name":"Ministry of Science and ICT, South Korea","doi-asserted-by":"publisher","award":["No.2019-0-00421"],"award-info":[{"award-number":["No.2019-0-00421"]}],"id":[{"id":"10.13039\/501100014188","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The evolution of learning techniques has led robotics to have a considerable influence in industrial and household applications. With the progress in technology revolution, the demand for service robots is rapidly growing and extends to many applications. However, efficient navigation of service robots in crowded environments, with unpredictable human behaviors, is still challenging. The robot is supposed to recognize surrounding information while navigating, and then act accordingly. To address this issue, the proposed method crowd Aware Memory-based Reinforcement Learning (CAM-RL) uses gated recurrent units to store the relative dependencies among the crowd, and utilizes the human\u2013robot interactions in the reinforcement learning framework for collision-free navigation. The proposed method is compared with the state-of-the-art techniques of multi-agent navigation, such as Collision Avoidance with Deep Reinforcement Learning (CADRL), Long Short-Term Memory Reinforcement Learning (LSTM-RL) and Social Attention Reinforcement Learning (SARL). Experimental results show that the proposed method can identify and learn human\u2013robot interactions more extensively and efficiently than above-mentioned methods while navigating in a crowded environment. The proposed method achieved a success rate of greater than or equal to <jats:inline-formula><jats:alternatives><jats:tex-math>$$99\\%$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mn>99<\/mml:mn>\n                    <mml:mo>%<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> and a collision rate of less than or equal to <jats:inline-formula><jats:alternatives><jats:tex-math>$$1\\%$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mrow>\n                    <mml:mn>1<\/mml:mn>\n                    <mml:mo>%<\/mml:mo>\n                  <\/mml:mrow>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula> in all test case scenarios, which is better compared to the previously proposed methods.<\/jats:p>","DOI":"10.1007\/s40747-022-00906-3","type":"journal-article","created":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T09:03:21Z","timestamp":1667898201000},"page":"2147-2158","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Memory-based crowd-aware robot navigation using deep reinforcement learning"],"prefix":"10.1007","volume":"9","author":[{"given":"Sunil Srivatsav","family":"Samsani","sequence":"first","affiliation":[]},{"given":"Husna","family":"Mutahira","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3036-4660","authenticated-orcid":false,"given":"Mannan Saeed","family":"Muhammad","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,11,8]]},"reference":[{"key":"906_CR1","doi-asserted-by":"crossref","unstructured":"Wirtz J, Patterson PG, Kunz WH, et al (2018) Brave new world: service robots in the frontline. J Serv Manag 29(5):907\u2013931","DOI":"10.1108\/JOSM-04-2018-0119"},{"issue":"22","key":"906_CR2","doi-asserted-by":"publisher","first-page":"10,702","DOI":"10.3390\/app112210702","volume":"11","author":"JA Gonzalez-Aguirre","year":"2021","unstructured":"Gonzalez-Aguirre JA, Osorio-Oliveros R, Rodr\u00edguez-Hern\u00e1ndez KL et al (2021) Service robots: trends and technology. Appl Sci 11(22):10,702","journal-title":"Appl Sci"},{"key":"906_CR3","doi-asserted-by":"crossref","unstructured":"Zielinska TT (2019) History of service robots and new trends. In: Dan Z, Bin W (eds) Novel design and applications of robotics technologies. IGI Global, pp 158\u2013187","DOI":"10.4018\/978-1-5225-5276-5.ch006"},{"key":"906_CR4","doi-asserted-by":"crossref","unstructured":"Das S, Das I, Shaw RN et al (2021) Advance machine learning and artificial intelligence applications in service robot. In: Rabindra S, Ankush G, Valentina B, Monica B (eds) Artificial intelligence for future generation robotics. Elsevier, pp 83\u201391","DOI":"10.1016\/B978-0-323-85498-6.00002-2"},{"issue":"3","key":"906_CR5","first-page":"598","volume":"62","author":"Y Sun","year":"2022","unstructured":"Sun Y, Wang R (2022) The research framework and evolution of service robots. J Comput Inf Syst 62(3):598\u2013608","journal-title":"J Comput Inf Syst"},{"key":"906_CR6","doi-asserted-by":"crossref","unstructured":"Lin J, Yang X, Zheng P et al (2019) End-to-end decentralized multi-robot navigation in unknown complex environments via deep reinforcement learning. In: 2019 IEEE international conference on mechatronics and automation (ICMA). IEEE, pp 2493\u20132500","DOI":"10.1109\/ICMA.2019.8816208"},{"key":"906_CR7","doi-asserted-by":"crossref","unstructured":"Kakoty NM, Mazumdar M, Sonowal D (2019) Mobile robot navigation in unknown dynamic environment inspired by human pedestrian behavior. In: Chhabi Rani P, Bibudhendu P, Prasant M, Rajkumar B, Kuan-Ching L (eds) Progress in advanced computing and intelligent engineering. Springer, pp 441\u2013451","DOI":"10.1007\/978-981-13-0224-4_40"},{"key":"906_CR8","doi-asserted-by":"crossref","unstructured":"Pico N, Jung Hr, Medrano J et\u00a0al (2022) Climbing control of autonomous mobile robot with estimation of wheel slip and wheel-ground contact angle. J Mech Sci Technol 36(2):959\u2013968","DOI":"10.1007\/s12206-022-0142-6"},{"key":"906_CR9","unstructured":"Veloso M, Biswas J, Coltin B et\u00a0al (2015) Cobots: robust symbiotic autonomous mobile service robots. In: Twenty-fourth international joint conference on artificial intelligence"},{"key":"906_CR10","doi-asserted-by":"crossref","unstructured":"K\u00e4stner L, Fatloun B, Shen Z et\u00a0al (2022) Human-following and-guiding in crowded environments using semantic deep-reinforcement-learning for mobile service robots. arXiv preprint arXiv:2206.05771","DOI":"10.1109\/ICRA46639.2022.9812111"},{"key":"906_CR11","doi-asserted-by":"crossref","unstructured":"Kolski S, Ferguson D, Bellino M et\u00a0al (2006) Autonomous driving in structured and unstructured environments. In: 2006 IEEE intelligent vehicles symposium. IEEE, pp 558\u2013563","DOI":"10.1109\/IROS.2006.282302"},{"key":"906_CR12","doi-asserted-by":"crossref","unstructured":"Visca M, Kuutti S, Powell R, et\u00a0al (2021) Deep learning traversability estimator for mobile robots in unstructured environments. In: Annual conference towards autonomous robotic systems. Springer, pp 203\u2013213","DOI":"10.1007\/978-3-030-89177-0_22"},{"issue":"10","key":"906_CR13","doi-asserted-by":"publisher","first-page":"1568","DOI":"10.1016\/j.robot.2014.05.006","volume":"62","author":"AV Savkin","year":"2014","unstructured":"Savkin AV, Wang C (2014) Seeking a path through the crowd: robot navigation in unknown dynamic environments with moving obstacles based on an integrated environment representation. Robot Auton Syst 62(10):1568\u20131580","journal-title":"Robot Auton Syst"},{"key":"906_CR14","doi-asserted-by":"crossref","unstructured":"Mavrogiannis C, Hutchinson AM, Macdonald J et\u00a0al (2019) Effects of distinct robot navigation strategies on human behavior in a crowded environment. In: 2019 14th ACM\/IEEE international conference on human\u2013robot interaction (HRI). IEEE, pp 421\u2013430","DOI":"10.1109\/HRI.2019.8673115"},{"issue":"5","key":"906_CR15","doi-asserted-by":"publisher","first-page":"4282","DOI":"10.1103\/PhysRevE.51.4282","volume":"51","author":"D Helbing","year":"1995","unstructured":"Helbing D, Molnar P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282","journal-title":"Phys Rev E"},{"key":"906_CR16","doi-asserted-by":"crossref","unstructured":"Van\u00a0den Berg J, Lin M, Manocha D (2008) Reciprocal velocity obstacles for real-time multi-agent navigation. In: 2008 IEEE international conference on robotics and automation. IEEE, pp 1928\u20131935","DOI":"10.1109\/ROBOT.2008.4543489"},{"key":"906_CR17","doi-asserted-by":"crossref","unstructured":"Van Den\u00a0Berg J, Guy SJ, Lin M et\u00a0al (2011) Reciprocal n-body collision avoidance. In: C\u00e9dric P, Roland S, Gerhard H (eds) Robotics research. Springer, pp 3\u201319","DOI":"10.1007\/978-3-642-19457-3_1"},{"key":"906_CR18","doi-asserted-by":"crossref","unstructured":"Chen YF, Liu M, Everett M et\u00a0al (2017) Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: 2017 IEEE international conference on robotics and automation (ICRA). IEEE, pp 285\u2013292","DOI":"10.1109\/ICRA.2017.7989037"},{"key":"906_CR19","doi-asserted-by":"crossref","unstructured":"Long P, Fan T, Liao X et\u00a0al (2018) Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 6252\u20136259","DOI":"10.1109\/ICRA.2018.8461113"},{"key":"906_CR20","doi-asserted-by":"crossref","unstructured":"Everett M, Chen YF, How JP (2018) Motion planning among dynamic, decision-making agents with deep reinforcement learning. In: 2018 IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3052\u20133059","DOI":"10.1109\/IROS.2018.8593871"},{"key":"906_CR21","doi-asserted-by":"crossref","unstructured":"Chen C, Liu Y, Kreiss S et\u00a0al (2019) Crowd\u2013robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. In: 2019 international conference on robotics and automation (ICRA). IEEE, pp 6015\u20136022","DOI":"10.1109\/ICRA.2019.8794134"},{"issue":"2","key":"906_CR22","doi-asserted-by":"publisher","first-page":"2754","DOI":"10.1109\/LRA.2020.2972868","volume":"5","author":"Y Chen","year":"2020","unstructured":"Chen Y, Liu C, Shi BE et al (2020) Robot navigation in crowds by graph convolutional networks with attention learned from human gaze. IEEE Robot Autom Lett 5(2):2754\u20132761","journal-title":"IEEE Robot Autom Lett"},{"key":"906_CR23","doi-asserted-by":"crossref","unstructured":"Nishimura M, Yonetani R (2020) L2b: learning to balance the safety-efficiency trade-off in interactive crowd-aware robot navigation. In: 2020 IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 11004\u201311010","DOI":"10.1109\/IROS45743.2020.9341519"},{"issue":"3","key":"906_CR24","doi-asserted-by":"publisher","first-page":"5223","DOI":"10.1109\/LRA.2021.3071954","volume":"6","author":"SS Samsani","year":"2021","unstructured":"Samsani SS, Muhammad MS (2021) Socially compliant robot navigation in crowded environment by human behavior resemblance using deep reinforcement learning. IEEE Robot Autom Lett 6(3):5223\u20135230","journal-title":"IEEE Robot Autom Lett"},{"key":"906_CR25","doi-asserted-by":"crossref","unstructured":"Kato Y, Nagano Y, Yokoyama H (2017) A pedestrian model in human\u2013robot coexisting environment for mobile robot navigation. In: 2017 IEEE\/SICE international symposium on system integration (SII). IEEE, pp 992\u2013997","DOI":"10.1109\/SII.2017.8279352"},{"key":"906_CR26","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1016\/j.procir.2020.08.003","volume":"97","author":"RA Rojas","year":"2021","unstructured":"Rojas RA, Garcia MAR, Gualtieri L et al (2021) Combining safety and speed in collaborative assembly systems-an approach to time optimal trajectories for collaborative robots. Procedia CIRP 97:308\u2013312","journal-title":"Procedia CIRP"},{"key":"906_CR27","doi-asserted-by":"crossref","unstructured":"Trautman P, Krause A (2010) Unfreezing the robot: navigation in dense, interacting crowds. In: 2010 IEEE\/RSJ international conference on intelligent robots and systems. IEEE, pp 797\u2013803","DOI":"10.1109\/IROS.2010.5654369"},{"key":"906_CR28","doi-asserted-by":"crossref","unstructured":"Trautman P (2017) Sparse interacting gaussian processes: efficiency and optimality theorems of autonomous crowd navigation. In: 2017 IEEE 56th annual conference on decision and control (CDC). IEEE, pp 327\u2013334","DOI":"10.1109\/CDC.2017.8263686"},{"issue":"1","key":"906_CR29","doi-asserted-by":"publisher","first-page":"168781401773665","DOI":"10.1177\/1687814017736653","volume":"10","author":"D Fethi","year":"2018","unstructured":"Fethi D, Nemra A, Louadj K et al (2018) Simultaneous localization, mapping, and path planning for unmanned vehicle using optimal control. Adv Mech Eng 10(1):1687814017736653","journal-title":"Adv Mech Eng"},{"key":"906_CR30","unstructured":"Chaplot DS, Gandhi D, Gupta S et\u00a0al (2020) Learning to explore using active neural slam. arXiv preprint arXiv:2004.05155"},{"key":"906_CR31","doi-asserted-by":"crossref","unstructured":"Chen YF, Everett M, Liu M et\u00a0al (2017) Socially aware motion planning with deep reinforcement learning. In: 2017 IEEE\/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 1343\u20131350","DOI":"10.1109\/IROS.2017.8202312"},{"key":"906_CR32","unstructured":"Vaswani A, Shazeer N, Parmar N et\u00a0al (2017) Attention is all you need. Adv Neur Inf Process Syst 30"},{"key":"906_CR33","doi-asserted-by":"crossref","unstructured":"Vemula A, Muelling K, Oh J (2018) Social attention: modeling attention in human crowds. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 4601\u20134607","DOI":"10.1109\/ICRA.2018.8460504"},{"key":"906_CR34","doi-asserted-by":"publisher","first-page":"15,140","DOI":"10.1109\/ACCESS.2019.2894626","volume":"7","author":"J Yuan","year":"2019","unstructured":"Yuan J, Wang H, Lin C et al (2019) A novel gru-rnn network model for dynamic path planning of mobile robot. IEEE Access 7:15,140-15,151","journal-title":"IEEE Access"},{"issue":"3","key":"906_CR35","doi-asserted-by":"publisher","first-page":"172988142092167","DOI":"10.1177\/1729881420921672","volume":"17","author":"H Quan","year":"2020","unstructured":"Quan H, Li Y, Zhang Y (2020) A novel mobile robot navigation method based on deep reinforcement learning. Int J Adv Robot Syst 17(3):1729881420921672","journal-title":"Int J Adv Robot Syst"},{"key":"906_CR36","doi-asserted-by":"crossref","unstructured":"Cho K, Van\u00a0Merri\u00ebnboer B, Bahdanau D et\u00a0al (2014) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259","DOI":"10.3115\/v1\/W14-4012"},{"issue":"8","key":"906_CR37","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780","journal-title":"Neural Comput"},{"key":"906_CR38","unstructured":"Chung J, Gulcehre C, Cho K et\u00a0al (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555"},{"key":"906_CR39","doi-asserted-by":"publisher","unstructured":"Gao Y, Huang CM (2021) Evaluation of socially-aware robot navigation. Front Robot AI 8:721317. https:\/\/doi.org\/10.3389\/frobt.2021.721317","DOI":"10.3389\/frobt.2021.721317"},{"issue":"2","key":"906_CR40","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1007\/s12369-021-00791-9","volume":"14","author":"Y Kobayashi","year":"2022","unstructured":"Kobayashi Y, Sugimoto T, Tanaka K et al (2022) Robot navigation based on predicting of human interaction and its reproducible evaluation in a densely crowded environment. Int J Soc Robot 14(2):373\u2013387","journal-title":"Int J Soc Robot"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-022-00906-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-022-00906-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-022-00906-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,18]],"date-time":"2023-04-18T09:44:28Z","timestamp":1681811068000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-022-00906-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,8]]},"references-count":40,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4]]}},"alternative-id":["906"],"URL":"https:\/\/doi.org\/10.1007\/s40747-022-00906-3","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,8]]},"assertion":[{"value":"23 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 November 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}