{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T19:12:28Z","timestamp":1775934748252,"version":"3.50.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,7,13]],"date-time":"2022-07-13T00:00:00Z","timestamp":1657670400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["NSF IIS-1734361"],"award-info":[{"award-number":["NSF IIS-1734361"]}]},{"DOI":"10.13039\/100009157","name":"National Institute on Disability, Independent Living, and Rehabilitation Research","doi-asserted-by":"crossref","award":["NIDILRR 90DPGE0003"],"award-info":[{"award-number":["NIDILRR 90DPGE0003"]}],"id":[{"id":"10.13039\/100009157","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2022,9,30]]},"abstract":"<jats:p>\n            The human-robot interaction community has developed many methods for robots to navigate safely and socially alongside humans. However, experimental procedures to evaluate these works are usually constructed on a per-method basis. Such disparate evaluations make it difficult to compare the performance of such methods across the literature. To bridge this gap, we introduce\n            <jats:italic>SocNavBench<\/jats:italic>\n            , a simulation framework for evaluating social navigation algorithms.\n            <jats:italic>SocNavBench<\/jats:italic>\n            comprises a simulator with photo-realistic capabilities and curated social navigation scenarios grounded in real-world pedestrian data. We also provide an implementation of a suite of metrics to quantify the performance of navigation algorithms on these scenarios. Altogether,\n            <jats:italic>SocNavBench<\/jats:italic>\n            provides a test framework for evaluating disparate social navigation methods in a consistent and interpretable manner. To illustrate its use, we demonstrate testing three existing social navigation methods and a baseline method on\n            <jats:italic>SocNavBench<\/jats:italic>\n            , showing how the suite of metrics helps infer their performance trade-offs. Our code is open-source, allowing the addition of new scenarios and metrics by the community to help evolve\n            <jats:italic>SocNavBench<\/jats:italic>\n            to reflect advancements in our understanding of social navigation.\n          <\/jats:p>","DOI":"10.1145\/3476413","type":"journal-article","created":{"date-parts":[[2022,2,14]],"date-time":"2022-02-14T18:54:53Z","timestamp":1644864893000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":53,"title":["SocNavBench: A Grounded Simulation Testing Framework for Evaluating Social Navigation"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4329-9738","authenticated-orcid":false,"given":"Abhijat","family":"Biswas","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]},{"given":"Allan","family":"Wang","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]},{"given":"Gustavo","family":"Silvera","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]},{"given":"Aaron","family":"Steinfeld","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]},{"given":"Henny","family":"Admoni","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA"}]}],"member":"320","published-online":{"date-parts":[[2022,7,13]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Pioneer3DX P3DX-RevA Specifications","author":"Inc Adept Technology","year":"2020","unstructured":"Adept Technology Inc. 2020. Pioneer3DX P3DX-RevA Specifications. Retrieved from https:\/\/www.generationrobots.com\/media\/Pioneer3DX-P3DX-RevA.pdf."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.110"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.170"},{"key":"e_1_3_2_5_2","unstructured":"Stefan Becker Ronny Hug Wolfgang H\u00fcbner and Michael Arens. 2018. An evaluation of trajectory prediction approaches and notes on the trajnet benchmark. Retrieved from https:\/\/arxiv:1805.07663."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8593411"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8206628"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00070-3"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8794192"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8794134"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8202312"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2011.2166435"},{"key":"e_1_3_2_13_2","first-page":"1688","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Ferrer Gonzalo","year":"2013","unstructured":"Gonzalo Ferrer, Anais Garrell, and Alberto Sanfeliu. 2013. Robot companion: A social-force based approach with human awareness-navigation in crowded environments. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems. 1688\u20131694."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1177\/027836499801700706"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-009-0037-z"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/100.580977"},{"key":"e_1_3_2_17_2","volume-title":"libPedSim","author":"Gloor Christian","year":"2020","unstructured":"Christian Gloor. 2020. libPedSim. Retrieved from http:\/\/pedsim.silmaril.org\/."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00240"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630610"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.51.4282"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509772"},{"key":"e_1_3_2_22_2","first-page":"1587","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI\u201910)","author":"Joseph Joshua","year":"2010","unstructured":"Joshua Joseph, Finale Doshi-Velez, and Nicholas Roy. 2010. A bayesian nonparametric approach to modeling mobility patterns. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI\u201910). AAAI Press, 1587\u20131593."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2011.133"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-015-0310-2"},{"key":"e_1_3_2_25_2","unstructured":"Rachel Kirby. 2010. Social robot navigation. https:\/\/www.ri.cmu.edu\/pub_files\/2010\/5\/rk_thesis.pdf."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2021.3069362"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364915619772"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ARSO.2012.6213404"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1177\/02783640122067453"},{"key":"e_1_3_2_30_2","first-page":"655","volume-title":"Computer Graphics Forum","author":"Lerner Alon","year":"2007","unstructured":"Alon Lerner, Yiorgos Chrysanthou, and Dani Lischinski. 2007. Crowds by example. In Computer Graphics Forum, Vol. 26. Wiley Online Library, 655\u2013664."},{"key":"e_1_3_2_31_2","first-page":"902","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Luber Matthias","year":"2012","unstructured":"Matthias Luber, Luciano Spinello, Jens Silva, and Kai O. Arras. 2012. Socially-aware robot navigation: A learning approach. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 902\u2013907."},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Luis J. Manso Pedro Nunez Luis V. Calderita Diego R. Faria and Pilar Bachiller. 2019. SocNav1: A dataset to benchmark and learn social navigation conventions. Retrieved from https:\/\/arXiv:1909.02993.","DOI":"10.3390\/data5010007"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.5555\/3378680.3378753"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8206601"},{"key":"e_1_3_2_35_2","volume-title":"Proceedings of the International Workshop on the Algorithmic Foundations of Robotics (WAFR\u201916)","author":"Mavrogiannis Christoforos I.","year":"2016","unstructured":"Christoforos I. Mavrogiannis and Ross A. Knepper. 2016. Decentralized multi-agent navigation planning with braids. In Proceedings of the International Workshop on the Algorithmic Foundations of Robotics (WAFR\u201916)."},{"key":"e_1_3_2_36_2","first-page":"361","volume-title":"Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction","author":"Mavrogiannis Christoforos I.","year":"2018","unstructured":"Christoforos I. Mavrogiannis, Wil B. Thomason, and Ross A. Knepper. 2018. Social momentum: A framework for legible navigation in dynamic multi-agent environments. In Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction. ACM, 361\u2013369."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01443"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1098\/rspb.2009.0405"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0010047"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/1349822.1349860"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487452"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2007.01090.x"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459260"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_33"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00144"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-014-0238-y"},{"key":"e_1_3_2_47_2","volume-title":"TUG Robot Product Page","author":"Inc ST Engineering Aethon","year":"2021","unstructured":"ST Engineering Aethon Inc. (2021). TUG Robot Product Page. Retrieved from https:\/\/aethon.com\/products\/."},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/1514095.1514115"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/1401132.1401207"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.1999.770401"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Varun Tolani Somil Bansal Aleksandra Faust and Claire Tomlin. 2020. Visual Navigation Among Humans with Optimal Control as a Supervisor. Retrieved from https:\/\/arXiv:2003.09354.","DOI":"10.1109\/LRA.2021.3060638"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2017.8263686"},{"key":"e_1_3_2_53_2","first-page":"797","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Trautman Peter","year":"2010","unstructured":"Peter Trautman and Andreas Krause. 2010. Unfreezing the robot: Navigation in dense, interacting crowds. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 797\u2013803."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914557874"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1142008"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-016-0352-0"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TASE.2017.2731371"},{"key":"e_1_3_2_58_2","first-page":"1928","volume-title":"Proceedings of the IEEE International Conference on Robotics and Automation","author":"Berg J. van den","year":"2008","unstructured":"J. van den Berg, Ming Lin, and D. Manocha. 2008. Reciprocal velocity obstacles for real-time multi-agent navigation. In Proceedings of the IEEE International Conference on Robotics and Automation. 1928\u20131935."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/1342250.1342272"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.492"},{"key":"e_1_3_2_61_2","first-page":"1341","volume-title":"Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Vasquez Dizan","year":"2014","unstructured":"Dizan Vasquez, Billy Okal, and Kai O. Arras. 2014. Inverse reinforcement learning algorithms and features for robot navigation in crowds: An experimental comparison. In Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 1341\u20131346."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460504"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298971"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/93\/68005"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476413","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3476413","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:45Z","timestamp":1750188645000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476413"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,13]]},"references-count":63,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9,30]]}},"alternative-id":["10.1145\/3476413"],"URL":"https:\/\/doi.org\/10.1145\/3476413","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"value":"2573-9522","type":"print"},{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,13]]},"assertion":[{"value":"2020-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}