{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T14:30:56Z","timestamp":1776090656030,"version":"3.50.1"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>In social robots, visual attention expresses awareness of the scenario components and dynamics. As in humans, their attention should be driven by a combination of different attention mechanisms. In this article, we introduce multi-modal saliency maps, i.e., spatial representations of saliency that dynamically integrate multiple attention sources depending on the context. We provide the mathematical formulation of the model and an open source software implementation. Finally, we present an initial exploration of its potential in social interaction scenarios with humans and evaluate its implementation.<\/jats:p>","DOI":"10.1145\/3737891","type":"journal-article","created":{"date-parts":[[2025,6,3]],"date-time":"2025-06-03T10:41:03Z","timestamp":1748947263000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["From Percepts to Semantics: A Multi-modal Saliency Map to Support Social Robots\u2019 Attention"],"prefix":"10.1145","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4823-6672","authenticated-orcid":false,"given":"Lorenzo","family":"Ferrini","sequence":"first","affiliation":[{"name":"PAL Robotics, Spain and Technische Universit\u00e4t Wien, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6641-6450","authenticated-orcid":false,"given":"Antonio","family":"Andriella","sequence":"additional","affiliation":[{"name":"Institut de Rob\u00f2tica i Inform\u00e0tica Industrial, CSIC UPC, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8295-6932","authenticated-orcid":false,"given":"Raquel","family":"Ros","sequence":"additional","affiliation":[{"name":"IIIA\/CSIC, Barcelona, Spain and PAL Robotics, Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3391-8876","authenticated-orcid":false,"given":"S\u00e9verin","family":"Lemaignan","sequence":"additional","affiliation":[{"name":"PAL Robotics, Barcelona, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,7,22]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.6.1.Admoni"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40415-3_22"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/BioRob49111.2020.9224303"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2010.2096505"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2009.2037511"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abc5044"},{"key":"e_1_3_1_8_2","article-title":"A context-dependent attention system for a social robot","author":"Breazeal Cynthia","year":"1999","unstructured":"Cynthia Breazeal and Brian Scassellati. 1999. A context-dependent attention system for a social robot. IJCAI International Joint Conference on Artificial Intelligence, 2.","journal-title":"IJCAI International Joint Conference on Artificial Intelligence, 2."},{"issue":"1","key":"e_1_3_1_9_2","first-page":"1","article-title":"A theory of attention for cognitive systems","volume":"4","author":"Bridewell Will","year":"2016","unstructured":"Will Bridewell and Paul Bello. 2016. A theory of attention for cognitive systems. Advances in Cognitive Systems 4, 1 (2016), 1\u201316.","journal-title":"Advances in Cognitive Systems"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI53351.2022.9889499"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13414-012-0322-z"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-019-01689-4"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-021-11112-7"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2020.00034"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1088\/2634-4386\/ac6b50"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/1763974.1764031"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.133.4.694"},{"key":"e_1_3_1_18_2","first-page":"303","volume-title":"2021 ACM\/IEEE International Conference on Human-Robot Interaction","author":"Gillet Sarah","unstructured":"Sarah Gillet, Ronald Cumbal, Andr\u00e9 Pereira, Jos\u00e9 Lopes, Olov Engwall, and Iolanda Leite. 2021. Robot gaze can mediate participation imbalance in groups with different skill levels. In 2021 ACM\/IEEE International Conference on Human-Robot Interaction, 303\u2013311."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI53351.2022.9889416"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.281849"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.3758\/BF03196685"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.730558"},{"key":"e_1_3_1_23_2","unstructured":"Glenn Jocher Ayush Chaurasia and Jing Qiu. 2023. YOLO by Ultralytics. Retrieved from https:\/\/github.com\/ultralytics\/ultralytics"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2638728.2641695"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0028-3932(01)00116-6"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2007.07.012"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICMT.2011.6002127"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2021.630386"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2023.1127626"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS51168.2021.9636816"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2023.1215771"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2504435.2504454"},{"key":"e_1_3_1_34_2","unstructured":"OpenAI. 2022. ChatGPT: A large language model by OpenAI. Retrieved from https:\/\/www.openai.com\/research\/chatgpt"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.502"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341297"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0010-0277(00)00156-6"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2008.4543329"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/0146-664X(80)90026-X"},{"key":"e_1_3_1_40_2","first-page":"1972","volume-title":"AAAI Conference on Artificial Intelligence","volume":"6","author":"Schermerhorn Paul W.","year":"2006","unstructured":"Paul W. Schermerhorn, James F. Kramer, Christopher Middendorff, and Matthias Scheutz. 2006. DIARC: A testbed for natural human-robot interaction. In AAAI Conference on Artificial Intelligence. Citeseer, Vol. 6, 1972\u20131973."},{"key":"e_1_3_1_41_2","volume-title":"Modern Algebra","author":"Warner Seth","year":"1990","unstructured":"Seth Warner. 1990. Modern Algebra. Courier Corporation."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2014.2303083"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3737891","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T19:26:57Z","timestamp":1753212417000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3737891"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,22]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3737891"],"URL":"https:\/\/doi.org\/10.1145\/3737891","relation":{},"ISSN":["2573-9522"],"issn-type":[{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,22]]},"assertion":[{"value":"2024-02-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}