{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T21:30:51Z","timestamp":1777411851783,"version":"3.51.4"},"reference-count":89,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T00:00:00Z","timestamp":1676419200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"crossref","award":["N00014-18-1-2503"],"award-info":[{"award-number":["N00014-18-1-2503"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2023,3,31]]},"abstract":"<jats:p>Robots need to explain their behavior to gain trust. Existing research has focused on explaining a robot\u2019s current behavior, yet it remains unknown yet challenging how to provide explanations of past actions in an environment that might change after a robot\u2019s actions, leading to critical missing causal information due to moved objects.<\/jats:p>\n          <jats:p>We conducted an experiment (N = 665) investigating how a robot could help participants infer the missing causal information by replaying the past behavior physically, using verbal explanations, and projecting visual information onto the environment. Participants watched videos of the robot replaying its completion of an integrated mobile kitting task. During the replay, the objects are already gone, so participants needed to infer where an object was picked, where a ground obstacle had been, and where the object was placed.<\/jats:p>\n          <jats:p>Based on the results, we recommend combining physical replay with speech and projection indicators (Replay-Project-Say) to help infer all the missing causal information (picking, navigation, and placement) from the robot\u2019s past actions. This condition had the best outcome in both task-based\u2014effectiveness, efficiency, and confidence\u2014and team-based metrics\u2014workload and trust. If one\u2019s focus is efficiency, then we recommend projection markers for navigation inferences and verbal markers for placing inferences.<\/jats:p>","DOI":"10.1145\/3568024","type":"journal-article","created":{"date-parts":[[2022,11,2]],"date-time":"2022-11-02T05:19:11Z","timestamp":1667366351000},"page":"1-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Communicating Missing Causal Information to Explain a Robot\u2019s Past Behavior"],"prefix":"10.1145","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8243-7411","authenticated-orcid":false,"given":"Zhao","family":"Han","sequence":"first","affiliation":[{"name":"University of Massachusetts Lowell, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9962-5865","authenticated-orcid":false,"given":"Holly","family":"Yanco","sequence":"additional","affiliation":[{"name":"University of Massachusetts Lowell, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,2,15]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139396"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.6.1.Admoni"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3029798.3038387"},{"key":"e_1_3_4_6_2","first-page":"1168","volume-title":"17th International Conference on Autonomous Agents and MultiAgent Systems","author":"Amir Dan","year":"2018","unstructured":"Dan Amir and Ofra Amir. 2018. Highlights: Summarizing agent behavior to people. In 17th International Conference on Autonomous Agents and MultiAgent Systems. 1168\u20131176."},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/3237383.3237877"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2856272"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_4_10_2","article-title":"In Proceedings of the AI-HRI Symposium at AAAI-FSS","author":"Bagchi Shelly","year":"2020","unstructured":"Shelly Bagchi, Jason R. Wilson, Muneeb I. Ahmad, Christian Dondrup, Zhao Han, Justin W. Hart, Matteo Leonetti, Katrin Lohan, Ross Mead, Emmanuel Senft, et\u00a0al. 2020. In Proceedings of the AI-HRI Symposium at AAAI-FSS. arXiv preprint arXiv:2010.13830 (2020).","journal-title":"arXiv preprint arXiv:2010.13830"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1111\/1467-839X.00024"},{"key":"e_1_3_4_12_2","volume-title":"Observational Learning","author":"Bandura Albert","year":"2008","unstructured":"Albert Bandura. 2008. Observational Learning. American Cancer Society."},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infbeh.2008.04.006"},{"key":"e_1_3_4_14_2","first-page":"139","article-title":"How people explain their own and others\u2019 behavior: A theory of lay causal explanations","volume":"6","author":"B\u00f6hm Gisela","year":"2015","unstructured":"Gisela B\u00f6hm and Hans-R\u00fcdiger Pfister. 2015. How people explain their own and others\u2019 behavior: A theory of lay causal explanations. Front. Psychol. 6 (2015), 139.","journal-title":"Front. Psychol."},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980058"},{"key":"e_1_3_4_16_2","first-page":"487","volume-title":"25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN\u201916)","author":"Brooks Daniel J.","year":"2016","unstructured":"Daniel J. Brooks, Momotaz Begum, and Holly A. Yanco. 2016. Analysis of reactions towards failures and recovery strategies for autonomous robots. In 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN\u201916). IEEE, 487\u2013492."},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cognition.2010.12.001"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infbeh.2017.04.001"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.1561\/2300000057"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ECMR.2015.7403771"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/23"},{"key":"e_1_3_4_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2012.2229950"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1201\/9780429489105"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2014.02.001"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434073.3444657"},{"key":"e_1_3_4_26_2","volume-title":"AAAI Fall Symposium Series","author":"Graaf Maartje M. A. De","year":"2017","unstructured":"Maartje M. A. De Graaf and Bertram F. Malle. 2017. How people explain action (and autonomous intelligent systems should too). In AAAI Fall Symposium Series."},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2019.8673308"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173386.3173568"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3461597"},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","DOI":"10.5555\/2447556.2447663"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2157689.2157702"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/2447556.2447672"},{"key":"e_1_3_4_33_2","first-page":"993","volume-title":"26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN\u201917)","author":"Elliott Sarah","year":"2017","unstructured":"Sarah Elliott, Zhe Xu, and Maya Cakmak. 2017. Learning generalizable surface cleaning actions from demonstration. In 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN\u201917). IEEE, 993\u2013999."},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193146"},{"key":"e_1_3_4_35_2","article-title":"Mummer: Socially intelligent human-robot interaction in public spaces","author":"Foster Mary Ellen","year":"2019","unstructured":"Mary Ellen Foster, Bart Craenen, Amol Deshmukh, Oliver Lemon, Emanuele Bastianelli, Christian Dondrup, Ioannis Papaioannou, Andrea Vanzo, Jean-Marc Odobez, Olivier Can\u00e9vet, et\u00a0al. 2019. Mummer: Socially intelligent human-robot interaction in public spaces. arXiv preprint arXiv:1909.06749 (2019).","journal-title":"arXiv preprint arXiv:1909.06749"},{"key":"e_1_3_4_36_2","doi-asserted-by":"publisher","DOI":"10.1080\/15248372.2010.542216"},{"key":"e_1_3_4_37_2","first-page":"108","volume-title":"International Conference on Applied Human Factors and Ergonomics","author":"Gautam Alvika","year":"2020","unstructured":"Alvika Gautam, Jacob W. Crandall, and Michael A. Goodrich. 2020. Self-assessment of proficiency of intelligent systems: Challenges and opportunities. In International Conference on Applied Human Factors and Ergonomics. Springer, 108\u2013113."},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.5296\/jse.v2i1.1007"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aay7120"},{"key":"e_1_3_4_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197395"},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457185"},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI53351.2022.9889354"},{"key":"e_1_3_4_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3469652"},{"key":"e_1_3_4_44_2","article-title":"Projection mapping implementation: Enabling direct externalization of perception results and action intent to improve robot explainability","author":"Han Zhao","year":"2020","unstructured":"Zhao Han, Alexander Wilkinson, Jenna Parrillo, Jordan Allspaw, and Holly A. Yanco. 2020. Projection mapping implementation: Enabling direct externalization of perception results and action intent to improve robot explainability. In AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (AI-HRI\u201920).","journal-title":"AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (AI-HRI\u201920)"},{"key":"e_1_3_4_45_2","volume-title":"AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (AI-HRI\u201922)","author":"Han Zhao","year":"2022","unstructured":"Zhao Han, Tom Williams, and Holly A. Yanco. 2022. Mixed-reality robot behavior replay: A system implementation. In AAAI Fall Symposium on Artificial Intelligence for Human-Robot Interaction (AI-HRI\u201922)."},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/3378680.3378755"},{"key":"e_1_3_4_47_2","first-page":"904","volume-title":"Human Factors and Ergonomics Society Annual Meeting","author":"Hart Sandra G.","year":"2006","unstructured":"Sandra G. Hart. 2006. NASA-task load index (NASA-TLX); 20 years later. In Human Factors and Ergonomics Society Annual Meeting, Vol. 50. Sage Publications, Los Angeles, CA, 904\u2013908."},{"key":"e_1_3_4_48_2","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1109\/ARSO.2013.6705520","volume-title":"IEEE Workshop on Advanced Robotics and its Social Impacts","author":"Hashimoto Kunimatsu","year":"2013","unstructured":"Kunimatsu Hashimoto, Fuminori Saito, Takashi Yamamoto, and Koichi Ikeda. 2013. A field study of the human support robot in the home environment. In IEEE Workshop on Advanced Robotics and its Social Impacts. IEEE, 143\u2013150."},{"key":"e_1_3_4_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/2909824.3020233"},{"key":"e_1_3_4_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dr.2018.12.002"},{"key":"e_1_3_4_51_2","doi-asserted-by":"publisher","DOI":"10.1108\/01439910110410051"},{"key":"e_1_3_4_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9560774"},{"key":"e_1_3_4_53_2","article-title":"The design of stretch: A compact, lightweight mobile manipulator for indoor human environments","author":"Kemp Charles C.","year":"2021","unstructured":"Charles C. Kemp, Aaron Edsinger, Henry M. Clever, and Blaine Matulevich. 2021. The design of stretch: A compact, lightweight mobile manipulator for indoor human environments. arXiv preprint arXiv:2109.10892 (2021).","journal-title":"arXiv preprint arXiv:2109.10892"},{"key":"e_1_3_4_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jecp.2012.02.006"},{"key":"e_1_3_4_55_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infbeh.2006.07.001"},{"key":"e_1_3_4_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6224965"},{"key":"e_1_3_4_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3171221.3171276"},{"key":"e_1_3_4_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2006.08.004"},{"key":"e_1_3_4_59_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0704452104"},{"key":"e_1_3_4_60_2","volume-title":"How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction","author":"Malle Bertram F.","year":"2006","unstructured":"Bertram F. Malle. 2006. How the Mind Explains Behavior: Folk Explanations, Meaning, and Social Interaction. MIT Press."},{"key":"e_1_3_4_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434074.3447130"},{"key":"e_1_3_4_62_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911430417"},{"key":"e_1_3_4_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_3_4_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559656"},{"key":"e_1_3_4_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(87)80013-5"},{"key":"e_1_3_4_66_2","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8624.00532"},{"key":"e_1_3_4_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594256"},{"key":"e_1_3_4_68_2","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(82)90012-1"},{"key":"e_1_3_4_69_2","volume-title":"International Conference on Human-robot Interaction","author":"Nguyen Hai","year":"2008","unstructured":"Hai Nguyen, Cressel Anderson, Alexander Trevor, Advait Jain, Zhe Xu, and Charles C. Kemp. 2008. El-e: An assistive robot that fetches objects from flat surfaces. In International Conference on Human-robot Interaction."},{"key":"e_1_3_4_70_2","article-title":"WaveNet: A generative model for raw audio","author":"Oord Aaron van den","year":"2016","unstructured":"Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. WaveNet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016).","journal-title":"arXiv preprint arXiv:1609.03499"},{"key":"e_1_3_4_71_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbef.2017.12.004"},{"key":"e_1_3_4_72_2","first-page":"1","volume-title":"CHI Conference on Human Factors in Computing Systems","author":"Poursabzi-Sangdeh Forough","year":"2021","unstructured":"Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In CHI Conference on Human Factors in Computing Systems. 1\u201352."},{"key":"e_1_3_4_73_2","doi-asserted-by":"publisher","DOI":"10.1177\/0018720810381071"},{"key":"e_1_3_4_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICHR.2005.1573565"},{"key":"e_1_3_4_75_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jecp.2019.104702"},{"key":"e_1_3_4_76_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-009-9160-9"},{"key":"e_1_3_4_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319502.3374802"},{"key":"e_1_3_4_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371382.3374849"},{"key":"e_1_3_4_79_2","doi-asserted-by":"publisher","DOI":"10.1037\/0021-9010.90.4.692"},{"key":"e_1_3_4_80_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.01962"},{"key":"e_1_3_4_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2016.7451741"},{"key":"e_1_3_4_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3380783"},{"key":"e_1_3_4_83_2","doi-asserted-by":"publisher","DOI":"10.29007\/hxkz"},{"key":"e_1_3_4_84_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0159920"},{"key":"e_1_3_4_85_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315665177"},{"key":"e_1_3_4_86_2","doi-asserted-by":"publisher","DOI":"10.1515\/pjbr-2021-0023"},{"key":"e_1_3_4_87_2","volume-title":"Workshop on Autonomous Mobile Service Robots","author":"Wise Melonee","year":"2016","unstructured":"Melonee Wise, Michael Ferguson, Derek King, Eric Diehr, and David Dymesich. 2016. Fetch and freight: Standard platforms for service robot applications. In Workshop on Autonomous Mobile Service Robots."},{"key":"e_1_3_4_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3239552"},{"key":"e_1_3_4_89_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-62056-1_8"},{"key":"e_1_3_4_90_2","doi-asserted-by":"publisher","DOI":"10.1080\/15250000802569884"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3568024","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3568024","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3568024","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T21:26:14Z","timestamp":1750281974000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3568024"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,15]]},"references-count":89,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,3,31]]}},"alternative-id":["10.1145\/3568024"],"URL":"https:\/\/doi.org\/10.1145\/3568024","relation":{},"ISSN":["2573-9522","2573-9522"],"issn-type":[{"value":"2573-9522","type":"print"},{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,15]]},"assertion":[{"value":"2021-08-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-22","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}