{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T02:40:16Z","timestamp":1792809616824,"version":"4.5.0"},"reference-count":54,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"68","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2022,7,27]]},"abstract":"<jats:p>A prerequisite for social coordination is bidirectional communication between teammates, each playing two roles simultaneously: as receptive listeners and expressive speakers. For robots working with humans in complex situations with multiple goals that differ in importance, failure to fulfill the expectation of either role could undermine group performance due to misalignment of values between humans and robots. Specifically, a robot needs to serve as an effective listener to infer human users\u2019 intents from instructions and feedback and as an expressive speaker to explain its decision processes to users. Here, we investigate how to foster effective bidirectional human-robot communications in the context of value alignment\u2014collaborative robots and users form an aligned understanding of the importance of possible task goals. We propose an explainable artificial intelligence (XAI) system in which a group of robots predicts users\u2019 values by taking in situ feedback into consideration while communicating their decision processes to users through explanations. To learn from human feedback, our XAI system integrates a cooperative communication model for inferring human values associated with multiple desirable goals. To be interpretable to humans, the system simulates human mental dynamics and predicts optimal explanations using graphical models. We conducted psychological experiments to examine the core components of the proposed computational framework. Our results show that real-time human-robot mutual understanding in complex cooperative tasks is achievable with a learning model based on bidirectional communication. We believe that this interaction framework can shed light on bidirectional value alignment in communicative XAI systems and, more broadly, in future human-machine teaming systems.<\/jats:p>","DOI":"10.1126\/scirobotics.abm4183","type":"journal-article","created":{"date-parts":[[2022,7,13]],"date-time":"2022-07-13T13:57:08Z","timestamp":1657720628000},"source":"Crossref","is-referenced-by-count":78,"title":["In situ bidirectional human-robot value alignment"],"prefix":"10.1126","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6624-1227","authenticated-orcid":true,"given":"Luyao","family":"Yuan","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of California, Los Angeles, Los Angeles, CA 90095, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3331-9846","authenticated-orcid":true,"given":"Xiaofeng","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California, Los Angeles, Los Angeles, CA 90095, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1219-5151","authenticated-orcid":true,"given":"Zilong","family":"Zheng","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Los Angeles, Los Angeles, CA 90095, USA."},{"name":"Beijing Institute for General Artificial Intelligence (BIGAI), Beijing 100080, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9136-1714","authenticated-orcid":true,"given":"Mark","family":"Edmonds","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Los Angeles, Los Angeles, CA 90095, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8029-3664","authenticated-orcid":true,"given":"Ying Nian","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California, Los Angeles, Los Angeles, CA 90095, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6544-7685","authenticated-orcid":true,"given":"Federico","family":"Rossano","sequence":"additional","affiliation":[{"name":"Department of Cognitive Science, University of California, San Diego, San Diego, CA 92093, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0660-1176","authenticated-orcid":true,"given":"Hongjing","family":"Lu","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California, Los Angeles, Los Angeles, CA 90095, USA."},{"name":"Department of Psychology, University of California, Los Angeles, Los Angeles, CA 90095, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7024-1545","authenticated-orcid":true,"given":"Yixin","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California, Los Angeles, Los Angeles, CA 90095, USA."},{"name":"Beijing Institute for General Artificial Intelligence (BIGAI), Beijing 100080, China."},{"name":"Institute for Artificial Intelligence, Peking University, Beijing 100871, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1925-5973","authenticated-orcid":true,"given":"Song-Chun","family":"Zhu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California, Los Angeles, Los Angeles, CA 90095, USA."},{"name":"Department of Statistics, University of California, Los Angeles, Los Angeles, CA 90095, USA."},{"name":"Beijing Institute for General Artificial Intelligence (BIGAI), Beijing 100080, China."},{"name":"Institute for Artificial Intelligence, Peking University, Beijing 100871, China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.131.3410.1355"},{"key":"e_1_3_2_3_2","first-page":"403","article-title":"Team mental model: Construct or metaphor?","volume":"20","author":"Klimoski R.","year":"1994","unstructured":"R. Klimoski, S. Mohammed, Team mental model: Construct or metaphor? J. Manage. 20, 403\u2013437 (1994).","journal-title":"J. Manage."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1075\/is.8.3.10gro"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"S. H. Schwartz Advances in Experimental Social Psychology (Elsevier 1992) vol. 25 pp. 1\u201365.","DOI":"10.1016\/S0065-2601(08)60281-6"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/21.199457"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"J. MacMillan E. E. Entin D. Serfaty Communication overhead: The hidden cost of team cognition in Team Cognition: Understanding the Factors That Drive Process and Performance E. Salas S. M. Fiore Eds. (American Psychological Association 2004).","DOI":"10.1037\/10690-004"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1177\/0018720816639712"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"V. V. Unhelkar S. Li J. A. Shah Decision-making for bidirectional communication in sequential human-robot collaborative tasks in Proceedings of the 2020 ACM\/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE 2020) pp. 329\u2013341.","DOI":"10.1145\/3319502.3374779"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"P. Abbeel A. Y. Ng Apprenticeship learning via inverse reinforcement learning in Proceedings of the Twenty-First International Conference on Machine Learning (ICML) (Association for Computing Machinery 2004.","DOI":"10.1145\/1015330.1015430"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"W. B. Knox P. Stone Interactively shaping agents via human reinforcement: The TAMER framework in Proceedings of the Fifth International Conference on Knowledge Capture (Association for Computing Machinery 2009).","DOI":"10.1145\/1597735.1597738"},{"key":"e_1_3_2_12_2","unstructured":"S. Griffith K. Subramanian J. Scholz C. L. Isbell A. L. Thomaz Policy shaping: Integrating human feedback with reinforcement learning in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2013)."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aay4663"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"M. T. Ribeiro S. Singh C. Guestrin \u201cWhy should I trust you?\u201d: Explaining the predictions of any classifier in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining Association for Computing Machinery 2016).","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"H. Liu Y. Zhang W. Si X. Xie Y. Zhu S.-C. Zhu Interactive robot knowledge patching using augmented reality in Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2018).","DOI":"10.1109\/ICRA.2018.8462837"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2982882"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Z. Zhang Y. Zhu S.-C. Zhu Graph-based hierarchical knowledge representation for robot task transfer from virtual to physical world in Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE 2021).","DOI":"10.1109\/IROS45743.2020.9340843"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"T. Chakraborti S. Sreedharan Y. Zhang S. Kambhampati Plan explanations as model reconciliation: Moving beyond explanation as soliloquy in Proceedings of International Joint Conference on Artificial Intelligence (IJCAI) (AAAI Press 2017) pp. 156\u2013163.","DOI":"10.24963\/ijcai.2017\/23"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"Z. Gong Y. Zhang Behavior explanation as intention signaling in human-robot teaming in Proceedings of International Symposium on Robot and Human Interactive Communication (RO-MAN) (IEEE 2018) pp. 1005\u20131011.","DOI":"10.1109\/ROMAN.2018.8525675"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"A. Tabrez S. Agrawal B. Hayes Explanation-based reward coaching to improve human performance via reinforcement learning in Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE 2019).","DOI":"10.1109\/HRI.2019.8673104"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-018-9771-0"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"T. Yuan H. Liu. L. Fan Z. Zhen T. Gao Y. Zhu S.-C. Zhu Joint inference of states robot knowledge and human (false-) beliefs in Proceedings of International Conference on Robotics and Automation (ICRA) (IEEE 2020).","DOI":"10.1109\/ICRA40945.2020.9197355"},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","unstructured":"X. Gao R. Gong Y. Zhao S. Wang T. Shu S.-C. Zhu Joint mind modeling for explanation generation in complex human-robot collaborative tasks in 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN) (IEEE 2020) pp. 1119\u20131126.","DOI":"10.1109\/RO-MAN47096.2020.9223595"},{"key":"e_1_3_2_24_2","unstructured":"S. Russell Human Compatible: Artificial Intelligence and the Problem of Control (Penguin 2019)."},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"H. H. Clark Using Language (Cambridge Univ. Press 1996).","DOI":"10.1017\/CBO9780511620539"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.2307\/2548836"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"B. A. Huberman The ecology of computation in Digest of Papers. COMPCON Spring 89. Thirty-Fourth IEEE Computer Society International Conference: Intellectual Leverage (IEEE 1988) p. 362.","DOI":"10.1109\/CMPCON.1989.301957"},{"key":"e_1_3_2_28_2","unstructured":"M. K. Ho M. Littman J. MacGlashan F. Cushman J. L. Austerweil Showing versus doing: Teaching by demonstration in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2016)."},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"H. P. Grice in Speech Acts (Brill 1975) pp. 41\u201358.","DOI":"10.1163\/9789004368811_003"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1111\/tops.12007"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogpsych.2013.12.004"},{"key":"e_1_3_2_32_2","unstructured":"L. Yuan D. Zhou J. Shen J. Gao J. L. Chen Q. Gu Y. N. Wu S.-C. Zhu Iterative teacher-aware learning in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2021)."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8721.2007.00517.x"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2020.1807710"},{"key":"e_1_3_2_35_2","first-page":"16","article-title":"No AI is an island: The case for teaming intelligence","volume":"40","author":"Johnson M.","year":"2019","unstructured":"M. Johnson, A. Vera, No AI is an island: The case for teaming intelligence. AI Mag. 40, 16\u201328 (2019).","journal-title":"AI Mag."},{"key":"e_1_3_2_36_2","unstructured":"A. Barreto W. Dabney R. Munos J. J. Hunt T. Schaul H. P . van Hasselt D. Silver Successor features for transfer in reinforcement learning in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2017)."},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"N. Wang D. V. Pynadath S. G. Hill Trust calibration within a human-robot team: Comparing automatically generated explanations in Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE 2016).","DOI":"10.1109\/HRI.2016.7451741"},{"key":"e_1_3_2_38_2","first-page":"21","article-title":"The computer as a communication device","volume":"76","author":"Licklider J. C.","year":"1968","unstructured":"J. C. Licklider, R. W. Taylor, The computer as a communication device. Sci. Technol. 76, 21\u201331 (1968).","journal-title":"Sci. Technol."},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","unstructured":"L. B. Resnick J. M. Levine S. Behrend Socially Shared Cognition (American Psychological Association 1991).","DOI":"10.1037\/10096-000"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103500"},{"key":"e_1_3_2_41_2","unstructured":"D. Silver J. Veness Monte-Carlo planning in large POMDPs in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2010)."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s42113-018-0005-5"},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"M. Chen S. Nikolaidis H. Soh D. Hsu S. Srinivasa Planning with trust for human-robot collaboration in Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE 2018) pp. 307\u2013315.","DOI":"10.1145\/3171221.3171264"},{"key":"e_1_3_2_44_2","unstructured":"N. J. Smith N. Goodman M. Frank Learning and using language via recursive pragmatic reasoning about other agents in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2013)."},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"R. Carston Informativeness relevance and scalar implicature in Pragmatics And Beyond New Series (John Benjamins Publishing Company 1998) pp. 179\u2013238.","DOI":"10.1075\/pbns.37.11car"},{"key":"e_1_3_2_46_2","unstructured":"A. Vogel M. Bodoia C. Potts D. Jurafsky Emergence of gricean maxims from multi-agent decision theory in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) (ACM 2013)."},{"key":"e_1_3_2_47_2","unstructured":"W. Liu B. Dai Z. Li Z. Liu J. Rehg L. Song Towards black-box iterative machine teaching in Proceedings of International Conference on Machine Learning (ICML) (PMLR 2018)."},{"key":"e_1_3_2_48_2","unstructured":"P. Wang J. Wang P. Paranamana P. Shafto A mathematical theory of cooperative communication in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2020)."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-015-9317-1"},{"key":"e_1_3_2_50_2","unstructured":"T. Peltola M. M. \u00c7elikok P. Daee S. Kaski Machine teaching of active sequential learners in Proceedings of Advances in Neural Information Processing Systems (NeurIPS) (IEEE 2019)."},{"key":"e_1_3_2_51_2","unstructured":"A. Lock The Guided Reinvention of Language (Academic Press 1980)."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"M. Tomasello J. Call Primate Cognition (Oxford Univ. Press 1997).","DOI":"10.1093\/oso\/9780195106237.001.0001"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","unstructured":"M. Tomasello Do apes ape in Social Learning in Animals: The Roots of Culture C. M. Heyes B. G. Galef Jr. Eds. (Academic Press 1996) pp. 319\u2013346.","DOI":"10.1016\/B978-012273965-1\/50016-9"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/34.1000239"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1063\/1.1699114"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.abm4183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,28]],"date-time":"2024-09-28T22:29:36Z","timestamp":1727562576000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.abm4183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,27]]},"references-count":54,"journal-issue":{"issue":"68","published-print":{"date-parts":[[2022,7,27]]}},"alternative-id":["10.1126\/scirobotics.abm4183"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.abm4183","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,27]]},"article-number":"eabm4183"}}