{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T18:45:54Z","timestamp":1781894754424,"version":"3.54.5"},"reference-count":105,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,2,25]],"date-time":"2023-02-25T00:00:00Z","timestamp":1677283200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Federal German Government and the Bavarian Government (StMWi), joint project Next generation Human-Centered Robotics Human embodiment and system agency in trustworthy AI for the Future of health"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>In human\u2013robot interaction, transparency is essential to ensure that humans understand and trust robots. Understanding is vital from an ethical perspective and benefits interaction, e.g., through appropriate trust. While there is research on explanations and their content, the methods used to convey the explanations are underexplored. It remains unclear which approaches are used to foster understanding. To this end, we contribute a systematic literature review exploring how robot transparency is fostered in papers published in the ACM Digital Library and IEEE Xplore. We found that researchers predominantly rely on monomodal visual or verbal explanations to foster understanding. Commonly, these explanations are external, as opposed to being integrated in the robot design. This paper provides an overview of how transparency is communicated in human\u2013robot interaction research and derives a classification with concrete recommendations for communicating transparency. Our results establish a solid base for consistent, transparent human\u2013robot interaction designs.<\/jats:p>","DOI":"10.3390\/mti7030025","type":"journal-article","created":{"date-parts":[[2023,2,27]],"date-time":"2023-02-27T02:10:46Z","timestamp":1677463846000},"page":"25","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["A Literature Survey of How to Convey Transparency in Co-Located Human\u2013Robot Interaction"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1281-0230","authenticated-orcid":false,"given":"Svenja Y.","family":"Sch\u00f6tt","sequence":"first","affiliation":[{"name":"LMU Munich, Frauenlobstr. 7a, 80337 Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4279-7778","authenticated-orcid":false,"given":"Rifat Mehreen","family":"Amin","sequence":"additional","affiliation":[{"name":"LMU Munich, Frauenlobstr. 7a, 80337 Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9007-9888","authenticated-orcid":false,"given":"Andreas","family":"Butz","sequence":"additional","affiliation":[{"name":"LMU Munich, Frauenlobstr. 7a, 80337 Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1145\/3209701","article-title":"The Science of Human-Robot Interaction","volume":"7","author":"Kiesler","year":"2018","journal-title":"ACM Trans. Hum. Robot. Interact."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Wortham, R.H., Theodorou, A., and Bryson, J.J. (2017, January 28\u201331). Improving robot transparency: Real-time visualisation of robot AI substantially improves understanding in naive observers. Proceedings of the 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Lisbon, Portugal.","DOI":"10.1109\/ROMAN.2017.8172491"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"83","DOI":"10.3389\/fnbot.2018.00083","article-title":"System Transparency in Shared Autonomy: A Mini Review","volume":"12","author":"Alonso","year":"2018","journal-title":"Front. Neurorobotics"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nesset, B., Robb, D.A., Lopes, J., and Hastie, H. (2021, January 8\u201311). Transparency in HRI: Trust and Decision Making in the Face of Robot Errors. Proceedings of the Companion of the 2021 ACM\/IEEE International Conference on Human-Robot Interaction, Boulder, CO, USA.","DOI":"10.1145\/3434074.3447183"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1109\/MTS.2020.2967493","article-title":"Robot Enhanced Therapy for Autistic Children: An Ethical Analysis","volume":"39","author":"McBride","year":"2020","journal-title":"IEEE Technol. Soc. Mag."},{"key":"ref_6","unstructured":"Lyons, J.B. (2013, January 25\u201327). Being transparent about transparency: A model for human-robot interaction. Proceedings of the 2013 AAAI Spring Symposium Series, Palo Alto, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cila, N. (2022, January 23\u201328). Designing Human-Agent Collaborations: Commitment, Responsiveness, and Support. Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI\u201922), New Orleans, MI, USA.","DOI":"10.1145\/3491102.3517500"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1109\/MRA.2019.2904644","article-title":"Robots and Transparency: The Multiple Dimensions of Transparency in the Context of Robot Technologies","volume":"26","author":"Felzmann","year":"2019","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1080\/09540091.2017.1310182","article-title":"Designing and implementing transparency for real time inspection of autonomous robots","volume":"29","author":"Theodorou","year":"2017","journal-title":"Connect. Sci."},{"key":"ref_10","first-page":"3457188","article-title":"Explainable Embodied Agents Through Social Cues: A Review","volume":"10","author":"Tulli","year":"2021","journal-title":"J. Hum. Robot Interact."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"264","DOI":"10.7326\/0003-4819-151-4-200908180-00135","article-title":"Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement","volume":"151","author":"Moher","year":"2009","journal-title":"Ann. Intern. Med."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Sui, Z., and Jenkins, O.C. (2018, January 1\u20135). Plenoptic Monte Carlo Object Localization for Robot Grasping Under Layered Translucency. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593629"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1109\/TRO.2008.924262","article-title":"Dynamic Performance of Mobile Haptic Interfaces","volume":"24","author":"Formaglio","year":"2008","journal-title":"IEEE Trans. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zanotto, D., Lenzi, T., Stegall, P., and Agrawal, S.K. (2013, January 24\u201326). Improving transparency of powered exoskeletons using force\/torque sensors on the supporting cuffs. Proceedings of the 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR), Seattle, WA, USA.","DOI":"10.1109\/ICORR.2013.6650404"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., and Hussmann, H. (2018, January 7\u201311). Bringing Transparency Design into Practice. Proceedings of the 23rd International Conference on Intelligent User Interfaces (IUI \u201918), Tokyo, Japan.","DOI":"10.1145\/3172944.3172961"},{"key":"ref_16","unstructured":"Norman, D.A. (2014). Mental Models, Psychology Press."},{"key":"ref_17","unstructured":"Gross, H.M., and Boehme, H.J. (2000, January 8\u201311). PERSES\u2014A vision-based interactive mobile shopping assistant. Proceedings of the 2000 IEEE International Conference on Systems, man and Cybernetics: \u2019Cybernetics Evolving to Systems, Humans, Organizations, and Their Complex Interactions\u2019, Nashville, TN, USA."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Eiband, M., Buschek, D., and Hussmann, H. (2021, January 14\u201317). How to support users in understanding intelligent systems? Structuring the discussion. Proceedings of the 26th International Conference on Intelligent User Interfaces, College Station, TX, USA.","DOI":"10.1145\/3397481.3450694"},{"key":"ref_19","first-page":"34","article-title":"Evaluation of a Humanoid Robot\u2019s Emotional Gestures for Transparent Interaction","volume":"11","author":"Rossi","year":"2021","journal-title":"Soc. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1145\/3241365","article-title":"Overtrust in the robotic age","volume":"61","author":"Wagner","year":"2018","journal-title":"Commun. ACM"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Powers, A., and Kiesler, S. (2006, January 2\u20133). The advisor robot: Tracing people\u2019s mental model from a robot\u2019s physical attributes. Proceedings of the 1st ACM SIGCHI\/SIGART conference on Human-Robot Interaction - HRI \u201906, Salt Lake City, UT, USA.","DOI":"10.1145\/1121241.1121280"},{"key":"ref_22","unstructured":"Molnar, C. (2022, September 08). Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. Available online: Lulu.com."},{"key":"ref_23","unstructured":"Anjomshoae, S., Najjar, A., Calvaresi, D., and Fr\u00e4mling, K. (2019, January 13\u201317). Explainable agents and robots: Results from a systematic literature review. Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019), Montreal, QC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1177\/0018720811417254","article-title":"A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction","volume":"53","author":"Hancock","year":"2011","journal-title":"Hum. Factors J. Hum. Factors Ergon. Soc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Esterwood, C., and Robert, L.P. (2020, January 10\u201313). Personality in Healthcare Human Robot Interaction H-HRI: A Literature Review and Brief Critique. Proceedings of the 8th International Conference on Human-Agent Interaction, Online.","DOI":"10.1145\/3406499.3415075"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, T., Du, Q., Yang, G., Chen, C.y., Wang, C., and Fang, Z. (2021, January 1\u20134). A Review of Compliant Control for Collaborative Robots. Proceedings of the 2021 IEEE 16th Conference on Industrial Electronics and Applications (ICIEA), Chengdu, China.","DOI":"10.1109\/ICIEA51954.2021.9516193"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Deng, Y., Tang, Y., Yang, B., Zheng, W., Liu, S., and Liu, C. (2021, January 3\u20135). A Review of Bilateral Teleoperation Control Strategies with Soft Environment. Proceedings of the 2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM), Chongqing, China.","DOI":"10.1109\/ICARM52023.2021.9536056"},{"key":"ref_28","unstructured":"Lee, H.K., Shin, M.H., and Chung, M.J. (1997, January 7\u20139). Adaptive controller of master-slave systems for transparent teleoperation. Proceedings of the 1997 8th International Conference on Advanced Robotics. Proceedings. ICAR\u201997, Monterey, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1109\/THMS.2021.3107516","article-title":"Levels of Automation and Transparency: Interaction Design Considerations in Assistive Robots for Older Adults","volume":"51","author":"Olatunji","year":"2021","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1016\/j.jclinepi.2009.06.006","article-title":"The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration","volume":"62","author":"Liberati","year":"2009","journal-title":"J. Clin. Epidemiol."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Kalra, A., Taamazyan, V., Rao, S.K., Venkataraman, K., Raskar, R., and Kadambi, A. (2020, January 13\u201319). Deep Polarization Cues for Transparent Object Segmentation. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00863"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Martinson, E., and Brock, D. (2007, January 9\u201311). Improving Human-Robot Interaction through Adaptation to the Auditory Scene. Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201907), Washington, DC, USA.","DOI":"10.1145\/1228716.1228732"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J. Chiropr. Med."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Moon, A., Troniak, D.M., Gleeson, B., Pan, M.K., Zheng, M., Blumer, B.A., MacLean, K., and Croft, E.A. (2014, January 3\u20136). Meet Me Where I\u2019m Gazing: How Shared Attention Gaze Affects Human-Robot Handover Timing. Proceedings of the 2014 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201914), Bielefeld, Germany.","DOI":"10.1145\/2559636.2559656"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Dragan, A.D., Lee, K.C., and Srinivasa, S.S. (2013, January 3\u20136). Legibility and predictability of robot motion. Proceedings of the 2013 8th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, Japan.","DOI":"10.1109\/HRI.2013.6483603"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sheikholeslami, S., Hart, J.W., Chan, W.P., Quintero, C.P., and Croft, E.A. (2018, January 5\u20138). Prediction and Production of Human Reaching Trajectories for Human-Robot Interaction. Proceedings of the Companion of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201918), Chicago, IL, USA.","DOI":"10.1145\/3173386.3176924"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Gong, Z., and Zhang, Y. (2018, January 27\u201331). Behavior Explanation as Intention Signaling in Human-Robot Teaming. Proceedings of the 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Nanjing, China.","DOI":"10.1109\/ROMAN.2018.8525675"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kulkarni, A., Sreedharan, S., Keren, S., Chakraborti, T., Smith, D.E., and Kambhampati, S. (2020, January 25\u201329). Designing Environments Conducive to Interpretable Robot Behavior. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340832"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Arntz, A., Eimler, S.C., Stra\u00dfmann, C., and Hoppe, H.U. (2021, January 8\u201311). On the Influence of Autonomy and Transparency on Blame and Credit in Flawed Human-Robot Collaboration. Proceedings of the Companion of the 2021 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201921), Boulder, CO, USA.","DOI":"10.1145\/3434074.3447196"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4289","DOI":"10.1109\/LRA.2018.2865034","article-title":"Operator Awareness in Human\u2013Robot Collaboration Through Wearable Vibrotactile Feedback","volume":"3","author":"Casalino","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Grushko, S., Vysock\u00fd, A., O\u0161\u010d\u00e1dal, P., Vocetka, M., Nov\u00e1k, P., and Bobovsk\u00fd, Z. (2021). Improved Mutual Understanding for Human-Robot Collaboration: Combining Human-Aware Motion Planning with Haptic Feedback Devices for Communicating Planned Trajectory. Sensors, 21.","DOI":"10.3390\/s21113673"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Grushko, S., Vysock\u00fd, A., Heczko, D., and Bobovsk\u00fd, Z. (2021). Intuitive Spatial Tactile Feedback for Better Awareness about Robot Trajectory during Human\u2013Robot Collaboration. Sensors, 21.","DOI":"10.3390\/s21175748"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5681","DOI":"10.1109\/LRA.2021.3083465","article-title":"Expectations Vs. Reality: Unreliability and Transparency in a Treasure Hunt Game With Icub","volume":"6","author":"Aroyo","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Attia, M., Hossny, M., Nahavandi, S., Dalvand, M., and Asadi, H. (2018, January 7\u201310). Towards Trusted Autonomous Surgical Robots. Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan.","DOI":"10.1109\/SMC.2018.00692"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1109\/TAFFC.2019.2893348","article-title":"Towards Transparent Robot Learning Through TDRL-Based Emotional Expressions","volume":"12","author":"Broekens","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Cantucci, F., and Falcone, R. (2020, January 7\u20139). Towards trustworthiness and transparency in social human-robot interaction. Proceedings of the 2020 IEEE International Conference on Human-Machine Systems (ICHMS), Rome, Italy.","DOI":"10.1109\/ICHMS49158.2020.9209397"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1109\/TRO.2020.2964824","article-title":"Efficient and Trustworthy Social Navigation via Explicit and Implicit Robot-Human Communication","volume":"36","author":"Che","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Chen, S., Boggess, K., and Feng, L. (2020, January 25\u201329). Towards Transparent Robotic Planning via Contrastive Explanations. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341773"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Das, D., Banerjee, S., and Chernova, S. (2021, January 13\u201316). Explainable AI for Robot Failures: Generating Explanations That Improve User Assistance in Fault Recovery. Proceedings of the 2021 International Conference on Human-Robot Interaction (HRI \u201921), Boulder, CO, USA.","DOI":"10.1145\/3434073.3444657"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Diethelm, I.G., Hansen, S.S., Leth, F.B., Fischer, K., and Palinko, O. (2021, January 8\u201311). Effects of Gaze and Speech in Human-Robot Medical Interactions. Proceedings of the Companion of the 2021 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201921), Boulder, CO, USA.","DOI":"10.1145\/3434074.3447190"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Hayes, B., and Shah, J.A. (2017, January 6\u20139). Improving Robot Controller Transparency Through Autonomous Policy Explanation. Proceedings of the 2017 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201917), Vienna, Austria.","DOI":"10.1145\/2909824.3020233"},{"key":"ref_52","unstructured":"Hindemith, L., Vollmer, A.L., Wiebel-Herboth, C.B., and Wrede, B. (2021). Improving HRI through robot architecture transparency. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Hirschmanner, M., Gross, S., Zafari, S., Krenn, B., Neubarth, F., and Vincze, M. (2021, January 8\u201312). Investigating Transparency Methods in a Robot Word-Learning System and Their Effects on Human Teaching Behaviors. Proceedings of the 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), Vancouver, BC, Canada.","DOI":"10.1109\/RO-MAN50785.2021.9515518"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Iucci, A., Hata, A., Terra, A., Inam, R., and Leite, I. (2021, January 6\u201310). Explainable Reinforcement Learning for Human-Robot Collaboration. Proceedings of the 2021 20th International Conference on Advanced Robotics (ICAR), Ljubljana, Slovenia.","DOI":"10.1109\/ICAR53236.2021.9659472"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Kallinen, K. (2017, January 6\u20139). The Effects of Transparency and Task Type on Trust, Stress, Quality of Work, and Co-Worker Preference During Human-Autonomous System Collaborative Work. Proceedings of the Companion of the 2017 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201917), Vienna, Austria.","DOI":"10.1145\/3029798.3038386"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Kaptein, F., Broekens, J., Hindriks, K., and Neerincx, M. (2019, January 3\u20136). Evaluating Cognitive and Affective Intelligent Agent Explanations in a Long-Term Health-Support Application for Children with Type 1 Diabetes. Proceedings of the 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), Cambridge, UK.","DOI":"10.1109\/ACII.2019.8925526"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Kim, T., and Hinds, P. (2006, January 6\u20138). Who Should I Blame? Effects of Autonomy and Transparency on Attributions in Human-Robot Interaction. In Proceedings of the ROMAN 2006\u2014The 15th IEEE International Symposium on Robot and Human Interactive Communication, Hatfield, UK.","DOI":"10.1109\/ROMAN.2006.314398"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Kwon, M., Huang, S.H., and Dragan, A.D. (2018, January 5\u20138). Expressing Robot Incapability. Proceedings of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201918), Chicago, IL, USA.","DOI":"10.1145\/3171221.3171276"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1109\/THMS.2021.3116119","article-title":"Toward Robots-ehavioral Transparency of Temporal Difference Reinforcement Learning With a Human Teacher","volume":"51","author":"Matarese","year":"2021","journal-title":"Trans. Hum. Mach. Syst."},{"key":"ref_60","unstructured":"Mobahi, H., and Ansari, S. (2003, January 8). Fuzzy perception, emotion and expression for interactive robots. Proceedings of the 2003 IEEE International Conference on Systems, Man and Cybernetics (SMC\u201903), Washington, DC, USA. Conference Theme\u2014System Security and Assurance (Cat. No.03CH37483)."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Mota, T., and Sridharan, M. (2021, January 23\u201326). Answer me this: Constructing Disambiguation Queries for Explanation Generation in Robotics. Proceedings of the 2021 IEEE International Conference on Development and Learning (ICDL), Beijing, China.","DOI":"10.1109\/ICDL49984.2021.9515605"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Murphy, R.R., Srinivasan, V., Henkel, Z., Suarez, J., Minson, M., Straus, J.C., Hempstead, S., Valdez, T., and Egawa, S. (2013, January 12\u201314). Interacting with trapped victims using robots. Proceedings of the 2013 IEEE International Conference on Technologies for Homeland Security (HST), Waltham, MA, USA.","DOI":"10.1109\/THS.2013.6698972"},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Niu, S., McCrickard, D.S., and Harrison, S. (2015, January 1\u20135). Exploring humanoid factors of robots through transparent and reflective interactions. Proceedings of the 2015 International Conference on Collaboration Technologies and Systems (CTS), Atlanta, GA, USA.","DOI":"10.1109\/CTS.2015.7210397"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Poulsen, A., Burmeister, O.K., and Tien, D. (2018, January 4\u20136). Care Robot Transparency Isn\u2019t Enough for Trust. Proceedings of the 2018 IEEE Region Ten Symposium (Tensymp), Sydney, NSW, Australia.","DOI":"10.1109\/TENCONSpring.2018.8692047"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Roncone, A., Mangin, O., and Scassellati, B. (June, January 29). Transparent role assignment and task allocation in human robot collaboration. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989122"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Rotsidis, A., Theodorou, A., Bryson, J.J., and Wortham, R.H. (2019, January 14\u201318). Improving Robot Transparency: An Investigation With Mobile Augmented Reality. Proceedings of the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India.","DOI":"10.1109\/RO-MAN46459.2019.8956390"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Sanders, T.L., Wixon, T., Schafer, K.E., Chen, J.Y.C., and Hancock, P.A. (2014, January 3\u20136). The influence of modality and transparency on trust in human-robot interaction. Proceedings of the 2014 IEEE International Inter-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), San Antonio, TX, USA.","DOI":"10.1109\/CogSIMA.2014.6816556"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"3365668","DOI":"10.1145\/3365668","article-title":"Transparency about a Robot\u2019s Lack of Human Psychological Capacities: Effects on Child-Robot Perception and Relationship Formation","volume":"9","author":"Straten","year":"2020","journal-title":"J. Hum. Robot Interact."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Struckmeier, O., Racca, M., and Kyrki, V. (2019, January 14\u201318). Autonomous Generation of Robust and Focused Explanations for Robot Policies. Proceedings of the 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), New Delhi, India.","DOI":"10.1109\/RO-MAN46459.2019.8956323"},{"key":"ref_70","unstructured":"Tabrez, A., Luebbers, M.B., and Hayes, B. (June, January 29). Descriptive and Prescriptive Visual Guidance to Improve Shared Situational Awareness in Human-Robot Teaming. Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS \u201922), London, UK."},{"key":"ref_71","unstructured":"Tojo, T., Matsusaka, Y., Ishii, T., and Kobayashi, T. (2000, January 8\u201311). A conversational robot utilizing facial and body expressions. Proceedings of the 2000 IEEE International Conference on Systems, Man and Cybernetics: \u201cCybernetics Evolving to Systems, Humans, Organizations, and their Complex Interactions\u201d, Nashville, TN, USA."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Valdivia, A.A., Shailly, R., Seth, N., Fuentes, F., Losey, D.P., and Blumenschein, L.H. (2022, January 4\u20138). Wrapped Haptic Display for Communicating Physical Robot Learning. Proceedings of the 2022 IEEE 5th International Conference on Soft Robotics (RoboSoft), Edinburgh, UK.","DOI":"10.1109\/RoboSoft54090.2022.9762210"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Virgolin, M., Bellone, M., Wolff, K., and Wahde, M. (2021, January 15\u201317). A Mobile Interactive Robot for Social Distancing in Hospitals. Proceedings of the 2021 Fifth IEEE International Conference on Robotic Computing (IRC), Taichung, Taiwan.","DOI":"10.1109\/IRC52146.2021.00020"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Vitale, J., Tonkin, M., Herse, S., Ojha, S., Clark, J., Williams, M.A., Wang, X., and Judge, W. (2018, January 5\u20138). Be More Transparent and Users Will Like You: A Robot Privacy and User Experience Design Experiment. Proceedings of the 2018 ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201918), Chicago, IL, USA.","DOI":"10.1145\/3171221.3171269"},{"key":"ref_75","unstructured":"Wang, N., Pynadath, D.V., and Hill, S.G. (2016, January 8\u201312). The Impact of POMDP-Generated Explanations on Trust and Performance in Human-Robot Teams. Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems (AAMAS \u201916), Sao Paulo, Brazil."},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Wang, N., Pynadath, D.V., and Hill, S.G. (2016, January 7\u201310). Trust Calibration within a Human-Robot Team: Comparing Automatically Generated Explanations. Proceedings of the The Eleventh ACM\/IEEE International Conference on Human Robot Interaction (HRI \u201916), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451741"},{"key":"ref_77","doi-asserted-by":"crossref","unstructured":"Wengefeld, T., H\u00f6chemer, D., Lewandowski, B., K\u00f6hler, M., Beer, M., and Gross, H.M. (September, January 31). A Laser Projection System for Robot Intention Communication and Human Robot Interaction. Proceedings of the 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy.","DOI":"10.1109\/RO-MAN47096.2020.9223517"},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Yigitbas, E., Karakaya, K., Jovanovikj, I., and Engels, G. (2021, January 18\u201321). Enhancing Human-in-the-Loop Adaptive Systems through Digital Twins and VR Interfaces. Proceedings of the 2021 International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS), Madrid, Spain.","DOI":"10.1109\/SEAMS51251.2021.00015"},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"eaay4663","DOI":"10.1126\/scirobotics.aay4663","article-title":"A tale of two explanations: Enhancing human trust by explaining robot behavior","volume":"4","author":"Edmonds","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Gao, X., Gong, R., Zhao, Y., Wang, S., Shu, T., and Zhu, S.C. (September, January 31). Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks. Proceedings of the 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), Naples, Italy.","DOI":"10.1109\/RO-MAN47096.2020.9223595"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Matsumaru, T. (2007, January 10\u201314). Mobile Robot with Preliminary-announcement and Indication Function of Forthcoming Operation using Flat-panel Display. Proceedings of the 2007 IEEE International Conference on Robotics and Automation, Rome, Italy.","DOI":"10.1109\/ROBOT.2007.363579"},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1177\/1541931213571287","article-title":"A Picture is Worth a Thousand Mental Models: Evaluating Human Understanding of Robot Teammates","volume":"57","author":"Ososky","year":"2013","journal-title":"Proc. Hum. Factors Ergon. Soc. Annu. Meet."},{"key":"ref_83","unstructured":"Shah, N., Verma, P., Angle, T., and Srivastava, S. (2022, January 9\u201313). JEDAI: A System for Skill-Aligned Explainable Robot Planning. Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS \u201922), Online."},{"key":"ref_84","first-page":"3532201","article-title":"Choreobot: A Reference Framework and Online Visual Dashboard for Supporting the Design of Intelligible Robotic Systems","volume":"6","author":"Bruyninckx","year":"2022","journal-title":"Proc. ACM Hum. Comput. Interact."},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Wlaszczyk, A., and Indurkhya, B. (2016, January 16\u201318). On the use of metaphors in designing educational interfaces. Proceedings of the 2016 7th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), Wroclaw, Poland.","DOI":"10.1109\/CogInfoCom.2016.7804573"},{"key":"ref_86","doi-asserted-by":"crossref","unstructured":"Zakershahrak, M., Marpally, S.R., Sharma, A., Gong, Z., and Zhang, Y. (June, January 30). Order Matters: Generating Progressive Explanations for Planning Tasks in Human-Robot Teaming. Proceedings of the 2021 International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9561762"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"3457183","DOI":"10.1145\/3457183","article-title":"Explaining in Time: Meeting Interactive Standards of Explanation for Robotic Systems","volume":"10","author":"Arnold","year":"2021","journal-title":"J. Hum. Robot Interact."},{"key":"ref_88","unstructured":"Brandao, M., Mansouri, M., Mohammed, A., Luff, P., and Coles, A. (2022, January 9\u201313). Explainability in Multi-Agent Path\/Motion Planning: User-Study-Driven Taxonomy and Requirements. Proceedings of the Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS \u201922), Online."},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"3457185","DOI":"10.1145\/3457185","article-title":"Building the Foundation of Robot Explanation Generation Using Behavior Trees","volume":"10","author":"Han","year":"2021","journal-title":"J. Hum. Robot Interact."},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Sheh, R. (2021, January 4\u20136). Explainable Artificial Intelligence Requirements for Safe, Intelligent Robots. Proceedings of the 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), Tokoname, Japan.","DOI":"10.1109\/ISR50024.2021.9419498"},{"key":"ref_91","doi-asserted-by":"crossref","unstructured":"Tabrez, A., Agrawal, S., and Hayes, B. (2019, January 11\u201314). Explanation-Based Reward Coaching to Improve Human Performance via Reinforcement Learning. Proceedings of the 14th ACM\/IEEE International Conference on Human-Robot Interaction (HRI \u201919), Daegu, Republic of Korea.","DOI":"10.1109\/HRI.2019.8673104"},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Zakershahrak, M., Gong, Z., Sadassivam, N., and Zhang, Y. (January, January 24). Online Explanation Generation for Planning Tasks in Human-Robot Teaming. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341792"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Fiazza, M.C., and Fiorini, P. (2021, January 4\u20136). Design for Interpretability: Meeting the Certification Challenge for Surgical Robots. Proceedings of the 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), Tokoname, Japan.","DOI":"10.1109\/ISR50024.2021.9419378"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Liu, H., Zhang, Y., Si, W., Xie, X., Zhu, Y., and Zhu, S.C. (2018, January 21\u201326). Interactive Robot Knowledge Patching Using Augmented Reality. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8462837"},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"Matsumaru, T. (2006, January 6\u20138). Mobile Robot with Preliminary-announcement and Display Function of Forthcoming Motion using Projection Equipment. Proceedings of the ROMAN 2006\u2014The 15th IEEE International Symposium on Robot and Human Interactive Communication, Hatfield, UK.","DOI":"10.1109\/ROMAN.2006.314368"},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Yonezawa, T., Yamazoe, H., and Abe, S. (2013, January 3\u20137). Physical contact using haptic and gestural expressions for ubiquitous partner robot. Proceedings of the 2013 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Tokyo, Japan.","DOI":"10.1109\/IROS.2013.6697179"},{"key":"ref_97","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1037\/0021-9010.85.2.273","article-title":"The influence of shared mental models on team process and performance","volume":"85","author":"Mathieu","year":"2000","journal-title":"J. Appl. Psychol."},{"key":"ref_98","first-page":"63","article-title":"Construction and Evaluation of a User Experience Questionnaire","volume":"5298","author":"Laugwitz","year":"2008","journal-title":"USAB"},{"key":"ref_99","unstructured":"Hoffman, R.R., Mueller, S.T., Klein, G., and Litman, J. (2018). Metrics for explainable AI: Challenges and prospects. arXiv."},{"key":"ref_100","unstructured":"Knapp, M.L., Hall, J.A., and Horgan, T.G. (2013). Nonverbal Communication in Human Interaction, Cengage Learning."},{"key":"ref_101","doi-asserted-by":"crossref","first-page":"1548","DOI":"10.1177\/1071181312561308","article-title":"Classification of robot form: Factors predicting perceived trustworthiness","volume":"Volume 56","author":"Schaefer","year":"2012","journal-title":"Proceedings of the Human Factors and Ergonomics Society Annual Meeting"},{"key":"ref_102","doi-asserted-by":"crossref","unstructured":"Natarajan, M., and Gombolay, M. (2020, January 7\u201310). Effects of anthropomorphism and accountability on trust in human robot interaction. Proceedings of the 2020 ACM\/IEEE International Conference on Human-Robot Interaction, Hokkaido, Japan.","DOI":"10.1145\/3319502.3374839"},{"key":"ref_103","doi-asserted-by":"crossref","first-page":"285","DOI":"10.3389\/fnhum.2012.00285","article-title":"A meta-analytic review of multisensory imagery identifies the neural correlates of modality-specific and modality-general imagery","volume":"6","author":"McNorgan","year":"2012","journal-title":"Front. Hum. Neurosci."},{"key":"ref_104","doi-asserted-by":"crossref","unstructured":"Maeda, T., and Kurahashi, T. (2019, January 11\u201312). Thermodule: Wearable and modular thermal feedback system based on a wireless platform. Proceedings of the 10th Augmented Human International Conference, Reims, France.","DOI":"10.1145\/3311823.3311826"},{"key":"ref_105","doi-asserted-by":"crossref","unstructured":"Suhonen, K., V\u00e4\u00e4n\u00e4nen-Vainio-Mattila, K., and M\u00e4kel\u00e4, K. (2012, January 12\u201314). User experiences and expectations of vibrotactile, thermal and squeeze feedback in interpersonal communication. Proceedings of the 26th BCS Conference on Human Computer Interaction, Birmingham, UK.","DOI":"10.14236\/ewic\/HCI2012.26"}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/7\/3\/25\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:42:39Z","timestamp":1760121759000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/7\/3\/25"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,25]]},"references-count":105,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["mti7030025"],"URL":"https:\/\/doi.org\/10.3390\/mti7030025","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,25]]}}}