{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:44:29Z","timestamp":1779295469535,"version":"3.51.4"},"reference-count":27,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T00:00:00Z","timestamp":1606435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MTI"],"abstract":"<jats:p>Intelligent virtual agents (VAs) already support us in a variety of everyday tasks such as setting up appointments, monitoring our fitness, and organizing messages. Adding a humanoid body representation to these mostly voice-based VAs has enormous potential to enrich the human\u2013agent communication process but, at the same time, raises expectations regarding the agent\u2019s social, spatial, and intelligent behavior. Embodied VAs may be perceived as less human-like if they, for example, do not return eye contact, or do not show a plausible collision behavior with the physical surroundings. In this article, we introduce a new model that extends human-to-human interaction to interaction with intelligent agents and covers different multi-modal and multi-sensory channels that are required to create believable embodied VAs. Theoretical considerations of the different aspects of human\u2013agent interaction are complemented by implementation guidelines to support the practical development of such agents. In this context, we particularly emphasize one aspect that is distinctive of embodied agents, i.e., interaction with the physical world. Since previous studies indicated negative effects of implausible physical behavior of VAs, we were interested in the initial responses of users when interacting with a VA with virtual\u2013physical capabilities for the first time. We conducted a pilot study to collect subjective feedback regarding two forms of virtual\u2013physical interactions. Both were designed and implemented in preparation of the user study, and represent two different approaches to virtual\u2013physical manipulations: (i) displacement of a robotic object, and (ii) writing on a physical sheet of paper with thermochromic ink. The qualitative results of the study indicate positive effects of agents with virtual\u2013physical capabilities in terms of their perceived realism as well as evoked emotional responses of the users. We conclude with an outlook on possible future developments of different aspects of human\u2013agent interaction in general and the physical simulation in particular.<\/jats:p>","DOI":"10.3390\/mti4040085","type":"journal-article","created":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T09:16:49Z","timestamp":1606468609000},"page":"85","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Intelligent Blended Agents: Reality\u2013Virtuality Interaction with Artificially Intelligent Embodied Virtual Humans"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8162-7694","authenticated-orcid":false,"given":"Susanne","family":"Schmidt","sequence":"first","affiliation":[{"name":"Human-Computer Interaction, Department of Informatics, Universit\u00e4t Hamburg, 22527 Hamburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8130-840X","authenticated-orcid":false,"given":"Oscar","family":"Ariza","sequence":"additional","affiliation":[{"name":"Human-Computer Interaction, Department of Informatics, Universit\u00e4t Hamburg, 22527 Hamburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9879-7414","authenticated-orcid":false,"given":"Frank","family":"Steinicke","sequence":"additional","affiliation":[{"name":"Human-Computer Interaction, Department of Informatics, Universit\u00e4t Hamburg, 22527 Hamburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1207\/s15427633scc0504_2","article-title":"Gesture in Spatial Cognition: Expressing, Communicating, and Thinking About Spatial Information","volume":"5","author":"Alibali","year":"2005","journal-title":"Spat. Cogn. Comput."},{"key":"ref_2","unstructured":"Schmidt, S., Bruder, G., and Steinicke, F. (2018, January 7\u20139). Effects of Embodiment on Generic and Content-Specific Intelligent Virtual Agents as Exhibition Guides. Proceedings of the International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE), Limassol, Cyprus."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cag.2019.06.002","article-title":"Effects of Virtual Agent and Object Representation on Experiencing Exhibited Artifacts","volume":"83","author":"Schmidt","year":"2019","journal-title":"Elsevier Comput. Graph."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Schmidt, S., Ariza, O., and Steinicke, F. (2019, January 19\u201320). Blended Agents: Manipulation of Physical Objects within Mixed Reality Environments and Beyond. Proceedings of the ACM Symposium on Spatial User Interaction (SUI), New Orleans, LA, USA.","DOI":"10.1145\/3357251.3357591"},{"key":"ref_5","unstructured":"Yee, N., Bailenson, J., and Rickertsen, K. (May, January 28). A Meta-Analysis of the Impact of the Inclusion and Realism of Human-Like Faces on User Experiences in Interfaces. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), San Jose, CA, USA."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kim, K., Boelling, L., Haesler, S., Bailenson, J., Bruder, G., and Welch, G. (2018, January 16\u201320). Does a Digital Assistant Need a Body? The Influence of Visual Embodiment and Social Behavior on the Perception of Intelligent Virtual Agents in AR. Proceedings of the IEEE Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2018.00039"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1006\/ijhc.1999.0325","article-title":"The Impact of Animated Interface Agents: A Review of Empirical Research","volume":"52","author":"Dehn","year":"2000","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1162\/105474605774785235","article-title":"The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self-Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments","volume":"14","author":"Bailenson","year":"2005","journal-title":"Presence Teleoper. Virtual Environ."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., and Sasse, M. (2003, January 5\u201310). The Impact of Avatar Realism and Eye Gaze Control on Perceived Quality of Communication in a Shared Immersive Virtual Environment. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI), Fort Lauderdale, FL, USA.","DOI":"10.1145\/642611.642703"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1162\/105474603322761289","article-title":"The Effect of the Agency and Anthropomorphism on Users\u2019 Sense of Telepresence, Copresence, and Social Presence in Virtual Environments","volume":"12","author":"Nowak","year":"2003","journal-title":"Presence Teleoper. Virtual Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e1771","DOI":"10.1002\/cav.1771","article-title":"The Effects of Virtual Human\u2019s Spatial and Behavioral Coherence with Physical Objects on Social Presence in AR","volume":"28","author":"Kim","year":"2017","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kim, K., Bruder, G., and Welch, G. (2017, January 8\u201310). Exploring the Effects of Observed Physicality Conflicts on Real\u2013Virtual Human Interaction in Augmented Reality. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST), Gothenburg, Sweden.","DOI":"10.1145\/3139131.3139151"},{"key":"ref_13","unstructured":"Lee, M., Norouzi, N., Bruder, G., Wisniewski, P., and Welch, G. (December, January 28). The Physical-Virtual Table: Exploring the Effects of a Virtual Human\u2019s Physical Influence on Social Interaction. Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST), Tokyo, Japan."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lee, M., Bruder, G., and Welch, G. (2017, January 18\u201322). Exploring the Effect of Vibrotactile Feedback Through the Floor on Social Presence in an Immersive Virtual Environment. Proceedings of the IEEE Conference on Virtual Reality (VR), Los Angeles, CA, USA.","DOI":"10.1109\/VR.2017.7892237"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Lee, M., Kim, K., Daher, S., Raij, A., Schubert, R., Bailenson, J., and Welch, G. (2016, January 19\u201323). The Wobbly Table: Increased Social Presence via Subtle Incidental Movement of a Real-Virtual Table. Proceedings of the IEEE Conference on Virtual Reality (VR), Greenville, SC, USA.","DOI":"10.1109\/VR.2016.7504683"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2947","DOI":"10.1109\/TVCG.2018.2868591","article-title":"Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008\u20132017)","volume":"24","author":"Kim","year":"2018","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Burdea, G., and Coiffet, P. (2003). Virtual Reality Technology, John Wiley & Sons.","DOI":"10.1162\/105474603322955950"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ekman, P., and Rosenberg, E. (1997). What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), Oxford University Press.","DOI":"10.1093\/oso\/9780195104462.001.0001"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kons, Z., Shechtman, S., Sorin, A., Rabinovitz, C., and Hoory, R. (2019). High Quality, Lightweight and Adaptable TTS Using LPCNet. arXiv.","DOI":"10.21437\/Interspeech.2019-1705"},{"key":"ref_20","unstructured":"Kim, K., Bruder, G., and Welch, G. (2018, January 7\u20139). Blowing in the Wind: Increasing Copresence with a Virtual Human via Airflow Influence in Augmented Reality. Proceedings of the International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE), Limassol, Cyprus."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_22","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kurata, G., Ramabhadran, B., Saon, G., and Sethy, A. (2017, January 16\u201320). Language Modeling with Highway LSTM. Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Okinawa, Japan.","DOI":"10.1109\/ASRU.2017.8268942"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1177\/0146167203029007002","article-title":"Interpersonal Distance in Immersive Virtual Environments","volume":"29","author":"Bailenson","year":"2003","journal-title":"Personal. Soc. Psychol. Bull."},{"key":"ref_25","unstructured":"Lombard, M., Ditton, T., and Weinstein, L. (2009, January 11\u201313). Measuring Presence: The Temple Presence Inventory. Proceedings of the International Workshop on Presence (PRESENCE), Los Angeles, CA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1162\/105474601300343612","article-title":"A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory","volume":"10","author":"Lessiter","year":"2001","journal-title":"Presence Teleoper. Virtual Environ."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1007\/s12369-008-0001-3","article-title":"Measurement Instruments for the Anthropomorphism, Animacy, Likeability, Perceived Intelligence, and Perceived Safety of Robots","volume":"1","author":"Bartneck","year":"2009","journal-title":"Int. J. Soc. Robot."}],"container-title":["Multimodal Technologies and Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/4\/85\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:38:11Z","timestamp":1760179091000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2414-4088\/4\/4\/85"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,27]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["mti4040085"],"URL":"https:\/\/doi.org\/10.3390\/mti4040085","relation":{},"ISSN":["2414-4088"],"issn-type":[{"value":"2414-4088","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,27]]}}}