{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,20]],"date-time":"2025-09-20T20:32:12Z","timestamp":1758400332597,"version":"3.37.3"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T00:00:00Z","timestamp":1651708800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T00:00:00Z","timestamp":1651708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Multimodal User Interfaces"],"published-print":{"date-parts":[[2022,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper we present a Multimodal Echoborg interface to explore the effect of different embodiments of an Embodied Conversational Agent (ECA) in an interaction. We compared an interaction where the ECA was embodied as a virtual human (VH) with one where it was embodied as an Echoborg, i.e, a person whose actions are covertly controlled by a dialogue system. The Echoborg in our study not only shadowed the speech output of the dialogue system but also its non-verbal actions. The interactions were structured as a debate between three participants on an ethical dilemma.  First, we collected a corpus of debate sessions with three humans debaters. This we used as baseline to design and implement our ECAs. For the experiment, we designed two debate conditions. In one the participant interacted with two ECAs both embodied by virtual humans). In the other the participant interacted with one ECA embodied by a VH and the other by an Echoborg. Our results show that a human embodiment of the ECA overall scores better on perceived social attributes of the ECA. In many other respects the Echoborg scores as poorly as the VH except <jats:italic>copresence<\/jats:italic>.<\/jats:p>","DOI":"10.1007\/s12193-022-00389-z","type":"journal-article","created":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T12:11:43Z","timestamp":1651752703000},"page":"293-302","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["The multimodal EchoBorg: not as smart as it looks"],"prefix":"10.1007","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9407-821X","authenticated-orcid":false,"given":"Sara","family":"Falcone","sequence":"first","affiliation":[]},{"given":"Jan","family":"Kolkmeier","sequence":"additional","affiliation":[]},{"given":"Merijn","family":"Bruijnes","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4288-3334","authenticated-orcid":false,"given":"Dirk","family":"Heylen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,5,5]]},"reference":[{"issue":"4","key":"389_CR1","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/0950-7051(93)90017-N","volume":"6","author":"N Dahlb\u00e4ck","year":"1993","unstructured":"Dahlb\u00e4ck N, J\u00f6nsson A, Ahrenberg L (1993) Wizard of Oz studies\u2014why and how. Knowledge-Based Syst 6(4):258\u2013266. https:\/\/doi.org\/10.1016\/0950-7051(93)90017-N","journal-title":"Knowledge-Based Syst"},{"key":"389_CR2","doi-asserted-by":"publisher","first-page":"634","DOI":"10.3389\/fpsyg.2015.00634","volume":"6","author":"K Corti","year":"2015","unstructured":"Corti K, Gillespie A (2015) A truly human interface: interacting face-to-face with someone whose words are determined by a computer program. Front Psychol 6:634. https:\/\/doi.org\/10.3389\/fpsyg.2015.00634","journal-title":"Front Psychol"},{"key":"389_CR3","doi-asserted-by":"publisher","first-page":"1300","DOI":"10.3389\/fpsyg.2016.01300","volume":"7","author":"A Gillespie","year":"2016","unstructured":"Gillespie A, Corti K (2016) The body that speaks: recombining bodies and speech sources in unscripted face-to-face communication. Front Psychol 7:1300","journal-title":"Front Psychol"},{"key":"389_CR4","unstructured":"Milgram S, van Gasteren L (1974) Das Milgram-Experiment. ) Rowohlt"},{"key":"389_CR5","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1016\/j.chb.2015.12.039","volume":"58","author":"K Corti","year":"2016","unstructured":"Corti K, Gillespie A (2016) Co-constructing intersubjectivity with artificial conversational agents: people are more likely to initiate repairs of misunderstandings with agents represented as human. Comput Human Behav 58:431\u2013442","journal-title":"Comput Human Behav"},{"issue":"3","key":"389_CR6","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1037\/0022-3514.64.3.431","volume":"64","author":"N Ambady","year":"1993","unstructured":"Ambady N, Rosenthal R (1993) Half a minute: predicting teacher evaluations from thin slices of nonverbal behavior and physical attractiveness. J Personal Soc Psychol 64(3):431","journal-title":"J Personal Soc Psychol"},{"issue":"1","key":"389_CR7","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1111\/j.2044-8260.1978.tb00893.x","volume":"17","author":"A Campbell","year":"1978","unstructured":"Campbell A, Rushton JP (1978) Bodily communication and personality. Br J Soc Clin Psychol 17(1):31\u201336","journal-title":"Br J Soc Clin Psychol"},{"issue":"6","key":"389_CR8","doi-asserted-by":"publisher","first-page":"1178","DOI":"10.1037\/0022-3514.65.6.1178","volume":"65","author":"MJ Levesque","year":"1993","unstructured":"Levesque MJ, Kenny DA (1993) Accuracy of behavioral predictions at zero acquaintance: a social relations analysis. J Personal Soc Psychol 65(6):1178","journal-title":"J Personal Soc Psychol"},{"key":"389_CR9","doi-asserted-by":"crossref","unstructured":"Argyle M (2013) Bodily communication. Routledge","DOI":"10.4324\/9780203753835"},{"key":"389_CR10","doi-asserted-by":"crossref","unstructured":"Neff M, Toothman N, Bowmani R, Tree JEF, Walker MA (2011) Don\u2019t scratch! self-adaptors reflect emotional stability. In: International workshop on intelligent virtual agents, Springer, pp 398\u2013411","DOI":"10.1007\/978-3-642-23974-8_43"},{"issue":"4","key":"389_CR11","first-page":"49","volume":"36","author":"HJ Smith","year":"2017","unstructured":"Smith HJ, Neff M (2017) Understanding the impact of animated gesture performance on personality perceptions. ACM Trans Graph (TOG) 36(4):49","journal-title":"ACM Trans Graph (TOG)"},{"key":"389_CR12","doi-asserted-by":"crossref","unstructured":"Neff M, Wang Y, Abbott R, Walker M (2010) Evaluating the effect of gesture and language on personality perception in conversational agents. In: International conference on intelligent virtual Agents, Springer, pp 222\u2013235","DOI":"10.1007\/978-3-642-15892-6_24"},{"key":"389_CR13","unstructured":"Di Maro M, Falcone S, Cutugno F (2018) Prosodic analysis in human-machine interaction. Studi AISV 1"},{"key":"389_CR14","doi-asserted-by":"crossref","unstructured":"Normoyle A, Liu F, Kapadia M, Badler NI, J\u00f6rg S (2013) The effect of posture and dynamics on the perception of emotion. In: Proceedings of the ACM symposium on applied perception, ACM, pp 91\u201398","DOI":"10.1145\/2492494.2492500"},{"key":"389_CR15","doi-asserted-by":"crossref","unstructured":"Kolkmeier J, Vroon J, Heylen D (2016) Interacting with virtual agents in shared space: Single and joint effects of gaze and proxemics. In: International conference on intelligent virtual agents, Springer, pp 1\u201314","DOI":"10.1007\/978-3-319-47665-0_1"},{"key":"389_CR16","doi-asserted-by":"crossref","unstructured":"Cafaro A, Vilhj\u00e1lmsson HH, Bickmore T, Heylen D, J\u00f3hannsd\u00f3ttir KR, Valgardsson GS (2012) First impressions: users\u2019 judgments of virtual agents\u2019 personality and interpersonal attitude in first encounters. In: International conference on intelligent virtual agents, Springer, pp 67\u201380","DOI":"10.1007\/978-3-642-33197-8_7"},{"key":"389_CR17","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.ijhcs.2015.01.001","volume":"77","author":"J Li","year":"2015","unstructured":"Li J (2015) The benefit of being physically present: a survey of experimental works comparing copresent robots, telepresent robots and virtual agents. Int J Human-Comput Stud 77:23\u201337","journal-title":"Int J Human-Comput Stud"},{"key":"389_CR18","doi-asserted-by":"crossref","unstructured":"Bailenson JN, Yee N, Merget D, Schroeder R (2006) The effect of behavioral realism and form realism of real-time avatar faces on verbal disclosure, nonverbal disclosure, emotion recognition, and copresence in dyadic interaction. Presence: Teleop Virtual Environ 15(4):359\u2013372","DOI":"10.1162\/pres.15.4.359"},{"key":"389_CR19","doi-asserted-by":"crossref","unstructured":"Weibel D, Wissmath B, Habegger S, Steiner Y, Groner R (2008) Playing online games against computer-vs. human-controlled opponents: Effects on presence, flow, and enjoyment. Comput Human Behav 24(5):2274\u20132291","DOI":"10.1016\/j.chb.2007.11.002"},{"issue":"1\u20132","key":"389_CR20","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.ijhcs.2009.09.008","volume":"68","author":"S Lim","year":"2010","unstructured":"Lim S, Reeves B (2010) Computer agents versus avatars: responses to interactive game characters controlled by a computer or other player. Int J Human-Comput Stud 68(1\u20132):57\u201368","journal-title":"Int J Human-Comput Stud"},{"key":"389_CR21","doi-asserted-by":"crossref","unstructured":"K\u00e4tsyri J, Hari R, Ravaja N, Nummenmaa L (2013) The opponent matters: elevated fmri reward responses to winning against a human versus a computer opponent during interactive video game playing. Cerebral Cortex 23(12):2829\u20132839","DOI":"10.1093\/cercor\/bhs259"},{"key":"389_CR22","doi-asserted-by":"crossref","unstructured":"de Melo CM, Gratch J (2015) Beyond believability: Quantifying the differences between real and virtual humans. In: Brinkman W-P, Broekens J, Heylen D (eds) Intelligent virtual agents. Springer, Cham, pp 109\u2013118","DOI":"10.1007\/978-3-319-21996-7_11"},{"issue":"2","key":"389_CR23","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1207\/S15327965PLI1302_01","volume":"13","author":"J Blascovich","year":"2002","unstructured":"Blascovich J, Loomis J, Beall AC, Swinth KR, Hoyt CL, Bailenson JN (2002) Immersive virtual environment technology as a methodological tool for social psychology. Psychol Inq 13(2):103\u2013124","journal-title":"Psychol Inq"},{"key":"389_CR24","doi-asserted-by":"crossref","unstructured":"Blascovich J, McCall C (2013) Social influence in virtual environments","DOI":"10.1093\/oxfordhb\/9780195398809.013.0017"},{"key":"389_CR25","doi-asserted-by":"crossref","unstructured":"Bevacqua E, Stankovi\u0107 I, Maatallaoui A, N\u00e9d\u00e9lec A, De Loor P (2014) Effects of coupling in human-virtual agent body interaction. In: International conference on intelligent virtual agents, pp 54\u201363. Springer","DOI":"10.1007\/978-3-319-09767-1_7"},{"key":"389_CR26","doi-asserted-by":"crossref","unstructured":"Corti K, Gillespie A (2015) Offscreen and in the chair next to your: conversational agents speaking through actual human bodies. In: Intelligent virtual agents, pp 405\u2013417 . Springer","DOI":"10.1007\/978-3-319-21996-7_44"},{"issue":"1","key":"389_CR27","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1007\/s12369-008-0001-3","volume":"1","author":"C Bartneck","year":"2009","unstructured":"Bartneck C, Kuli\u0107 D, Croft E, Zoghbi S (2009) Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int J Soc Robot 1(1):71\u201381","journal-title":"Int J Soc Robot"},{"key":"389_CR28","unstructured":"Okita SY, Bailenso J, Schwartz DL (2007) The mere belief of social interaction improves learning. In: Proceedings of the annual meeting of the cognitive science society, Vol 29"},{"key":"389_CR29","unstructured":"IJsselsteijn W, De Kort Y, Poels K (2013) The game experience questionnaire"},{"issue":"1","key":"389_CR30","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/0005-7916(94)90063-9","volume":"25","author":"MM Bradley","year":"1994","unstructured":"Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Therapy Exper Psychiatry 25(1):49\u201359","journal-title":"J Behav Therapy Exper Psychiatry"},{"key":"389_CR31","doi-asserted-by":"crossref","unstructured":"Van Welbergen H, Reidsma D, Kopp S (2012) An incremental multimodal realizer for behavior co-articulation and coordination. In: International conference on intelligent virtual agents, pp 175\u2013188 . Springer","DOI":"10.1007\/978-3-642-33197-8_18"},{"key":"389_CR32","doi-asserted-by":"crossref","unstructured":"Kolkmeier J, Bruijnes M, Reidsma D, Heylen D (2017) An asap realizer-unity3d bridge for virtual and mixed reality applications. In: International conference on intelligent virtual agents, pp 227\u2013230. Springer","DOI":"10.1007\/978-3-319-67401-8_27"},{"issue":"3","key":"389_CR33","first-page":"1","volume":"17","author":"J Lawrence","year":"2017","unstructured":"Lawrence J, Snaith M, Konat B, Budzynska K, Reed C (2017) Debating technology for dialogical argument: sensemaking, engagement, and analytics. ACM Trans Internet Technol (TOIT) 17(3):1\u201323","journal-title":"ACM Trans Internet Technol (TOIT)"},{"key":"389_CR34","doi-asserted-by":"publisher","unstructured":"van Waterschoot J, Bruijnes M, Flokstra J, Reidsma D, Davison D, Theune M, Heylen D (2018) Flipper 2.0: a pragmatic dialogue engine for embodied conversational agents. In: Proceedings of the 18th international conference on intelligent virtual agents. IVA \u201918, pp. 43\u201350. ACM, Sydney, NSW, Australia. https:\/\/doi.org\/10.1145\/3267851.3267882","DOI":"10.1145\/3267851.3267882"},{"issue":"2","key":"389_CR35","doi-asserted-by":"publisher","first-page":"204","DOI":"10.5840\/monist197659224","volume":"59","author":"JJ Thomson","year":"1976","unstructured":"Thomson JJ (1976) Killing, letting die, and the trolley problem. The Monist 59(2):204\u2013217","journal-title":"The Monist"},{"key":"389_CR36","doi-asserted-by":"crossref","unstructured":"Nowak KL, Biocca F (2003) The effect of the agency and anthropomorphism on users\u2019 sense of telepresence, copresence, and social presence in virtual environments. Presence: Teleop Virtual Environ 12(5):481\u2013494","DOI":"10.1162\/105474603322761289"}],"container-title":["Journal on Multimodal User Interfaces"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12193-022-00389-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12193-022-00389-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12193-022-00389-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,22]],"date-time":"2022-08-22T13:00:48Z","timestamp":1661173248000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12193-022-00389-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,5]]},"references-count":36,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,9]]}},"alternative-id":["389"],"URL":"https:\/\/doi.org\/10.1007\/s12193-022-00389-z","relation":{},"ISSN":["1783-7677","1783-8738"],"issn-type":[{"type":"print","value":"1783-7677"},{"type":"electronic","value":"1783-8738"}],"subject":[],"published":{"date-parts":[[2022,5,5]]},"assertion":[{"value":"27 August 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 April 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 May 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}