{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:41:48Z","timestamp":1787017308522,"version":"3.56.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T00:00:00Z","timestamp":1635379200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Appl. Percept."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n                    For conversational agents\u2019 speech, either all possible sentences have to be prerecorded by voice actors or the required utterances can be synthesized. While synthesizing speech is more flexible and economic in production, it also potentially reduces the perceived naturalness of the agents among others due to mistakes at various linguistic levels. In our article, we are interested in the impact of adequate and inadequate prosody, here particularly in terms of accent placement, on the perceived naturalness and aliveness of the agents. We compare (1) inadequate prosody, as generated by off-the-shelf text-to-speech (TTS) engines with synthetic output; (2) the same inadequate prosody imitated by trained human speakers; and (3) adequate prosody produced by those speakers. The speech was presented either as audio-only or by embodied, anthropomorphic agents, to investigate the potential masking effect by a simultaneous visual representation of those virtual agents. To this end, we conducted an online study with 40 participants listening to four different dialogues each presented in the three\n                    <jats:italic>Speech<\/jats:italic>\n                    levels and the two\n                    <jats:italic>Embodiment<\/jats:italic>\n                    levels. Results confirmed that adequate prosody in human speech is perceived as more natural (and the agents are perceived as more alive) than inadequate prosody in both human (2) and synthetic speech (1). Thus, it is not sufficient to just use a human voice for an agents\u2019 speech to be perceived as natural\u2014it is decisive whether the\n                    <jats:italic>prosodic realisation<\/jats:italic>\n                    is adequate or not. Furthermore, and surprisingly, we found no masking effect by speaker embodiment, since neither a human voice with inadequate prosody nor a synthetic voice was judged as more natural, when a virtual agent was visible compared to the audio-only condition. On the contrary, the human voice was even judged as less \u201calive\u201d when accompanied by a virtual agent. In sum, our results emphasize, on the one hand, the importance of adequate prosody for perceived naturalness, especially in terms of accents being placed on important words in the phrase, while showing, on the other hand, that the embodiment of virtual agents plays a minor role in the naturalness ratings of voices.\n                  <\/jats:p>","DOI":"10.1145\/3486580","type":"journal-article","created":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T13:45:14Z","timestamp":1635428714000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Do Prosody and Embodiment Influence the Perceived Naturalness of Conversational Agents\u2019 Speech?"],"prefix":"10.1145","volume":"18","author":[{"given":"Jonathan","family":"Ehret","sequence":"first","affiliation":[{"name":"Visual Computing Institute, RWTH Aachen University, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"B\u00f6nsch","sequence":"additional","affiliation":[{"name":"Visual Computing Institute, RWTH Aachen University, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"Asp\u00f6ck","sequence":"additional","affiliation":[{"name":"Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christine T.","family":"R\u00f6hr","sequence":"additional","affiliation":[{"name":"IfL Phonetik, University of Cologne, K\u00f6ln, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Baumann","sequence":"additional","affiliation":[{"name":"IfL Phonetik, University of Cologne, K\u00f6ln, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martine","family":"Grice","sequence":"additional","affiliation":[{"name":"IfL Phonetik, University of Cologne, K\u00f6ln, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Janina","family":"Fels","sequence":"additional","affiliation":[{"name":"Institute for Hearing Technology and Acoustics, RWTH Aachen University, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Torsten W.","family":"Kuhlen","sequence":"additional","affiliation":[{"name":"Visual Computing Institute, RWTH Aachen University, Aachen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,28]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34584-5_9"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.5555\/2770015.2770053"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jml.2007.12.005"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1162\/105474601753272844"},{"key":"e_1_3_3_6_2","first-page":"59","volume-title":"IEEE International Conference on Automatic Face and Gesture Recognition","author":"Baltrusaitis Tadas","year":"2018","unstructured":"Tadas Baltrusaitis, Amir Zadeh, Yao Chong Lim, and Louis Philippe Morency. 2018. OpenFace 2.0: Facial behavior analysis toolkit. In IEEE International Conference on Automatic Face and Gesture Recognition. 59\u201366. https:\/\/doi.org\/10.1109\/FG.2018.00019"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v067.i01"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom.2013.39"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2017-325"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359325"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/332051.332075"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/11573548_84"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1177\/2051570719829432"},{"issue":"010117","key":"e_1_3_3_14_2","article-title":"Learning from avatars: Learning assistants practice physics pedagogy in a classroom simulator","volume":"12","author":"Chini Jacquelyn J.","year":"2016","unstructured":"Jacquelyn J. Chini, Carrie L. Straub, and Kevin H. Thomas. 2016. Learning from avatars: Learning assistants practice physics pedagogy in a classroom simulator. Physical Review Physics Education Research 12, 010117 (2016), 1\u201315. https:\/\/doi.org\/10.1103\/PhysRevPhysEducRes.12.010117","journal-title":"Physical Review Physics Education Research"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2806060"},{"key":"e_1_3_3_16_2","first-page":"220","volume-title":"Proceedings of the Cognitive Science Society","author":"Cohn Michelle","year":"2020","unstructured":"Michelle Cohn, Patrik Jonell, Taylor Kim, Jonas Beskow, and Georgia Zellou. 2020. Embodiment and gender interact in alignment to TTS voices. In Proceedings of the Cognitive Science Society. 220\u2013226."},{"key":"e_1_3_3_17_2","first-page":"67","volume-title":"Errors in Linguistic Performance: Slips of the Tongue, Ear, Pen and Hand","author":"Cutler Anne","year":"1980","unstructured":"Anne Cutler. 1980. Errors of stress and intonation. In Errors in Linguistic Performance: Slips of the Tongue, Ear, Pen and Hand, V. A. Fromkin (Ed.). New York, Academic Press, 67\u201380."},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.103605"},{"issue":"576","key":"e_1_3_3_19_2","article-title":"Is it the real deal? Perception of virtual characters versus humans: An affective cognitive neuroscience perspective","volume":"6","author":"Borst Aline W. de","year":"2015","unstructured":"Aline W. de Borst and Beatrice de Gelder. 2015. Is it the real deal? Perception of virtual characters versus humans: An affective cognitive neuroscience perspective. Frontiers in Psychology 6, 576 (2015), 1\u201312. https:\/\/doi.org\/10.3389\/fpsyg.2015.00576","journal-title":"Frontiers in Psychology"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2020.07.007"},{"key":"e_1_3_3_21_2","first-page":"3519","volume-title":"Proceedings of the 8th International Conference on Language Resources and Evaluation","author":"Georgila Kallirroi","year":"2012","unstructured":"Kallirroi Georgila, Alan W. Black, Kenji Sagae, and David Traum. 2012. Practical evaluation of human and synthesized speech for virtual human dialogue systems. In Proceedings of the 8th International Conference on Language Resources and Evaluation. 3519\u20133526."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-2958.2007.00295.x"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-47665-0_25"},{"key":"e_1_3_3_24_2","first-page":"47","volume-title":"Concept to Speech Generation Systems","author":"Hiyakumoto Laurie","year":"1997","unstructured":"Laurie Hiyakumoto, Scott Prevost, and Justine Cassell. 1997. Semantic and discourse information for text-to-speech intonation. In Concept to Speech Generation Systems. 47\u201356."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.10.008"},{"issue":"390","key":"e_1_3_3_26_2","article-title":"A review of empirical evidence on different uncanny valley hypotheses: Support for perceptual mismatch as one road to the valley of eeriness","volume":"6","author":"K\u00e4tsyri Jari","year":"2015","unstructured":"Jari K\u00e4tsyri, Klaus F\u00f6rger, Meeri M\u00e4k\u00e4r\u00e4inen, and Tapio Takala. 2015. A review of empirical evidence on different uncanny valley hypotheses: Support for perceptual mismatch as one road to the valley of eeriness. Frontiers in Psychology 6, 390 (2015), 1\u201316. https:\/\/doi.org\/10.3389\/fpsyg.2015.00390","journal-title":"Frontiers in Psychology"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-014-0569-0"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2020.593732"},{"key":"e_1_3_3_29_2","unstructured":"D. J. Leiner. 2021. SoSci Survey (Version 3.2.28) [Computer software]. https:\/\/www.soscisurvey.de."},{"issue":"26","key":"e_1_3_3_30_2","article-title":"Breaking bad behaviors: A new tool for learning classroom management using virtual reality","volume":"3","author":"Lugrin Jean-Luc","year":"2016","unstructured":"Jean-Luc Lugrin, Marc Erich Latoschik, Michael Habel, Daniel Roth, Christian Seufert, and Silke Grafe. 2016. Breaking bad behaviors: A new tool for learning classroom management using virtual reality. Frontiers in ICT 3, 26 (2016), 1\u201321. https:\/\/doi.org\/10.3389\/fict.2016.00026","journal-title":"Frontiers in ICT"},{"key":"e_1_3_3_31_2","first-page":"257","volume-title":"10th ISCA Speech Synthesis Workshop","author":"Malisz Zofia","year":"2019","unstructured":"Zofia Malisz, Harald Berthelsen, Jonas Beskow, and Joakim Gustafson. 2019. PROMIS: A statistical-parametric speech synthesis system with prominence control via a prominence network. In 10th ISCA Speech Synthesis Workshop. 257\u2013262. https:\/\/doi.org\/10.21437\/SSW.2019-46"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/2485895.2485900"},{"issue":"114","key":"e_1_3_3_33_2","article-title":"A systematic review of social presence: Definition, antecedents, and implications","volume":"5","author":"Oh Catherine S.","year":"2018","unstructured":"Catherine S. Oh, Jeremy N. Bailenson, and Gregory F. Welch. 2018. A systematic review of social presence: Definition, antecedents, and implications. Frontiers in Robotics and AI 5, 114 (2018), 1\u201335. https:\/\/doi.org\/10.3389\/frobt.2018.00114","journal-title":"Frontiers in Robotics and AI"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-019-01571-3"},{"key":"e_1_3_3_35_2","unstructured":"R Core Team. 2015. R: A Language and Environment for Statistical Computing. http:\/\/www.r-project.org\/."},{"key":"e_1_3_3_36_2","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1007\/978-3-319-47665-0_23","volume-title":"International Conference on Intelligent Virtual Agents (IVA\u201916)","author":"P\u00fctten Astrid M. Rosenthal-von der","year":"2016","unstructured":"Astrid M. Rosenthal-von der P\u00fctten, Carolin Stra\u00dfmann, and Nicole C. Kr\u00e4mer. 2016. Robots or agents-neither helps you more or less during second language acquisition. In International Conference on Intelligent Virtual Agents (IVA\u201916). 256\u2013268. https:\/\/doi.org\/10.1007\/978-3-319-47665-0_23"},{"key":"e_1_3_3_37_2","first-page":"3253","volume-title":"12th Annual Conference of the International Speech Communication Association","author":"Schr\u00f6der Marc","year":"2011","unstructured":"Marc Schr\u00f6der, Marcela Charfuelan, Sathish Pammi, and Ingmar Steiner. 2011. Open source voice creation toolkit for the MARY TTS platform. In 12th Annual Conference of the International Speech Communication Association. 3253\u20133256. http:\/\/mary.dfki.de\/."},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386867"},{"key":"e_1_3_3_39_2","first-page":"4779","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201918)","author":"Shen Jonathan","year":"2018","unstructured":"Jonathan Shen, Ruoming Pang, Ron J. Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, Rif A. Saurous, Yannis Agiomvrgiannakis, and Yonghui Wu. 2018. Natural TTS synthesis by conditioning wavenet on MEL spectrogram predictions. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201918). 4779\u20134783. https:\/\/doi.org\/10.1109\/ICASSP.2018.8461368"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3267851.3267918"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2021.1898851"},{"key":"e_1_3_3_42_2","unstructured":"Mark West Rebecca Kraut and Han Ei Chew. 2019. I\u2019d blush if I could: Closing gender divides in digital skills through education. https:\/\/unesdoc.unesco.org\/ark:\/48223\/pf0000367416."}],"container-title":["ACM Transactions on Applied Perception"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3486580","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3486580","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:48:38Z","timestamp":1750178918000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3486580"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,28]]},"references-count":41,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3486580"],"URL":"https:\/\/doi.org\/10.1145\/3486580","relation":{},"ISSN":["1544-3558","1544-3965"],"issn-type":[{"value":"1544-3558","type":"print"},{"value":"1544-3965","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,28]]},"assertion":[{"value":"2021-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}