{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T21:51:12Z","timestamp":1776117072779,"version":"3.50.1"},"reference-count":93,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2022,11,7]]},"abstract":"<jats:p>Conversational agents (CAs) provide opportunities for improving the interaction in evaluation surveys. To investigate if and how a user-centered conversational evaluation tool impacts users' response quality and their experience, we build EVA - a novel conversational course evaluation tool for educational scenarios. In a field experiment with 128 students, we compared EVA against a static web survey. Our results confirm prior findings from literature about the positive effect of conversational evaluation tools in the domain of education. Second, we then investigate the differences between a voice-based and text-based conversational human-computer interaction of EVA in the same experimental set-up. Against our prior expectation, the students of the voice-based interaction answered with higher information quality but with lower quantity of information compared to the text-based modality. Our findings indicate that using a conversational CA (voice and text-based) results in a higher response quality and user experience compared to a static web survey interface.<\/jats:p>","DOI":"10.1145\/3555619","type":"journal-article","created":{"date-parts":[[2022,11,11]],"date-time":"2022-11-11T22:58:54Z","timestamp":1668207534000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Designing Conversational Evaluation Tools"],"prefix":"10.1145","volume":"6","author":[{"given":"Thiemo","family":"Wambsganss","sequence":"first","affiliation":[{"name":"EPFL, Lausanne, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Naim","family":"Zierau","sequence":"additional","affiliation":[{"name":"University of St. Gallen, St. Gallen, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"S\u00f6llner","sequence":"additional","affiliation":[{"name":"University of Kassel, Kassel, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tanja","family":"K\u00e4ser","sequence":"additional","affiliation":[{"name":"EPFL, Lausanne, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenneth R.","family":"Koedinger","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan Marco","family":"Leimeister","sequence":"additional","affiliation":[{"name":"University of St. Gallen &amp; University of Kassel, St. Gallen, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11,11]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Time Flies When You're Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly 24, 4 (12","author":"Agarwal Ritu","year":"2000","unstructured":"Ritu Agarwal and Elena Karahanna . 2000. Time Flies When You're Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly 24, 4 (12 2000 ), 665. https:\/\/doi.org\/10.2307\/3250951 10.2307\/3250951 Ritu Agarwal and Elena Karahanna. 2000. Time Flies When You're Having Fun: Cognitive Absorption and Beliefs about Information Technology Usage. MIS Quarterly 24, 4 (12 2000), 665. https:\/\/doi.org\/10.2307\/3250951"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1080\/0022027032000152987"},{"key":"e_1_2_1_3_1","volume-title":"Human Memory: A Proposed System and its Control Processes. Psychology of Learning and Motivation - Advances in Research and Theory 2, C","author":"Atkinson R. C.","year":"1968","unstructured":"R. C. Atkinson and R. M. Shiffrin . 1968 . Human Memory: A Proposed System and its Control Processes. Psychology of Learning and Motivation - Advances in Research and Theory 2, C (1968), 89--195. https:\/\/doi.org\/10.1016\/S0079--7421(08)60422--3 10.1016\/S0079--7421(08)60422--3 R. C. Atkinson and R. M. Shiffrin. 1968. Human Memory: A Proposed System and its Control Processes. Psychology of Learning and Motivation - Advances in Research and Theory 2, C (1968), 89--195. https:\/\/doi.org\/10.1016\/S0079--7421(08)60422--3"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602938.2013.875984"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602930301683"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.116.3.457"},{"key":"e_1_2_1_8_1","first-page":"104","article-title":"Organizing knowledge syntheses: A taxonomy of literature reviews","volume":"1","author":"Cooper Harris M.","year":"1988","unstructured":"Harris M. Cooper . 1988 . Organizing knowledge syntheses: A taxonomy of literature reviews . Knowledge in Society 1 , 1 (1988), 104 -- 126 . https:\/\/doi.org\/10.1007\/BF03177550 10.1007\/BF03177550 Harris M. Cooper. 1988. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society 1, 1 (1988), 104--126. https:\/\/doi.org\/10.1007\/BF03177550","journal-title":"Knowledge in Society"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602938.2010.493970"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1287\/mnsc.32.5.554"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5334\/irsp.82"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2017388.2017395"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1080\/02699939208411068"},{"key":"e_1_2_1_14_1","unstructured":"eMarketer. 2017. Alexa Say What?! Voice-Enabled Speaker Usage to Grow Nearly 130% This Year. Technical Report. https:\/\/www.emarketer.com\/Articles\/Print.aspx?R=1015812  eMarketer. 2017. Alexa Say What?! Voice-Enabled Speaker Usage to Grow Nearly 130% This Year. Technical Report. https:\/\/www.emarketer.com\/Articles\/Print.aspx?R=1015812"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.114.4.864"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/14623943.2016.1206877"},{"key":"#cr-split#-e_1_2_1_17_1.1","unstructured":"Julia Fink. 2012. Anthropomorphism and human likeness in the design of robots and human-robot interaction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7621 LNAI (2012) 199--208. https:\/\/doi.org\/10.1007\/978--3--642--34103--8{_}20 10.1007\/978--3--642--34103--8"},{"key":"#cr-split#-e_1_2_1_17_1.2","doi-asserted-by":"crossref","unstructured":"Julia Fink. 2012. Anthropomorphism and human likeness in the design of robots and human-robot interaction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7621 LNAI (2012) 199--208. https:\/\/doi.org\/10.1007\/978--3--642--34103--8{_}20","DOI":"10.1007\/978-3-642-34103-8_20"},{"key":"e_1_2_1_18_1","volume-title":"Marks of readable style","author":"Flesch R","year":"1943","unstructured":"R Flesch . 1943. Marks of readable style ; a study in adult education. Teachers College Contributions to Education 897 ( 1943 ). R Flesch. 1943. Marks of readable style; a study in adult education. Teachers College Contributions to Education 897 (1943)."},{"key":"e_1_2_1_19_1","volume-title":"Public Opinion Quarterly","author":"Fricker Scott","unstructured":"Scott Fricker , Mirta Galesic , Roger Tourangeau , and Ting Yan . 2005. An experimental comparison of web and telephone surveys . In Public Opinion Quarterly , Vol. 69 . Oxford Academic , 370--392. https:\/\/doi.org\/10.1093\/poq\/nfi027 10.1093\/poq Scott Fricker, Mirta Galesic, Roger Tourangeau, and Ting Yan. 2005. An experimental comparison of web and telephone surveys. In Public Opinion Quarterly, Vol. 69. Oxford Academic, 370--392. https:\/\/doi.org\/10.1093\/poq\/nfi027"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.2307\/249720"},{"key":"e_1_2_1_21_1","volume-title":"Experteninterviews und qualitative Inhaltsanalyse : als Instrumente rekonstruierender Untersuchungen","author":"Grit Glaser","unstructured":"Jochen. Glaser and Grit . Laudel. 2010. Experteninterviews und qualitative Inhaltsanalyse : als Instrumente rekonstruierender Untersuchungen . VS Verlag fur Sozialwiss. http:\/\/www.springer.com\/de\/book\/9783531172385 Jochen. Glaser and Grit. Laudel. 2010. Experteninterviews und qualitative Inhaltsanalyse : als Instrumente rekonstruierender Untersuchungen. VS Verlag fur Sozialwiss. http:\/\/www.springer.com\/de\/book\/9783531172385"},{"key":"e_1_2_1_22_1","volume-title":"26th European Conference on Information Systems (ECIS) 2018","author":"Gnewuch Ulrich","year":"2018","unstructured":"Ulrich Gnewuch , Stefan Morana , Marc T P Adam , and Alexander Maedche . 2018 . Faster is not always better: Understanding the effect of dynamic response delays in human-chatbot interaction . In 26th European Conference on Information Systems (ECIS) 2018 . https:\/\/aisel.aisnet.org\/ecis2018_rp\/113 Ulrich Gnewuch, Stefan Morana, Marc T P Adam, and Alexander Maedche. 2018. Faster is not always better: Understanding the effect of dynamic response delays in human-chatbot interaction. In 26th European Conference on Information Systems (ECIS) 2018. https:\/\/aisel.aisnet.org\/ecis2018_rp\/113"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.89.4.743"},{"key":"#cr-split#-e_1_2_1_24_1.1","doi-asserted-by":"crossref","unstructured":"H.P. Grice. 1975. Logic and Conversation. In Speech Acts. BRILL 41--58. https:\/\/doi.org\/10.1163\/9789004368811{_}003 10.1163\/9789004368811","DOI":"10.1163\/9789004368811_003"},{"key":"#cr-split#-e_1_2_1_24_1.2","doi-asserted-by":"crossref","unstructured":"H.P. Grice. 1975. Logic and Conversation. In Speech Acts. BRILL 41--58. https:\/\/doi.org\/10.1163\/9789004368811{_}003","DOI":"10.1163\/9789004368811_003"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1093\/poq\/nfn045"},{"key":"e_1_2_1_26_1","volume-title":"Conversational robo advisors as surrogates of trust: onboarding experience, firm perception, and consumer financial decision making. Journal of the Academy of Marketing Science","author":"Hildebrand Christian","year":"2020","unstructured":"Christian Hildebrand and Anouk Bergner . 2020. Conversational robo advisors as surrogates of trust: onboarding experience, firm perception, and consumer financial decision making. Journal of the Academy of Marketing Science ( 2020 ). https:\/\/doi.org\/10.1007\/s11747-020-00753-z 10.1007\/s11747-020-00753-z Christian Hildebrand and Anouk Bergner. 2020. Conversational robo advisors as surrogates of trust: onboarding experience, firm perception, and consumer financial decision making. Journal of the Academy of Marketing Science (2020). https:\/\/doi.org\/10.1007\/s11747-020-00753-z"},{"key":"e_1_2_1_27_1","volume-title":"14th International Conference on Wirtschaftsinformatik","author":"Hobert Sebastian","year":"2019","unstructured":"Sebastian Hobert and Raphael Meyer Von Wolff . 2019 . Say Hello to Your New Automated Tutor -- A Structured Literature Review on Pedagogical Conversational Agents . 14th International Conference on Wirtschaftsinformatik , Siegen, Germany (2019). Sebastian Hobert and Raphael Meyer Von Wolff. 2019. Say Hello to Your New Automated Tutor -- A Structured Literature Review on Pedagogical Conversational Agents. 14th International Conference on Wirtschaftsinformatik, Siegen, Germany (2019)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1086\/346010"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2007"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2006.11.014"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1719970.1720024"},{"key":"e_1_2_1_32_1","first-page":"309","article-title":"A virtual change agent: Motivating pre-service teachers to integrate technology in their future classrooms","volume":"11","author":"Kim Chan Min","year":"2008","unstructured":"Chan Min Kim and Amy L Baylor . 2008 . A virtual change agent: Motivating pre-service teachers to integrate technology in their future classrooms . Educational Technology and Society 11 , 2 (2008), 309 -- 321 . Chan Min Kim and Amy L Baylor. 2008. A virtual change agent: Motivating pre-service teachers to integrate technology in their future classrooms. Educational Technology and Society 11, 2 (2008), 309--321.","journal-title":"Educational Technology and Society"},{"key":"e_1_2_1_33_1","volume-title":"Conference on Human Factors in Computing Systems - Proceedings","author":"Kim Soomin","year":"2019","unstructured":"Soomin Kim , Joonhwan Lee , and Gahgene Gweon . 2019 . Comparing data from chatbot and web surveys effects of platform and conversational style on survey response quality . Conference on Human Factors in Computing Systems - Proceedings (2019), 1--12. https:\/\/doi.org\/10.1145\/3290605.3300316 10.1145\/3290605.3300316 Soomin Kim, Joonhwan Lee, and Gahgene Gweon. 2019. Comparing data from chatbot and web surveys effects of platform and conversational style on survey response quality. Conference on Human Factors in Computing Systems - Proceedings (2019), 1--12. https:\/\/doi.org\/10.1145\/3290605.3300316"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196784"},{"key":"e_1_2_1_35_1","doi-asserted-by":"crossref","unstructured":"Marios Koufaris. 2002. Applying the Technology Acceptance Model and Flow Theory to Online Consumer Behavior on JSTOR. https:\/\/www.jstor.org\/stable\/23011056  Marios Koufaris. 2002. Applying the Technology Acceptance Model and Flow Theory to Online Consumer Behavior on JSTOR. https:\/\/www.jstor.org\/stable\/23011056","DOI":"10.1287\/isre.13.2.205.83"},{"key":"e_1_2_1_36_1","volume-title":"Persuasive Embodied Agents for Behavior Change (PEACH2017)","author":"Kowatsch Tobias","year":"2017","unstructured":"Tobias Kowatsch , Marcia Ni\u00dfen , Chen-hsuan Iris Shih , Dominik R\u00fcegger , Andreas Filler , Florian K\u00fcnzler , Filipe Barata , Severin Haug , Bj\u00f6rn Brogle , Katrin Heldt , Pauline Gindrat , Nathalie Farpour-lambert, and Dagmar Allemand . 2017 . Text-based Healthcare Chatbots Supporting Patient and Health Professional Teams : Preliminary Results of a Randomized Controlled Trial on Childhood Obesity . Persuasive Embodied Agents for Behavior Change (PEACH2017) Workshop 1, Iva 2017 (2017), 1--10. Tobias Kowatsch, Marcia Ni\u00dfen, Chen-hsuan Iris Shih, Dominik R\u00fcegger, Andreas Filler, Florian K\u00fcnzler, Filipe Barata, Severin Haug, Bj\u00f6rn Brogle, Katrin Heldt, Pauline Gindrat, Nathalie Farpour-lambert, and Dagmar Allemand. 2017. Text-based Healthcare Chatbots Supporting Patient and Health Professional Teams : Preliminary Results of a Randomized Controlled Trial on Childhood Obesity. Persuasive Embodied Agents for Behavior Change (PEACH2017) Workshop 1, Iva 2017 (2017), 1--10."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.4236\/ce.2018.911126"},{"key":"e_1_2_1_38_1","volume-title":"Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5, 3 (5","author":"Krosnick Jon A.","year":"1991","unstructured":"Jon A. Krosnick . 1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5, 3 (5 1991 ), 213--236. https:\/\/doi.org\/10.1002\/acp.2350050305 10.1002\/acp.2350050305 Jon A. Krosnick. 1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5, 3 (5 1991), 213--236. https:\/\/doi.org\/10.1002\/acp.2350050305"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.50.1.537"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.3102\/0034654315581420"},{"key":"e_1_2_1_41_1","volume-title":"Chatbot Acceptance in Healthcare: Explaining User Adoption of Conversational Agents for Disease Diagnosis. Twenty-Seventh European Conference on Information Systems (ECIS2019)","author":"Laumer Sven","year":"2019","unstructured":"Sven Laumer , Christian Maier , and Fabian Tobias Gubler . 2019 . Chatbot Acceptance in Healthcare: Explaining User Adoption of Conversational Agents for Disease Diagnosis. Twenty-Seventh European Conference on Information Systems (ECIS2019) , Stockholm-Uppsala, Sweden (2019), 0--18. https:\/\/aisel.aisnet.org\/ecis2019_rp\/88 Sven Laumer, Christian Maier, and Fabian Tobias Gubler. 2019. Chatbot Acceptance in Healthcare: Explaining User Adoption of Conversational Agents for Disease Diagnosis. Twenty-Seventh European Conference on Information Systems (ECIS2019), Stockholm-Uppsala, Sweden (2019), 0--18. https:\/\/aisel.aisnet.org\/ecis2019_rp\/88"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.IJHCS.2017.02.005"},{"key":"e_1_2_1_43_1","volume-title":"Per Baekgaard, Jakob E Bardram, Kevin Doherty, and Kevin 2021 Doherty. (n.d.). Can we talk? Design Implications for theQuestionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent","author":"Maharjan Raju","year":"2021","unstructured":"Raju Maharjan , Darius Adam Rohani , Per Baekgaard, Jakob E Bardram, Kevin Doherty, and Kevin 2021 Doherty. (n.d.). Can we talk? Design Implications for theQuestionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent ; Can we talk? Design Implications for theQuestionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent. CUI 2021 - 3rd Conference on Conversational User Interfaces 11 ([n. d.]). https:\/\/doi.org\/10.1145\/3469595 10.1145\/3469595 Raju Maharjan, Darius Adam Rohani, Per Baekgaard, Jakob E Bardram, Kevin Doherty, and Kevin 2021 Doherty. (n.d.). Can we talk? Design Implications for theQuestionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent; Can we talk? Design Implications for theQuestionnaire-Driven Self-Report of Health and Wellbeing via Conversational Agent. CUI 2021 - 3rd Conference on Conversational User Interfaces 11 ([n. d.]). https:\/\/doi.org\/10.1145\/3469595"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1086\/297862"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1086\/209566"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/0022-4537.00153"},{"key":"e_1_2_1_47_1","volume-title":"Gender of Prompt, and Gender of User. International Journal of Speech Technology 2003 6:2 6, 2 (4","author":"Nass Clifford","year":"2003","unstructured":"Clifford Nass , Erica Robles , Charles Heenan , Hilary Bienstock , and Marissa Treinen . 2003. Speech-Based Disclosure Systems: Effects of Modality , Gender of Prompt, and Gender of User. International Journal of Speech Technology 2003 6:2 6, 2 (4 2003 ), 113--121. https:\/\/doi.org\/10.1023\/A:1022378312670 10.1023\/A:1022378312670 Clifford Nass, Erica Robles, Charles Heenan, Hilary Bienstock, and Marissa Treinen. 2003. Speech-Based Disclosure Systems: Effects of Modality, Gender of Prompt, and Gender of User. International Journal of Speech Technology 2003 6:2 6, 2 (4 2003), 113--121. https:\/\/doi.org\/10.1023\/A:1022378312670"},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94","author":"Nass Clifford","year":"1916","unstructured":"Clifford Nass , Jonathan Steuer , and Ellen R. Tauber . 1994. Computers are social actors . In Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94 . ACM Press, New York, New York, USA, 72--78. https:\/\/doi.org\/10.1145\/ 1916 66.191703 10.1145\/191666.191703 Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94. ACM Press, New York, New York, USA, 72--78. https:\/\/doi.org\/10.1145\/191666.191703"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94","author":"Nass Clifford","year":"1916","unstructured":"Clifford Nass , Jonathan Steuer , and Ellen R. Tauber . 1994. Computers are social actors . In Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94 . ACM Press, New York, New York, USA, 72--78. https:\/\/doi.org\/10.1145\/ 1916 66.191703 10.1145\/191666.191703 Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers are social actors. In Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI '94. ACM Press, New York, New York, USA, 72--78. https:\/\/doi.org\/10.1145\/191666.191703"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000011"},{"key":"e_1_2_1_51_1","volume-title":"The virtual university's faculty: An overview of educational agents. Applied Artificial Intelligence 17, 1 (1","author":"Payr Sabine","year":"2003","unstructured":"Sabine Payr . 2003. The virtual university's faculty: An overview of educational agents. Applied Artificial Intelligence 17, 1 (1 2003 ), 1--19. https:\/\/doi.org\/10.1080\/713827053 10.1080\/713827053 Sabine Payr. 2003. The virtual university's faculty: An overview of educational agents. Applied Artificial Intelligence 17, 1 (1 2003), 1--19. https:\/\/doi.org\/10.1080\/713827053"},{"key":"e_1_2_1_52_1","volume-title":"Raising response rates: What works? New Directions for Institutional Research","author":"Porter Stephen R.","year":"2004","unstructured":"Stephen R. Porter . 2004. Raising response rates: What works? New Directions for Institutional Research 2004 , 121 (1 2004), 5--21. https:\/\/doi.org\/10.1002\/IR.97 10.1002\/IR.97 Stephen R. Porter. 2004. Raising response rates: What works? New Directions for Institutional Research 2004, 121 (1 2004), 5--21. https:\/\/doi.org\/10.1002\/IR.97"},{"key":"e_1_2_1_53_1","unstructured":"Eric Ries. 2011. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses.  Eric Ries. 2011. The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1177\/147078530404600301"},{"key":"e_1_2_1_55_1","volume-title":"Zhengneng Qiu, Yeshuang Zhu, Elizabeth L. Murnane, Emma Brunskill, and James A. Landay.","author":"Ruan Sherry","year":"2019","unstructured":"Sherry Ruan , Liwei Jiang , Justin Xu , Bryce Joe-Kun Tham , Zhengneng Qiu, Yeshuang Zhu, Elizabeth L. Murnane, Emma Brunskill, and James A. Landay. 2019 . QuizBot: A Dialogue-based Adaptive Learning System System for Factual Knowledge. Chi ( 2019), 1--13. https:\/\/doi.org\/10.1145\/3290605.3300587 10.1145\/3290605.3300587 Sherry Ruan, Liwei Jiang, Justin Xu, Bryce Joe-Kun Tham, Zhengneng Qiu, Yeshuang Zhu, Elizabeth L. Murnane, Emma Brunskill, and James A. Landay. 2019. QuizBot: A Dialogue-based Adaptive Learning System System for Factual Knowledge. Chi (2019), 1--13. https:\/\/doi.org\/10.1145\/3290605.3300587"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1108\/07378831011096196"},{"key":"e_1_2_1_57_1","volume-title":"Twenty-Ninth European Conference on Information Systems (ECIS","author":"Schmitt Anuschka","year":"2021","unstructured":"Anuschka Schmitt , Naim Zierau , Andreas Janson , and Jan Marco Leimeister . 2021 . Voice as a Contemporary Frontier of Interaction Design . In Twenty-Ninth European Conference on Information Systems (ECIS 2021). Virtual, 1--17. Anuschka Schmitt, Naim Zierau, Andreas Janson, and Jan Marco Leimeister. 2021. Voice as a Contemporary Frontier of Interaction Design. In Twenty-Ninth European Conference on Information Systems (ECIS 2021). Virtual, 1--17."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.24251\/HICSS.2018.061"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2018.08.011"},{"key":"e_1_2_1_60_1","volume-title":"35th International Conference on Information Systems \"Building a Better World Through Information Systems\", ICIS 2014","author":"Grimes G Mark","year":"2014","unstructured":"RyanMSchuetzler, G Mark Grimes , Justin Scott Giboney , and Joseph Buckman . 2014 . Facilitating natural conversational agent interactions: Lessons from a deception experiment . 35th International Conference on Information Systems \"Building a Better World Through Information Systems\", ICIS 2014 (2014). RyanMSchuetzler, G Mark Grimes, Justin Scott Giboney, and Joseph Buckman. 2014. Facilitating natural conversational agent interactions: Lessons from a deception experiment. 35th International Conference on Information Systems \"Building a Better World Through Information Systems\", ICIS 2014 (2014)."},{"key":"#cr-split#-e_1_2_1_61_1.1","doi-asserted-by":"crossref","unstructured":"Katie Seaborn Norihisa P Miyake Peter Pennefather and Mihoko Otake-Matsuura. 2021. Voice in human-agent interaction: A survey. https:\/\/doi.org\/10.1145\/3386867 10.1145\/3386867","DOI":"10.1145\/3386867"},{"key":"#cr-split#-e_1_2_1_61_1.2","doi-asserted-by":"crossref","unstructured":"Katie Seaborn Norihisa P Miyake Peter Pennefather and Mihoko Otake-Matsuura. 2021. Voice in human-agent interaction: A survey. https:\/\/doi.org\/10.1145\/3386867","DOI":"10.1145\/3386867"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1075\/ijcl.10.4.06sha"},{"key":"e_1_2_1_64_1","volume-title":"Voicepedia: Towards speech-based access to unstructured information","author":"Sherwani J.","year":"2007","unstructured":"J. Sherwani , Dong Yu , Tim Paek , Mary Czerwinski , Y. C. Ju , and Alex Acero . 2007 . Voicepedia: Towards speech-based access to unstructured information . In International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association , Interspeech 2007, Vol. 3 . 2245--2248. https:\/\/doi.org\/10.21437\/ interspeech.2007--60 J. Sherwani, Dong Yu, Tim Paek, Mary Czerwinski, Y. C. Ju, and Alex Acero. 2007. Voicepedia: Towards speech-based access to unstructured information. In International Speech Communication Association - 8th Annual Conference of the International Speech Communication Association, Interspeech 2007, Vol. 3. 2245--2248. https:\/\/doi.org\/10.21437\/ interspeech.2007--60"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1108\/QAE-12-2013-0049"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/HSI.2017.8005002"},{"key":"e_1_2_1_67_1","first-page":"598","article-title":"On the Validity of Student Evaluation of Teaching","volume":"83","author":"Spooren Pieter","year":"2013","unstructured":"Pieter Spooren , Bert Brockx , and Dimitri Mortelmans . 2013 . On the Validity of Student Evaluation of Teaching : The State of the Art. Vol. 83. 598 -- 642 pages. https:\/\/doi.org\/10.3102\/0034654313496870 10.3102\/0034654313496870 Pieter Spooren, Bert Brockx, and Dimitri Mortelmans. 2013. On the Validity of Student Evaluation of Teaching: The State of the Art. Vol. 83. 598--642 pages. https:\/\/doi.org\/10.3102\/0034654313496870","journal-title":"The State of the Art."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602938.2018.1466266"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.166.3903.343"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300833"},{"key":"e_1_2_1_71_1","volume-title":"ON THE NATURE AND DEVELOPMENT OF SOCIAL PRESENCE IN ONLINE COURSE DISCUSSIONS. Online Learning 9, 3","author":"Swan Karen","year":"2005","unstructured":"Karen Swan and Li Fang Shih . 2005. ON THE NATURE AND DEVELOPMENT OF SOCIAL PRESENCE IN ONLINE COURSE DISCUSSIONS. Online Learning 9, 3 ( 2005 ). https:\/\/doi.org\/10.24059\/olj.v9i3.1788 10.24059\/olj.v9i3.1788 Karen Swan and Li Fang Shih. 2005. ON THE NATURE AND DEVELOPMENT OF SOCIAL PRESENCE IN ONLINE COURSE DISCUSSIONS. Online Learning 9, 3 (2005). https:\/\/doi.org\/10.24059\/olj.v9i3.1788"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411763.3451623"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1080\/07294360802259067"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1080\/10494820600924750"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.2307\/25148660"},{"key":"e_1_2_1_76_1","volume-title":"Jang Bahadur Singh, and Yogesh K. Dwivedi.","author":"Vimalkumar M.","year":"2021","unstructured":"M. Vimalkumar , Sujeet Kumar Sharma , Jang Bahadur Singh, and Yogesh K. Dwivedi. 2021 . ?Okay google, what about my privacy?': User's privacy perceptions and acceptance of voice based digital assistants. Computers in Human Behavior 120 (7 2021), 106763. https:\/\/doi.org\/10.1016\/J.CHB.2021.106763 10.1016\/J.CHB.2021.106763 M. Vimalkumar, Sujeet Kumar Sharma, Jang Bahadur Singh, and Yogesh K. Dwivedi. 2021. ?Okay google, what about my privacy?': User's privacy perceptions and acceptance of voice based digital assistants. Computers in Human Behavior 120 (7 2021), 106763. https:\/\/doi.org\/10.1016\/J.CHB.2021.106763"},{"key":"e_1_2_1_77_1","volume-title":"Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research. Communications of the Association for Information Systems 37, 1 (8","author":"vom Brocke Jan","year":"2015","unstructured":"Jan vom Brocke , Alexander Simons , Kai Riemer , Bjoern Niehaves , Ralf Plattfaut , and Anne Cleven . 2015. Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research. Communications of the Association for Information Systems 37, 1 (8 2015 ), 205--224. https:\/\/doi.org\/10.17705\/1cais.03709 10.17705\/1cais.03709 Jan vom Brocke, Alexander Simons, Kai Riemer, Bjoern Niehaves, Ralf Plattfaut, and Anne Cleven. 2015. Standing on the shoulders of giants: Challenges and recommendations of literature search in information systems research. Communications of the Association for Information Systems 37, 1 (8 2015), 205--224. https:\/\/doi.org\/10.17705\/1cais.03709"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2006.05.002"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.599"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.74"},{"key":"e_1_2_1_82_1","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Wambsganss Thiemo","year":"1865","unstructured":"Thiemo Wambsganss , Christina Niklaus , Matthias S\u00f6llner , Siegfried Handschuh , and Jan Marco Leimeister . 2021. Supporting Cognitive and Emotional Empathic Writing of Students . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics , Online , 4063--4077. https:\/\/doi.org\/10. 1865 3\/v1\/2021.acl-long.314 10.18653\/v1 Thiemo Wambsganss, Christina Niklaus, Matthias S\u00f6llner, Siegfried Handschuh, and Jan Marco Leimeister. 2021. Supporting Cognitive and Emotional Empathic Writing of Students. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 4063--4077. https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.314"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3382805"},{"key":"e_1_2_1_84_1","volume-title":"Forty-Second International Conference on Information Systems","author":"R\u00fcttimann Dominic","year":"2021","unstructured":"FlorianWeber, ThiemoWambsganss, Dominic R\u00fcttimann , and Matthias S\u00f6llner . 2021 . Pedagogical Agents for Interactive Lernaing : A Taxonomy of Conversational Agents in Education . In Forty-Second International Conference on Information Systems . Austin, Texas, 1--17. FlorianWeber, ThiemoWambsganss, Dominic R\u00fcttimann, and Matthias S\u00f6llner. 2021. Pedagogical Agents for Interactive Lernaing : A Taxonomy of Conversational Agents in Education. In Forty-Second International Conference on Information Systems. Austin, Texas, 1--17."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376781"},{"key":"e_1_2_1_86_1","volume-title":"Meeting, Annual Chicago, A O M","author":"Winkler R.","year":"2018","unstructured":"R. Winkler and M. S\u00f6llner . 2018. Unleashing the Potential of Chatbots in Education : A State-Of-The-Art Analysis . In : Academy of Management . Meeting, Annual Chicago, A O M ( 2018 ). https:\/\/www.alexandria.unisg.ch\/254848\/1\/JML_699.pdf R. Winkler and M. S\u00f6llner. 2018. Unleashing the Potential of Chatbots in Education : A State-Of-The-Art Analysis . In : Academy of Management. Meeting, Annual Chicago, A O M (2018). https:\/\/www.alexandria.unisg.ch\/254848\/1\/JML_699.pdf"},{"key":"e_1_2_1_87_1","volume-title":"Enhancing problem-solving skills with smart personal assistant technology. Computers and Education 165","author":"Winkler Rainer","year":"2021","unstructured":"Rainer Winkler , Matthias S\u00f6llner , and Jan Marco Leimeister . 2021. Enhancing problem-solving skills with smart personal assistant technology. Computers and Education 165 ( 2021 ). https:\/\/doi.org\/10.1016\/j.compedu.2021.104148 10.1016\/j.compedu.2021.104148 Rainer Winkler, Matthias S\u00f6llner, and Jan Marco Leimeister. 2021. Enhancing problem-solving skills with smart personal assistant technology. Computers and Education 165 (2021). https:\/\/doi.org\/10.1016\/j.compedu.2021.104148"},{"key":"e_1_2_1_88_1","volume-title":"Conference on Human Factors in Computing Systems - Proceedings (5","author":"Wozniak Pawel W.","year":"2021","unstructured":"Pawel W. Wozniak , Jakob Karolus , Florian Lang , Caroline Eckerth , Johannes Schoning , Yvonne Rogers , and Jasmin Niess . 2021 . Creepy technology: What is it and how do you measure it? Conference on Human Factors in Computing Systems - Proceedings (5 2021). https:\/\/doi.org\/10.1145\/3411764.3445299 10.1145\/3411764.3445299 Pawel W. Wozniak, Jakob Karolus, Florian Lang, Caroline Eckerth, Johannes Schoning, Yvonne Rogers, and Jasmin Niess. 2021. Creepy technology: What is it and how do you measure it? Conference on Human Factors in Computing Systems - Proceedings (5 2021). https:\/\/doi.org\/10.1145\/3411764.3445299"},{"key":"e_1_2_1_89_1","volume-title":"Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. 27, 3","author":"Xiao Ziang","year":"2019","unstructured":"Ziang Xiao , Michelle X Zhou , Q Vera Liao , Gloria Mark , Changyan Chi , Wenxi Chen , and Huahai Yang . 2019. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. 27, 3 ( 2019 ). https:\/\/doi.org\/10.1145\/3381804 10.1145\/3381804 Ziang Xiao, Michelle X Zhou, Q Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. 2019. Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-ended Questions. 27, 3 (2019). https:\/\/doi.org\/10.1145\/3381804"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025496"},{"key":"#cr-split#-e_1_2_1_91_1.1","unstructured":"Naim Zierau Christian Hildebrand Anouk Bergner Francesc Busquet Anuschka Schmitt and Jan Marco Leimeister. 2022. Voice bots on the frontline: Voice-based interfaces enhance flow-like consumer experiences & boost service outcomes. (2022). https:\/\/doi.org\/10.1007\/s11747-022-00868--5 10.1007\/s11747-022-00868--5"},{"key":"#cr-split#-e_1_2_1_91_1.2","doi-asserted-by":"crossref","unstructured":"Naim Zierau Christian Hildebrand Anouk Bergner Francesc Busquet Anuschka Schmitt and Jan Marco Leimeister. 2022. Voice bots on the frontline: Voice-based interfaces enhance flow-like consumer experiences & boost service outcomes. (2022). https:\/\/doi.org\/10.1007\/s11747-022-00868--5","DOI":"10.1007\/s11747-022-00868-5"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555619","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3555619","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:03Z","timestamp":1750182543000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3555619"}},"subtitle":["A Comparison of Text and Voice Modalities to Improve Response Quality in Course Evaluations"],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":93,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2022,11,7]]}},"alternative-id":["10.1145\/3555619"],"URL":"https:\/\/doi.org\/10.1145\/3555619","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}