{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T14:12:10Z","timestamp":1762956730412,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":21,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,18]],"date-time":"2021-10-18T00:00:00Z","timestamp":1634515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information and Communications Technology Planning and Evaluation (IITP)","award":["2017-0-00162"],"award-info":[{"award-number":["2017-0-00162"]}]},{"DOI":"10.13039\/501100001729","name":"Stiftelsen f\u00f6r\u00a0Strategisk Forskning","doi-asserted-by":"publisher","award":["RIT15-0107"],"award-info":[{"award-number":["RIT15-0107"]}],"id":[{"id":"10.13039\/501100001729","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004063","name":"Knut och Alice Wallenbergs Stiftelse","doi-asserted-by":"publisher","award":["WASP"],"award-info":[{"award-number":["WASP"]}],"id":[{"id":"10.13039\/501100004063","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Flemish Research Foundation","award":["1S95020N"],"award-info":[{"award-number":["1S95020N"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,18]]},"DOI":"10.1145\/3462244.3479957","type":"proceedings-article","created":{"date-parts":[[2021,10,15]],"date-time":"2021-10-15T15:01:58Z","timestamp":1634310118000},"page":"707-711","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["HEMVIP: Human Evaluation of Multiple Videos in Parallel"],"prefix":"10.1145","author":[{"given":"Patrik","family":"Jonell","sequence":"first","affiliation":[{"name":"KTH, Sweden"}]},{"given":"Youngwoo","family":"Yoon","sequence":"additional","affiliation":[{"name":"Electronics and Telecommunications Research Institute (ETRI) and Korea Advanced Institute of Science and Technology (KAIST), Korea, Republic of"}]},{"given":"Pieter","family":"Wolfert","sequence":"additional","affiliation":[{"name":"Ghent University - imec, Belgium"}]},{"given":"Taras","family":"Kucherenko","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Sweden"}]},{"given":"Gustav Eje","family":"Henter","sequence":"additional","affiliation":[{"name":"KTH Royal Institute of Technology, Sweden"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13946"},{"key":"e_1_3_2_1_3_1","volume-title":"Proceedings of the International Conference on Autonomous Agents and Multi-agent Systems. ACM, 781\u2013788","author":"Chiu Chung-Cheng","year":"2014","unstructured":"Chung-Cheng Chiu and Stacy Marsella . 2014 . Gesture generation with low-dimensional embeddings . In Proceedings of the International Conference on Autonomous Agents and Multi-agent Systems. ACM, 781\u2013788 . https:\/\/doi.org\/10.5555\/2615731.2615857 10.5555\/2615731.2615857 Chung-Cheng Chiu and Stacy Marsella. 2014. Gesture generation with low-dimensional embeddings. In Proceedings of the International Conference on Autonomous Agents and Multi-agent Systems. ACM, 781\u2013788. https:\/\/doi.org\/10.5555\/2615731.2615857"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0190393"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267851.3267898"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267851.3267866"},{"key":"e_1_3_2_1_7_1","unstructured":"ITU-R BS.1534-3. 2015. Method for the Subjective Assessment of Intermediate Quality Level of Audio Systems. Standard. ITU. https:\/\/www.itu.int\/rec\/R-REC-BS.1534-3-201510-I  ITU-R BS.1534-3. 2015. Method for the Subjective Assessment of Intermediate Quality Level of Audio Systems. Standard. ITU. https:\/\/www.itu.int\/rec\/R-REC-BS.1534-3-201510-I"},{"key":"e_1_3_2_1_8_1","unstructured":"ITU-R BT.1788. 2007. Methodology for the Subjective Assessment of Video Quality in Multimedia Applications. Standard. ITU. https:\/\/www.itu.int\/rec\/R-REC-BT.1788  ITU-R BT.1788. 2007. Methodology for the Subjective Assessment of Video Quality in Multimedia Applications. Standard. ITU. https:\/\/www.itu.int\/rec\/R-REC-BT.1788"},{"key":"e_1_3_2_1_9_1","unstructured":"ITU-T P.800. 1996. Methods for Subjective Determination of Transmission Quality. Standard. ITU. https:\/\/www.itu.int\/rec\/T-REC-P.800-199608-I  ITU-T P.800. 1996. Methods for Subjective Determination of Transmission Quality. Standard. ITU. https:\/\/www.itu.int\/rec\/T-REC-P.800-199608-I"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3383652.3423911"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the ACM International Conference on Intelligent Virtual Agents. ACM, Article 31","author":"Jonell Patrik","year":"2020","unstructured":"Patrik Jonell , Taras Kucherenko , Ilaria Torre , and Jonas Beskow . 2020 . Can we trust online crowdworkers? Comparing online and offline participants in a preference test of virtual agents .. In Proceedings of the ACM International Conference on Intelligent Virtual Agents. ACM, Article 31 , 8\u00a0pages. https:\/\/doi.org\/10.1145\/3383652.3423860 10.1145\/3383652.3423860 Patrik Jonell, Taras Kucherenko, Ilaria Torre, and Jonas Beskow. 2020. Can we trust online crowdworkers? Comparing online and offline participants in a preference test of virtual agents.. In Proceedings of the ACM International Conference on Intelligent Virtual Agents. ACM, Article 31, 8\u00a0pages. https:\/\/doi.org\/10.1145\/3383652.3423860"},{"key":"#cr-split#-e_1_3_2_1_12_1.1","doi-asserted-by":"crossref","unstructured":"Taras Kucherenko Dai Hasegawa Naoshi Kaneko Gustav\u00a0Eje Henter and Hedvig Kjellstr\u00f6m. 2021. Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation. International Journal of Human-Computer Interaction 37 14(2021) 1300-1316. https:\/\/doi.org\/10.1080\/10447318.2021.1883883 10.1080\/10447318.2021.1883883","DOI":"10.1080\/10447318.2021.1883883"},{"key":"#cr-split#-e_1_3_2_1_12_1.2","doi-asserted-by":"crossref","unstructured":"Taras Kucherenko Dai Hasegawa Naoshi Kaneko Gustav\u00a0Eje Henter and Hedvig Kjellstr\u00f6m. 2021. Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation. International Journal of Human-Computer Interaction 37 14(2021) 1300-1316. https:\/\/doi.org\/10.1080\/10447318.2021.1883883","DOI":"10.1080\/10447318.2021.1883883"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3382507.3418815"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450692"},{"key":"e_1_3_2_1_15_1","first-page":"1","article-title":"A technique for the measurement of attitudes","volume":"140","author":"Likert Rensis","year":"1932","unstructured":"Rensis Likert . 1932 . A technique for the measurement of attitudes . Archives of Psychology 140 (1932), 1 \u2013 55 . Rensis Likert. 1932. A technique for the measurement of attitudes. Archives of Psychology 140 (1932), 1\u201355.","journal-title":"Archives of Psychology"},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the Annual Conference of the International Speech Communication Association. ISCA, 1586\u20131590","author":"Ribeiro Manuel\u00a0Sam","year":"2015","unstructured":"Manuel\u00a0Sam Ribeiro , Junichi Yamagishi , and Robert A . \u00a0J. Clark. 2015. A perceptual investigation of wavelet-based decomposition of f0 for text-to-speech synthesis . In Proceedings of the Annual Conference of the International Speech Communication Association. ISCA, 1586\u20131590 . https:\/\/www.isca-speech.org\/archive\/interspeech_ 2015 \/i15_1586.html Manuel\u00a0Sam Ribeiro, Junichi Yamagishi, and Robert A.\u00a0J. Clark. 2015. A perceptual investigation of wavelet-based decomposition of f0 for text-to-speech synthesis. In Proceedings of the Annual Conference of the International Speech Communication Association. ISCA, 1586\u20131590. https:\/\/www.isca-speech.org\/archive\/interspeech_2015\/i15_1586.html"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5334\/jors.187"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3371382.3380739"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2015-689"},{"key":"e_1_3_2_1_20_1","unstructured":"Pieter Wolfert Nicole Robinson and Tony Belpaeme. 2021. A review of evaluation practices of gesture generation in embodied conversational agents. arxiv:2101.03769  Pieter Wolfert Nicole Robinson and Tony Belpaeme. 2021. A review of evaluation practices of gesture generation in embodied conversational agents. arxiv:2101.03769"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793720"}],"event":{"name":"ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Montr\u00e9al QC Canada","acronym":"ICMI '21"},"container-title":["Proceedings of the 2021 International Conference on Multimodal Interaction"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3462244.3479957","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3462244.3479957","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:55Z","timestamp":1750193335000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3462244.3479957"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,18]]},"references-count":21,"alternative-id":["10.1145\/3462244.3479957","10.1145\/3462244"],"URL":"https:\/\/doi.org\/10.1145\/3462244.3479957","relation":{},"subject":[],"published":{"date-parts":[[2021,10,18]]},"assertion":[{"value":"2021-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}