{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T17:52:43Z","timestamp":1776102763131,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":52,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","award":["UIDB\/50021\/2020"],"award-info":[{"award-number":["UIDB\/50021\/2020"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,9,6]]},"DOI":"10.1145\/3514197.3549672","type":"proceedings-article","created":{"date-parts":[[2022,8,31]],"date-time":"2022-08-31T05:35:32Z","timestamp":1661924132000},"page":"1-8","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["A cautionary tale of side-by-side evaluations while developing emotional expression for intelligent virtual agents"],"prefix":"10.1145","author":[{"given":"Ricardo","family":"Rodrigues","sequence":"first","affiliation":[{"name":"INESC-ID and Instituto Superior T\u00e9cnico, Lisbon, Portugal"}]},{"given":"Ricardo","family":"Silva","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Lisbon, Portugal"}]},{"given":"Ricardo","family":"Pereira","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Lisbon, Portugal"}]},{"given":"Carlos","family":"Martinho","sequence":"additional","affiliation":[{"name":"INESC-ID and Instituto Superior T\u00e9cnico, Lisbon, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273613"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207777"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1155\/2009\/462315"},{"key":"e_1_3_2_1_4_1","volume-title":"How emotions are made: The secret life of the brain","author":"Barrett Lisa Feldman","unstructured":"Lisa Feldman Barrett . 2017. How emotions are made: The secret life of the brain . Pan MacMillan , Boston, USA. Lisa Feldman Barrett. 2017. How emotions are made: The secret life of the brain. Pan MacMillan, Boston, USA."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1037\/\/0022-3514.37.11.2049"},{"key":"e_1_3_2_1_6_1","volume-title":"Intelligent Virtual Agents","author":"Bevacqua Elisabetta","unstructured":"Elisabetta Bevacqua , Maurizio Mancini , and Catherine Pelachaud . 2008. A listening agent exhibiting variable behaviour . In Intelligent Virtual Agents . Springer , Springer, Tokyo Japan, 262--269. Elisabetta Bevacqua, Maurizio Mancini, and Catherine Pelachaud. 2008. A listening agent exhibiting variable behaviour. In Intelligent Virtual Agents. Springer, Springer, Tokyo Japan, 262--269."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASL.2006.885910"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273584"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273580"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10919-019-00294-2"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2009.5349585"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273657"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273607"},{"key":"e_1_3_2_1_14_1","volume-title":"An argument for basic emotions. Cognition & emotion 6, 3--4","author":"Ekman Paul","year":"1992","unstructured":"Paul Ekman . 1992. An argument for basic emotions. Cognition & emotion 6, 3--4 ( 1992 ), 169--200. Paul Ekman. 1992. An argument for basic emotions. Cognition & emotion 6, 3--4 (1992), 169--200."},{"key":"e_1_3_2_1_15_1","volume-title":"Emotions revealed: Recognizing faces and feelings to improve communication and emotional life","author":"Ekman Paul","unstructured":"Paul Ekman . 2007. Emotions revealed: Recognizing faces and feelings to improve communication and emotional life . Macmillan , New York , New York, USA. Paul Ekman. 2007. Emotions revealed: Recognizing faces and feelings to improve communication and emotional life. Macmillan, New York, New York, USA."},{"key":"e_1_3_2_1_16_1","volume-title":"Differences among unpleasant feelings. Motivation and Emotion","author":"Ekman Paul","unstructured":"Paul Ekman and WV Friesen . 1978. Facial action coding system: A technique for the measurement of facial movement . In Differences among unpleasant feelings. Motivation and Emotion . Plenum Publishing Corp , New York , New York, USA. Paul Ekman and WV Friesen. 1978. Facial action coding system: A technique for the measurement of facial movement. In Differences among unpleasant feelings. Motivation and Emotion. Plenum Publishing Corp, New York, New York, USA."},{"key":"e_1_3_2_1_17_1","unstructured":"Paul Ekman and Wallace V Friesen. 2003. Unmasking the face: A guide to recognizing emotions from facial clues. Ishk SAN JOSE CA USA.  Paul Ekman and Wallace V Friesen. 2003. Unmasking the face: A guide to recognizing emotions from facial clues. Ishk SAN JOSE CA USA."},{"key":"e_1_3_2_1_18_1","volume-title":"Emotion in the Human Face","author":"Ekman Paul","unstructured":"Paul Ekman , Wallace V Friesen , and Phoebe Ellsworth . 1972. Emotion in the Human Face ( 3 rd ed.). Pergamon , Berlin, Germany . Paul Ekman, Wallace V Friesen, and Phoebe Ellsworth. 1972. Emotion in the Human Face (3rd ed.). Pergamon, Berlin, Germany.","edition":"3"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/642611.642677"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273592"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273656"},{"key":"e_1_3_2_1_22_1","volume-title":"AIC 2004 Color and Paints, Interim Meeting of the In ternational Color Association, Proceedings. Association Internationale de la Couleur","author":"Kaya Naz","year":"2004","unstructured":"Naz Kaya and Helen H Epps . 2004 . Color-emotion associations: Past experience and personal preference . In AIC 2004 Color and Paints, Interim Meeting of the In ternational Color Association, Proceedings. Association Internationale de la Couleur , Porto Alegre, Brazil, 31--34. Naz Kaya and Helen H Epps. 2004. Color-emotion associations: Past experience and personal preference. In AIC 2004 Color and Paints, Interim Meeting of the In ternational Color Association, Proceedings. Association Internationale de la Couleur, Porto Alegre, Brazil, 31--34."},{"key":"e_1_3_2_1_23_1","volume-title":"Epps","author":"Kaya Naz","year":"2004","unstructured":"Naz Kaya and Helen H . Epps . 2004 . Relationship between Color and Emotion: A Study of College Students. College student journal 38 (2004), 396. Naz Kaya and Helen H. Epps. 2004. Relationship between Color and Emotion: A Study of College Students. College student journal 38 (2004), 396."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273578"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273577"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74997-4_8"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, ACM","author":"Lee Jina","year":"2010","unstructured":"Jina Lee , Zhiyang Wang , and Stacy Marsella . 2010 . Evaluating models of speaker head nods for virtual agents . In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, ACM , Richland, SC, 1257--1264. Jina Lee, Zhiyang Wang, and Stacy Marsella. 2010. Evaluating models of speaker head nods for virtual agents. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, ACM, Richland, SC, 1257--1264."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273574"},{"key":"e_1_3_2_1_29_1","volume-title":"Handbook of emotions","author":"Lewis Michael","unstructured":"Michael Lewis , Jeannette M Haviland-Jones , and Lisa Feldman Barrett . 2010. Handbook of emotions . Guilford Press , New York , New York, USA. Michael Lewis, Jeannette M Haviland-Jones, and Lisa Feldman Barrett. 2010. Handbook of emotions. Guilford Press, New York, New York, USA."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1177\/1557234X11410385"},{"key":"e_1_3_2_1_31_1","volume-title":"Believable Agents: Building Interactive Personalities. Technical Report","author":"Loyall Aaron B","year":"1997","unstructured":"Aaron B Loyall . 1997 . Believable Agents: Building Interactive Personalities. Technical Report . Carnegie Mellon University. Pittsburgh, PA. Department of Computer Science . Aaron B Loyall. 1997. Believable Agents: Building Interactive Personalities. Technical Report. Carnegie Mellon University. Pittsburgh, PA. Department of Computer Science."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273637"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/11550617_3"},{"key":"e_1_3_2_1_34_1","volume-title":"Intelligent Virtual Agents","author":"M\u00e4k\u00e4r\u00e4inen Meeri","unstructured":"Meeri M\u00e4k\u00e4r\u00e4inen and Tapio Takala . 2009. An Approach for Creating and Blending Synthetic Facial Expressions of Emotion .. In Intelligent Virtual Agents . Springer , Springer, Netherlands , 243--249. Meeri M\u00e4k\u00e4r\u00e4inen and Tapio Takala. 2009. An Approach for Creating and Blending Synthetic Facial Expressions of Emotion.. In Intelligent Virtual Agents. Springer, Springer, Netherlands, 243--249."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/1470357209343375"},{"key":"e_1_3_2_1_36_1","volume-title":"Understanding Comics: The Invisible Art","author":"McCloud Scott","year":"1993","unstructured":"Scott McCloud . 1993 . Understanding Comics: The Invisible Art . HarperPerennial , New York , New York, USA. Scott McCloud. 1993. Understanding Comics: The Invisible Art. HarperPerennial, New York, New York, USA."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0016562"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2009.2021722"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273598"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0015962"},{"key":"e_1_3_2_1_41_1","volume-title":"Emotions in Humans and Artifacts. MIT","author":"Ortony A","unstructured":"A Ortony . 2003. On making believable emotional agents believable . In Emotions in Humans and Artifacts. MIT , Cambridge , Massachusetts, USA , 189--211. https:\/\/books.google.es\/books?hl=es&lr=&id=jTgMIhy6YZMC&oi=fnd&pg=PA189&dq=On+making+believable+emotional+agents+believable&ots=l9fIhgy2AP&sig=Xd4a0oPqk679Kjk0s1gyhCeldfs A Ortony. 2003. On making believable emotional agents believable. In Emotions in Humans and Artifacts. MIT, Cambridge, Massachusetts, USA, 189--211. https:\/\/books.google.es\/books?hl=es&lr=&id=jTgMIhy6YZMC&oi=fnd&pg=PA189&dq=On+making+believable+emotional+agents+believable&ots=l9fIhgy2AP&sig=Xd4a0oPqk679Kjk0s1gyhCeldfs"},{"key":"e_1_3_2_1_42_1","volume-title":"Expressing Emotions through Animated Speech Balloons. Master's thesis. Instituto Superior T\u00e9cnico","author":"Pereira Ricardo","year":"1972","unstructured":"Ricardo Pereira . 2018. Expressing Emotions through Animated Speech Balloons. Master's thesis. Instituto Superior T\u00e9cnico , University of Lisbon. https :\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-t\/dissertacao\/ 1972 678479054259 Ricardo Pereira. 2018. Expressing Emotions through Animated Speech Balloons. Master's thesis. Instituto Superior T\u00e9cnico, University of Lisbon. https:\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-t\/dissertacao\/1972678479054259"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67401-8{_}46"},{"key":"e_1_3_2_1_44_1","unstructured":"W Scott Neal Reilly. 1996. Believable Social and Emotional Agents. 288 pages. http:\/\/oai.dtic.mil\/oai\/oai?verb=getRecord&metadataPrefix=html&identifier=ADA310766  W Scott Neal Reilly. 1996. Believable Social and Emotional Agents. 288 pages. http:\/\/oai.dtic.mil\/oai\/oai?verb=getRecord&metadataPrefix=html&identifier=ADA310766"},{"key":"e_1_3_2_1_45_1","volume-title":"Perception of Emotion Expression in Synthetic Characters. Master's thesis. Instituto Superior T\u00e9cnico","author":"Ribeiro Ta\u00edssa","year":"2035","unstructured":"Ta\u00edssa Ribeiro . 2021. Perception of Emotion Expression in Synthetic Characters. Master's thesis. Instituto Superior T\u00e9cnico , University of Lisbon. https :\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-a\/dissertacao\/1691 2035 02344371 Ta\u00edssa Ribeiro. 2021. Perception of Emotion Expression in Synthetic Characters. Master's thesis. Instituto Superior T\u00e9cnico, University of Lisbon. https:\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-a\/dissertacao\/1691203502344371"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2019.8925522"},{"key":"e_1_3_2_1_47_1","unstructured":"Vera Sacharin Katja Schlegel and Klaus R Scherer. 2012. Geneva Emotion Wheel Rating Study. https:\/\/archive-ouverte.unige.ch\/unige:97849  Vera Sacharin Katja Schlegel and Klaus R Scherer. 2012. Geneva Emotion Wheel Rating Study. https:\/\/archive-ouverte.unige.ch\/unige:97849"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.2190\/DUGG-P24E-52WK-6CDG"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273653"},{"key":"e_1_3_2_1_50_1","volume-title":"Supporting Affective Expression in Multi-party Interactions. Master's thesis. Instituto Superior T\u00e9cnico","author":"Silva Ricardo","unstructured":"Ricardo Silva . 2018. Supporting Affective Expression in Multi-party Interactions. Master's thesis. Instituto Superior T\u00e9cnico , University of Lisbon. https :\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-t\/dissertacao\/1409728525632118 Ricardo Silva. 2018. Supporting Affective Expression in Multi-party Interactions. Master's thesis. Instituto Superior T\u00e9cnico, University of Lisbon. https:\/\/fenix.tecnico.ulisboa.pt\/cursos\/meic-t\/dissertacao\/1409728525632118"},{"key":"e_1_3_2_1_51_1","volume-title":"Disney animation: The illusion of life","author":"Thomas Frank","unstructured":"Frank Thomas , Ollie Johnston , and Walton Rawls . 1981. Disney animation: The illusion of life . Vol. 4 . Disney Editions; Rev Sub edition, Burbank, California, USA. Frank Thomas, Ollie Johnston, and Walton Rawls. 1981. Disney animation: The illusion of life. Vol. 4. Disney Editions; Rev Sub edition, Burbank, California, USA."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2017.8273575"}],"event":{"name":"IVA '22: ACM International Conference on Intelligent Virtual Agents","location":"Faro Portugal","acronym":"IVA '22","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence"]},"container-title":["Proceedings of the 22nd ACM International Conference on Intelligent Virtual Agents"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3514197.3549672","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3514197.3549672","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:59:56Z","timestamp":1750186796000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3514197.3549672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":52,"alternative-id":["10.1145\/3514197.3549672","10.1145\/3514197"],"URL":"https:\/\/doi.org\/10.1145\/3514197.3549672","relation":{},"subject":[],"published":{"date-parts":[[2022,9,6]]},"assertion":[{"value":"2022-09-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}