{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T23:15:04Z","timestamp":1784675704273,"version":"3.55.0"},"reference-count":112,"publisher":"Association for Computing Machinery (ACM)","issue":"5","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,9,30]]},"abstract":"<jats:p>Despite their popularity in providing digital mental health support, mobile conversational agents primarily rely on verbal input, which limits their ability to respond to emotional expressions. We therefore envision using the sensory equipment of today\u2019s devices to increase the nonverbal, empathic capabilities of chatbots. We initially validated that multi-modal LLMs (MLLM) can infer emotional expressions from facial expressions with high accuracy. In a user study (N=200), we then investigated the effects of such multi-modal input on response generation and perceived system empathy in emotional support scenarios. We found significant effects on cognitive and affective dimensions of linguistic expression in system responses, yet no significant increases in perceived empathy. Our research demonstrates the general potential of using nonverbal context to adapt LLM response behavior, providing input for future research on augmented interaction in empathic MLLM-based systems.<\/jats:p>","DOI":"10.1145\/3743724","type":"journal-article","created":{"date-parts":[[2025,9,9]],"date-time":"2025-09-09T14:28:48Z","timestamp":1757428128000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Using Nonverbal Cues in Empathic Multi-Modal LLM-Driven Chatbots for Mental Health Support MHCI039"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8609-8716","authenticated-orcid":false,"given":"Matthias","family":"Schmidmaier","sequence":"first","affiliation":[{"name":"LMU Munich","place":["Munich, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6983-9574","authenticated-orcid":false,"given":"Jonathan","family":"Rupp","sequence":"additional","affiliation":[{"name":"University of Innsbruck","place":["Innsbruck, Austria"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0573-5050","authenticated-orcid":false,"given":"Cedrik","family":"Harrich","sequence":"additional","affiliation":[{"name":"LMU Munich","place":["Munich, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5462-8782","authenticated-orcid":false,"given":"Sven","family":"Mayer","sequence":"additional","affiliation":[{"name":"LMU Munich","place":["Munich, Germany"]},{"name":"TU Dortmund University","place":["Munich, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,9]]},"reference":[{"key":"e_1_3_4_2_2","doi-asserted-by":"publisher","unstructured":"Mahyar Abbasian Iman Azimi Mohammad Feli Amir\u00a0M Rahmani and Ramesh Jain. 2024. Empathy Through Multimodality in Conversational Interfaces. 10.48550\/arXiv.2405.04777","DOI":"10.48550\/arXiv.2405.04777"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","unstructured":"Alaa\u00a0A Abd-alrazaq Mohannad Alajlani Ali\u00a0Abdallah Alalwan Bridgette\u00a0M Bewick Peter Gardner and Mowafa Househ. 2019. An overview of the features of chatbots in mental health: A scoping review. Int. J. Med. Inform. 132 (Dec. 2019) 103978. 10.1016\/j.ijmedinf.2019.103978","DOI":"10.1016\/j.ijmedinf.2019.103978"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","unstructured":"Jean-Baptiste Alayrac Jeff Donahue Pauline Luc Antoine Miech Iain Barr Yana Hasson Karel Lenc A Mensch Katie Millican Malcolm Reynolds Roman Ring Eliza Rutherford Serkan Cabi Tengda Han Zhitao Gong Sina Samangooei Marianne Monteiro Jacob Menick Sebastian Borgeaud Andy Brock Aida Nematzadeh Sahand Sharifzadeh Mikolaj Binkowski Ricardo Barreira O Vinyals Andrew Zisserman and K Simonyan. 2022. Flamingo: A visual language model for few-shot learning. 10.48550\/arXiv.2204.14198","DOI":"10.48550\/arXiv.2204.14198"},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","unstructured":"Nouar AlDahoul Myles Joshua\u00a0Toledo Tan Harishwar\u00a0Reddy Kasireddy and Yasir Zaki. 2024. Exploring vision language models for facial attribute recognition: Emotion race gender and age. 10.48550\/arXiv.2410.24148","DOI":"10.48550\/arXiv.2410.24148"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203753835"},{"key":"e_1_3_4_7_2","doi-asserted-by":"publisher","unstructured":"John\u00a0W Ayers Adam Poliak Mark Dredze Eric\u00a0C Leas Zechariah Zhu Jessica\u00a0B Kelley Dennis\u00a0J Faix Aaron\u00a0M Goodman Christopher\u00a0A Longhurst Michael Hogarth and Davey\u00a0M Smith. 2023. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern. Med. 183 6 (June 2023) 589\u2013596. 10.1001\/jamainternmed.2023.1838","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","unstructured":"Elahe Bagheri Pablo\u00a0G Esteban Hoang-Long Cao Albert\u00a0De Beir Dirk Lefeber and Bram Vanderborght. 2020. An autonomous cognitive empathy model responsive to users\u2019 facial emotion expressions. ACM Trans. Interact. Intell. Syst. 10 3 (Sept. 2020) 1\u201323. 10.1145\/3341198","DOI":"10.1145\/3341198"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","unstructured":"Luke Balcombe. 2023. AI chatbots in digital mental health. Informatics (MDPI) 10 4 (Oct. 2023) 82. 10.3390\/informatics10040082","DOI":"10.3390\/informatics10040082"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","unstructured":"Lisa\u00a0Feldman Barrett Ralph Adolphs Stacy Marsella Aleix\u00a0M Martinez and Seth\u00a0D Pollak. 2019. Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychol. Sci. Public Interest 20 1 (July 2019) 1\u201368. 10.1177\/1529100619832930","DOI":"10.1177\/1529100619832930"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2993148.2993165"},{"key":"e_1_3_4_12_2","doi-asserted-by":"publisher","unstructured":"C.\u00a0Daniel Batson. 2009. These things called empathy: Eight related but distinct phenomena. The social neuroscience of empathy. 255 (2009) 3\u201315. 10.7551\/mitpress\/9780262012973.003.0002","DOI":"10.7551\/mitpress\/9780262012973.003.0002"},{"key":"e_1_3_4_13_2","doi-asserted-by":"publisher","unstructured":"Yifan Bian Dennis K\u00fcster Hui Liu and Eva\u00a0G Krumhuber. 2023. Understanding naturalistic facial expressions with deep learning and multimodal large language models. Sensors (Basel) 24 1 (Dec. 2023) 126. 10.3390\/s24010126","DOI":"10.3390\/s24010126"},{"key":"e_1_3_4_14_2","doi-asserted-by":"publisher","unstructured":"Eliane\u00a0M Boucher Nicole\u00a0R Harake Haley\u00a0E Ward Sarah\u00a0Elizabeth Stoeckl Junielly Vargas Jared Minkel Acacia\u00a0C Parks and Ran Zilca. 2021. Artificially intelligent chatbots in digital mental health interventions: a review. Expert Rev. Med. Devices 18 sup1 (Dec. 2021) 37\u201349. 10.1080\/17434440.2021.2013200","DOI":"10.1080\/17434440.2021.2013200"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","unstructured":"R\u00a0L Boyd A Ashokkumar S Seraj and J\u00a0W Pennebaker. 2022. The development and psychometric properties of LIWC-22. Austin TX: University of Texas at Austin 10 (2022) 1\u201347. 10.13140\/RG.2.2.23890.43205","DOI":"10.13140\/RG.2.2.23890.43205"},{"key":"e_1_3_4_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445318"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","unstructured":"R Buck V Savin Robert\u00a0E Miller and W\u00a0F Caul. 1972. Communication of affect through facial expressions in humans. J. Pers. Soc. Psychol. 23 3 (Sept. 1972) 362\u2013371. 10.1037\/h0033171","DOI":"10.1037\/h0033171"},{"key":"e_1_3_4_18_2","doi-asserted-by":"publisher","unstructured":"Brant\u00a0R Burleson. 2003. The experience and effects of emotional support: What the study of cultural and gender differences can tell us about close relationships emotion and interpersonal communication. Pers. Relatsh. 10 1 (March 2003) 1\u201323. 10.1111\/1475-6811.00033","DOI":"10.1111\/1475-6811.00033"},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","unstructured":"Carlos Busso Murtaza Bulut Chi-Chun Lee Abe Kazemzadeh Emily Mower Samuel Kim Jeannette\u00a0N Chang Sungbok Lee and Shrikanth\u00a0S Narayanan. 2008. IEMOCAP: Interactive emotional dyadic motion capture database. Language resources and evaluation 42 (2008) 335\u2013359. 10.1007\/s10579-008-9076-6","DOI":"10.1007\/s10579-008-9076-6"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-34960-722"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858498"},{"key":"e_1_3_4_22_2","first-page":"7","volume-title":"Workshop on Towards Intelligent Social Robots: From Naive Robots to Robot Sapiens at the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2018)","author":"Charrier Laurianne","year":"2018","unstructured":"Laurianne Charrier, Alexandre Galdeano, Am\u00e9lie Cordier, and Mathieu Lefort. 2018. Empathy display influence on Human-Robot Interactions: A pilot study. In Workshop on Towards Intelligent Social Robots: From Naive Robots to Robot Sapiens at the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS 2018). IEEE, New York, NY, USA, 7."},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","unstructured":"Siyuan Chen Mengyue Wu Kenny\u00a0Q. Zhu Kunyao Lan Zhiling Zhang and Lyuchun Cui. 2023. LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluation. arxiv:https:\/\/arXiv.org\/abs\/2305.1361410.48550\/arXiv.2305.13614","DOI":"10.48550\/arXiv.2305.13614"},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","unstructured":"S Concannon I Roberts and M Tomalin. 2023. An interactional account of empathy in human-machine communication. Human-Machine Communication 6 (July 2023) 87\u2013116. 10.30658\/hmc.6.6","DOI":"10.30658\/hmc.6.6"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642336"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","unstructured":"Benjamin M.\u00a0P. Cuff Sarah\u00a0J. Brown Laura Taylor and Douglas\u00a0J. Howat. 2016. Empathy: A Review of the Concept. Emot. Rev. 8 2 (April 2016) 144\u2013153. 10.1177\/1754073914558466","DOI":"10.1177\/1754073914558466"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","unstructured":"Douglas\u00a0W. Cunningham Mario Kleiner Christian Wallraven and Heinrich\u00a0H. B\u00fclthoff. 2005. Manipulating Video Sequences to Determine the Components of Conversational Facial Expressions. ACM Trans. Appl. Percept. 2 3 (jul 2005) 251\u2013269. 10.1145\/1077399.1077404","DOI":"10.1145\/1077399.1077404"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383652.3423864"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","unstructured":"Mark\u00a0H. Davis. 1983. Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of personality and social psychology 44 1 (1983) 113. 10.1037\/0022-3514.44.1.113","DOI":"10.1037\/0022-3514.44.1.113"},{"key":"e_1_3_4_30_2","doi-asserted-by":"publisher","unstructured":"Munmun De\u00a0Choudhury Sachin\u00a0R Pendse and Neha Kumar. 2023. Benefits and harms of large language models in digital mental health. 10.48550\/arXiv.2311.14693","DOI":"10.48550\/arXiv.2311.14693"},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","unstructured":"Mauro de Gennaro Eva\u00a0G. Krumhuber and Gale Lucas. 2019. Effectiveness of an Empathic Chatbot in Combating Adverse Effects of Social Exclusion on Mood. Front. Psychol. 10 (2019) 3061. 10.3389\/fpsyg.2019.03061","DOI":"10.3389\/fpsyg.2019.03061"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9780262012973.003.0010"},{"key":"e_1_3_4_33_2","volume-title":"Human communication the basic course (9th ed.)","author":"DeVito Joseph\u00a0A","year":"2003","unstructured":"Joseph\u00a0A DeVito. 2003. Human communication the basic course (9th ed.). Allyn and Bacon, Boston."},{"key":"e_1_3_4_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3651132"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","unstructured":"Nia Dowell and J Berman. 2013. Therapist nonverbal behavior and perceptions of empathy alliance and treatment credibility. Journal of Psychotherapy Integration 23 (June 2013) 158\u2013165. 10.1037\/a0031421","DOI":"10.1037\/a0031421"},{"key":"e_1_3_4_36_2","volume-title":"Darwin and facial expression: A century of research in review","author":"Ekman Paul","year":"2006","unstructured":"Paul Ekman. 2006. Darwin and facial expression: A century of research in review. Ishk, San Jose, CA, USA."},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","unstructured":"Paul Ekman and Wallace\u00a0V Friesen. 1969. The Repertoire of Nonverbal Behavior: Categories Origins Usage and Coding. Semiotica 1 1 (Jan. 1969) 49\u201398. 10.1515\/semi.1969.1.1.49","DOI":"10.1515\/semi.1969.1.1.49"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","unstructured":"Robert Elliott Arthur\u00a0C. Bohart Jeanne\u00a0C. Watson and Leslie\u00a0S. Greenberg. 2011. Empathy. Psychotherapy 48 1 (March 2011) 43\u201349. 10.1037\/a0022187","DOI":"10.1037\/a0022187"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","unstructured":"Robert Elliott Arthur\u00a0C Bohart Jeanne\u00a0C Watson and David Murphy. 2018. Therapist empathy and client outcome: An updated meta-analysis. Psychotherapy (Chic.) 55 4 (Dec. 2018) 399\u2013410. 10.1037\/pst0000175","DOI":"10.1037\/pst0000175"},{"key":"e_1_3_4_40_2","doi-asserted-by":"publisher","unstructured":"Zohar Elyoseph Dorit Hadar-Shoval Kfir Asraf and Maya Lvovsky. 2023. ChatGPT outperforms humans in emotional awareness evaluations. Front. Psychol. 14 (May 2023) 1199058. 10.3389\/fpsyg.2023.1199058","DOI":"10.3389\/fpsyg.2023.1199058"},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","unstructured":"Hadas Erel Denis Trayman Chen Levy Adi Manor Mario Mikulincer and Oren Zuckerman. 2022. Enhancing emotional support: The effect of a robotic object on human\u2013human support quality. Int. J. Soc. Robot. 14 1 (Jan. 2022) 257\u2013276. 10.1007\/s12369-021-00779-5","DOI":"10.1007\/s12369-021-00779-5"},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","unstructured":"Yasaman Etesam \u00d6zge\u00a0Nilay Yal\u00e7\u0131n Chuxuan Zhang and Angelica Lim. 2023. Emotional theory of mind: Bridging Fast visual processing with Slow linguistic reasoning. 10.48550\/arXiv.2310.19995","DOI":"10.48550\/arXiv.2310.19995"},{"key":"e_1_3_4_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858535"},{"key":"e_1_3_4_44_2","doi-asserted-by":"publisher","unstructured":"Franz Faul Edgar Erdfelder Albert-Georg Lang and Axel Buchner. 2007. G*Power 3: a flexible statistical power analysis program for the social behavioral and biomedical sciences. Behav. Res. Methods 39 2 (may 2007) 175\u2013191. 10.3758\/bf03193146","DOI":"10.3758\/bf03193146"},{"key":"e_1_3_4_45_2","doi-asserted-by":"publisher","unstructured":"C\u00a0P Feller and R\u00a0R Cottone. 2003. The Importance of Empathy in the Therapeutic Alliance. Journal of HUMANISIT COUNSELING EDUCATION AND DEVELOPMENT 42 (2003) 53\u201361. 10.1002\/j.2164-490X.2003.tb00168.x","DOI":"10.1002\/j.2164-490X.2003.tb00168.x"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","unstructured":"Pamela Fitzgerald and Ivan Leudar. 2010. On active listening in person-centred solution-focused psychotherapy. J. Pragmat. 42 12 (Dec. 2010) 3188\u20133198. 10.1016\/j.pragma.2010.07.007","DOI":"10.1016\/j.pragma.2010.07.007"},{"key":"e_1_3_4_47_2","doi-asserted-by":"publisher","unstructured":"Kathleen\u00a0Kara Fitzpatrick Alison Darcy and Molly Vierhile. 2017. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): A randomized controlled trial. JMIR Ment. Health 4 2 (June 2017) e19. 10.2196\/mental.7785","DOI":"10.2196\/mental.7785"},{"key":"e_1_3_4_48_2","unstructured":"Gretchen\u00a0N Foley and Julie\u00a0P Gentile. 2010. Nonverbal communication in psychotherapy. Psychiatry (Edgmont) 7 6 (June 2010) 38\u201344."},{"key":"e_1_3_4_49_2","doi-asserted-by":"publisher","unstructured":"Chris Frith. 2009. Role of facial expressions in social interactions. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364 1535 (Dec. 2009) 3453\u20133458. 10.1098\/rstb.2009.0142","DOI":"10.1098\/rstb.2009.0142"},{"key":"e_1_3_4_50_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1307.0414"},{"key":"e_1_3_4_51_2","doi-asserted-by":"publisher","unstructured":"Md\u00a0Romael Haque and Sabirat Rubya. 2022. An overview of chatbot-based mobile mental health apps: Insights from app description and user reviews. JMIR MHealth UHealth 11 (Dec. 2022) e44838. 10.2196\/44838","DOI":"10.2196\/44838"},{"key":"e_1_3_4_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640794.3665536"},{"key":"e_1_3_4_53_2","doi-asserted-by":"publisher","unstructured":"Arthur\u00a0Bran Herbener and Malene\u00a0Flensborg Damholdt. 2025. Are lonely youngsters turning to chatbots for companionship? The relationship between chatbot usage and social connectedness in Danish high-school students. Int. J. Hum. Comput. Stud. 196 103409 (Feb. 2025) 103409. 10.1016\/j.ijhcs.2024.103409","DOI":"10.1016\/j.ijhcs.2024.103409"},{"key":"e_1_3_4_54_2","doi-asserted-by":"publisher","unstructured":"Arthur\u00a0Bran Herbener Micha\u0142 Klincewicz and Malene\u00a0Flensborg Damholdt. 2024. A narrative review of the active ingredients in psychotherapy delivered by conversational agents. Comput. Hum. Behav. Rep. 14 100401 (May 2024) 100401. 10.1016\/j.chbr.2024.100401","DOI":"10.1016\/j.chbr.2024.100401"},{"key":"e_1_3_4_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411763.3451660"},{"key":"e_1_3_4_56_2","doi-asserted-by":"publisher","unstructured":"B\u00a0D Jani D\u00a0N Blane and S\u00a0W Mercer. 2012. The role of empathy in therapy and the physician-patient relationship. Forschende Komplement\u00e4rmedizin\/Research in Complementary Medicine 19 5 (2012) 252\u2013257. 10.1159\/000342998","DOI":"10.1159\/000342998"},{"key":"e_1_3_4_57_2","doi-asserted-by":"publisher","unstructured":"Sooyeon Jeong Laura Aymerich-Franch Sharifa Alghowinem Rosalind\u00a0W Picard Cynthia\u00a0L Breazeal and Hae\u00a0Won Park. 2023. A robotic companion for psychological well-being: A long-term investigation of companionship and therapeutic alliance. Proc. ACM SIGCHI 2023 (March 2023) 484\u2013495. 10.1145\/3568162.3578625","DOI":"10.1145\/3568162.3578625"},{"key":"e_1_3_4_58_2","doi-asserted-by":"publisher","unstructured":"Deborah Johanson Ho\u00a0Seok Ahn Rishab Goswami Kazuki Saegusa and Elizabeth Broadbent. 2023. The Effects of Healthcare Robot Empathy Statements and Head Nodding on Trust and Satisfaction: A Video Study. J. Hum.-Robot Interact. 12 1 (Feb. 2023) 1\u201321. 10.1145\/3549534","DOI":"10.1145\/3549534"},{"key":"e_1_3_4_59_2","doi-asserted-by":"publisher","unstructured":"Susanne\u00a0M Jones Graham\u00a0D Bodie and Sam\u00a0D Hughes. 2019. The impact of mindfulness on empathy active listening and perceived provisions of emotional support. Communic. Res. 46 6 (Aug. 2019) 838\u2013865. 10.1177\/0093650215626983","DOI":"10.1177\/0093650215626983"},{"key":"e_1_3_4_60_2","doi-asserted-by":"publisher","unstructured":"Kathrin Kaulard Douglas\u00a0W Cunningham Heinrich\u00a0H B\u00fclthoff and Christian Wallraven. 2012. The MPI facial expression database\u2013a validated database of emotional and conversational facial expressions. PLoS One 7 3 (March 2012) e32321. 10.1371\/journal.pone.0032321","DOI":"10.1371\/journal.pone.0032321"},{"key":"e_1_3_4_61_2","doi-asserted-by":"publisher","unstructured":"Ronak Kosti Jose\u00a0M Alvarez Adria Recasens and Agata Lapedriza. 2020. Context based emotion recognition using EMOTIC dataset. 10.48550\/arXiv.2003.13401","DOI":"10.48550\/arXiv.2003.13401"},{"key":"e_1_3_4_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN57019.2023.10309551"},{"key":"e_1_3_4_63_2","doi-asserted-by":"publisher","unstructured":"Fran\u00e7ois Lauzier-Jobin and Janie Houle. 2022. A comparison of formal and informal help in the context of mental health recovery. Int. J. Soc. Psychiatry 68 4 (June 2022) 729\u2013737. 10.1177\/00207640211004988","DOI":"10.1177\/00207640211004988"},{"key":"e_1_3_4_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376175"},{"key":"e_1_3_4_65_2","doi-asserted-by":"publisher","unstructured":"Yoon\u00a0Kyung Lee Yoonwon Jung Gyuyi Kang and Sowon Hahn. 2023. Developing Social Robots with Empathetic Non-Verbal Cues Using Large Language Models. 10.48550\/arXiv.2308.16529","DOI":"10.48550\/arXiv.2308.16529"},{"key":"e_1_3_4_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/1877826.1877839"},{"key":"e_1_3_4_67_2","doi-asserted-by":"publisher","unstructured":"Xingwei Liang Geng Tu Jiachen Du and Ruifeng Xu. 2024. Multi-modal Attentive Prompt learning for few-shot emotion recognition in conversations. J. Artif. Intell. Res. 79 (March 2024) 825\u2013863. 10.1613\/jair.1.15301","DOI":"10.1613\/jair.1.15301"},{"key":"e_1_3_4_68_2","first-page":"128","volume-title":"Workshop on Computational Modeling of People\u2019s Opinions, Personality, and Emotions in Social Media","author":"Litvak Marina","year":"2016","unstructured":"Marina Litvak, Jahna Otterbacher, Chee\u00a0Siang Ang, and David Atkins. 2016. Social and Linguistic Behavior and its Correlation to Trait Empathy. In Workshop on Computational Modeling of People\u2019s Opinions, Personality, and Emotions in Social Media. The COLING 2016 Organizing Committee, Osaka, Japan, 128\u2013137."},{"key":"e_1_3_4_69_2","doi-asserted-by":"publisher","unstructured":"Bingjie Liu and S\u00a0Shyam Sundar. 2018. Should Machines Express Sympathy and Empathy? Experiments with a Health Advice Chatbot. Cyberpsychol. Behav. Soc. Netw. 21 10 (Oct. 2018) 625\u2013636. 10.1089\/cyber.2018.0110","DOI":"10.1089\/cyber.2018.0110"},{"key":"e_1_3_4_70_2","doi-asserted-by":"publisher","unstructured":"Siru Liu Allison\u00a0B McCoy Aileen\u00a0P Wright Babatunde Carew Julian\u00a0Z Genkins Sean\u00a0S Huang Josh\u00a0F Peterson Bryan Steitz and Adam Wright. 2024. Leveraging large language models for generating responses to patient messages\u2014a subjective analysis. Journal of the American Medical Informatics Association 31 6 (May 2024) 1367\u20131379. 10.1101\/2023.07.14.23292669","DOI":"10.1101\/2023.07.14.23292669"},{"key":"e_1_3_4_71_2","doi-asserted-by":"publisher","unstructured":"Kate Loveys Mark Sagar and Elizabeth Broadbent. 2020. The effect of multimodal emotional expression on responses to a digital human during a self-disclosure conversation: A computational analysis of user language. J. Med. Syst. 44 9 (July 2020) 143. 10.1007\/s10916-020-01624-4","DOI":"10.1007\/s10916-020-01624-4"},{"key":"e_1_3_4_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/IE49459.2020.9155004"},{"key":"e_1_3_4_73_2","doi-asserted-by":"publisher","unstructured":"Mina Marmpena Angelica Lim and Torbj\u00f8rn\u00a0S Dahl. 2018. How does the robot feel? Perception of valence and arousal in emotional body language. Paladyn Journal of Behavioral Robotics 9 1 (July 2018) 168\u2013182. 10.1515\/pjbr-2018-0012","DOI":"10.1515\/pjbr-2018-0012"},{"key":"e_1_3_4_74_2","doi-asserted-by":"publisher","unstructured":"Gary McKeown Michel Valstar Roddy Cowie Maja Pantic and Marc Schroder. 2012. The SEMAINE Database: Annotated Multimodal Records of Emotionally Colored Conversations between a Person and a Limited Agent. IEEE Transactions on Affective Computing 3 1 (Jan. 2012) 5\u201317. 10.1109\/T-AFFC.2011.20","DOI":"10.1109\/T-AFFC.2011.20"},{"key":"e_1_3_4_75_2","doi-asserted-by":"publisher","unstructured":"Vaibhav Mehra Guy Laban and Hatice Gunes. 2025. Beyond Vision: How large Language Models interpret facial expressions from Valence-Arousal values. 10.48550\/arXiv.2502.06875","DOI":"10.48550\/arXiv.2502.06875"},{"key":"e_1_3_4_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025548"},{"key":"e_1_3_4_77_2","doi-asserted-by":"publisher","unstructured":"Ali Mollahosseini Behzad Hasani and Mohammad\u00a0H Mahoor. 2017. AffectNet: A database for facial expression valence and arousal computing in the wild. 10.48550\/arXiv.1708.03985","DOI":"10.48550\/arXiv.1708.03985"},{"key":"e_1_3_4_78_2","doi-asserted-by":"publisher","unstructured":"Robert\u00a0R Morris Kareem Kouddous Rohan Kshirsagar and Stephen\u00a0M Schueller. 2018. Towards an Artificially Empathic Conversational Agent for Mental Health Applications: System Design and User Perceptions. J. Med. Internet Res. 20 6 (June 2018) e10148. 10.2196\/10148","DOI":"10.2196\/10148"},{"key":"e_1_3_4_79_2","doi-asserted-by":"publisher","unstructured":"Maria Moudatsou Areti Stavropoulou Anastas Philalithis and Sofia Koukouli. 2020. The role of empathy in health and social care professionals. Healthcare (Basel) 8 1 (Jan. 2020) 26. 10.3390\/healthcare8010026","DOI":"10.3390\/healthcare8010026"},{"key":"e_1_3_4_80_2","doi-asserted-by":"publisher","unstructured":"Mohammad Nadeem Shahab\u00a0Saquib Sohail Laeeba Javed Faisal Anwer Abdul Khader\u00a0Jilani Saudagar and Khan Muhammad. 2024. Vision-Enabled Large Language and Deep Learning Models for Image-Based Emotion Recognition. Cogn Comput 16 (May 2024) 2566\u2013\u20132579. 10.1007\/s12559-024-10281-5","DOI":"10.1007\/s12559-024-10281-5"},{"key":"e_1_3_4_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3383062"},{"key":"e_1_3_4_82_2","doi-asserted-by":"publisher","unstructured":"Alice Nardelli Antonio Sgorbissa and Carmine\u00a0Tommaso Recchiuto. 2025. Designing empathetic companions: Exploring personality emotion and trust in social robots. 10.48550\/arXiv.2504.13964","DOI":"10.48550\/arXiv.2504.13964"},{"key":"e_1_3_4_83_2","doi-asserted-by":"publisher","unstructured":"Melanie Neumann Jozien Bensing Stewart Mercer Nicole Ernstmann Oliver Ommen and Holger Pfaff. 2009. Analyzing the \u201cnature\u201d and \u201cspecific effectiveness\u201d of clinical empathy: a theoretical overview and contribution towards a theory-based research agenda. Patient Educ. Couns. 74 3 (March 2009) 339\u2013346. 10.1016\/j.pec.2008.11.013","DOI":"10.1016\/j.pec.2008.11.013"},{"key":"e_1_3_4_84_2","doi-asserted-by":"publisher","unstructured":"Jacob\u00a0B Nienhuis Jesse Owen Jeffrey\u00a0C Valentine Stephanie Winkeljohn\u00a0Black Tyler\u00a0C Halford Stephanie\u00a0E Parazak Stephanie Budge and Mark Hilsenroth. 2018. Therapeutic alliance empathy and genuineness in individual adult psychotherapy: A meta-analytic review. Psychother. Res. 28 4 (July 2018) 593\u2013605. 10.1080\/10503307.2016.1204023","DOI":"10.1080\/10503307.2016.1204023"},{"key":"e_1_3_4_85_2","doi-asserted-by":"publisher","unstructured":"Ana Paiva Iolanda Leite Hana Boukricha and Ipke Wachsmuth. 2017. Empathy in Virtual Agents and Robots: A Survey. ACM Trans. Interact. Intell. Syst. 7 3 Article MHCI039 (Sept. 2017) 40\u00a0pages. 10.1145\/2912150","DOI":"10.1145\/2912150"},{"key":"e_1_3_4_86_2","doi-asserted-by":"publisher","unstructured":"Sung Park and Mincheol Whang. 2022. Empathy in Human-Robot Interaction: Designing for Social Robots. Int. J. Environ. Res. Public Health 19 3 (Feb. 2022) 1\u201321. 10.3390\/ijerph19031889","DOI":"10.3390\/ijerph19031889"},{"key":"e_1_3_4_87_2","doi-asserted-by":"publisher","unstructured":"Dhaval Parmar Stefan Olafsson Dina Utami Prasanth Murali and Timothy Bickmore. 2022. Designing Empathic Virtual Agents: Manipulating Animation Voice Rendering and Empathy to Create Persuasive Agents. Auton. Agent. Multi. Agent. Syst. 36 1 (April 2022) 31\u00a0pages. 10.1007\/s10458-021-09539-1","DOI":"10.1007\/s10458-021-09539-1"},{"key":"e_1_3_4_88_2","doi-asserted-by":"publisher","unstructured":"Paul\u00a0R Peluso and Robert\u00a0R Freund. 2018. Therapist and client emotional expression and psychotherapy outcomes: A meta-analysis. Psychotherapy (Chic.) 55 4 (Dec. 2018) 461\u2013472. 10.1037\/pst0000165","DOI":"10.1037\/pst0000165"},{"key":"e_1_3_4_89_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-61140-741"},{"key":"e_1_3_4_90_2","doi-asserted-by":"publisher","unstructured":"Kay\u00a0T Pham Amir Nabizadeh and Salih Selek. 2022. Artificial intelligence and chatbots in psychiatry. Psychiatr. Q. 93 1 (March 2022) 249\u2013253. 10.1007\/s11126-022-09973-8","DOI":"10.1007\/s11126-022-09973-8"},{"key":"e_1_3_4_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519797"},{"key":"e_1_3_4_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300475"},{"key":"e_1_3_4_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642035"},{"key":"e_1_3_4_94_2","doi-asserted-by":"publisher","unstructured":"Vera Sorin Danna Brin Yiftach Barash Eli Konen Alexander Charney Girish Nadkarni and Eyal Klang. 2023. Large Language Models (LLMs) and empathy - A systematic review. 10.1101\/2023.08.07.23293769","DOI":"10.1101\/2023.08.07.23293769"},{"key":"e_1_3_4_95_2","doi-asserted-by":"publisher","unstructured":"Benjamin\u00a0A Tabak Gareth Leng Angela Szeto Karen\u00a0J Parker Joseph\u00a0G Verbalis Toni\u00a0E Ziegler Mary\u00a0R Lee Inga\u00a0D Neumann and Armando\u00a0J Mendez. 2023. Advances in human oxytocin measurement: challenges and proposed solutions. Mol. Psychiatry 28 1 (Jan. 2023) 127\u2013140. 10.1038\/s41380-022-01719-z","DOI":"10.1038\/s41380-022-01719-z"},{"key":"e_1_3_4_96_2","doi-asserted-by":"publisher","unstructured":"Yla\u00a0R Tausczik and James\u00a0W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. J. Lang. Soc. Psychol. 29 1 (March 2010) 24\u201354. 10.1177\/0261927X09351676","DOI":"10.1177\/0261927X09351676"},{"key":"e_1_3_4_97_2","doi-asserted-by":"publisher","unstructured":"Gerben Van\u00a0Kleef. 2017. The social effects of emotions are functionally equivalent across expressive modalities. Psychological Inquiry 28 2-3 (2017) 211\u2013216. 10.1080\/1047840X.2017.1338102","DOI":"10.1080\/1047840X.2017.1338102"},{"key":"e_1_3_4_98_2","doi-asserted-by":"publisher","unstructured":"Gerben\u00a0A Van\u00a0Kleef. 2010. The emerging view of emotion as social information. Social and Personality Psychology Compass 4 5 (2010) 331\u2013343. 10.1111\/j.1751-9004.2010.00262.x","DOI":"10.1111\/j.1751-9004.2010.00262.x"},{"key":"e_1_3_4_99_2","doi-asserted-by":"publisher","unstructured":"Gerben\u00a0A van Kleef and St\u00e9phane C\u00f4t\u00e9. 2022. The social effects of emotions. Annu. Rev. Psychol. 73 1 (Jan. 2022) 629\u2013658. 10.1146\/annurev-psych-020821-010855","DOI":"10.1146\/annurev-psych-020821-010855"},{"key":"e_1_3_4_100_2","doi-asserted-by":"publisher","unstructured":"Lesley Verhofstadt Inge Devoldre Ann Buysse Michael Stevens C\u00e9line Hinnekens William Ickes and Mark Davis. 2016. The role of cognitive and affective empathy in spouses\u2019 support interactions: An observational study. PLoS One 11 2 (Feb. 2016) e0149944. 10.1371\/journal.pone.0149944","DOI":"10.1371\/journal.pone.0149944"},{"key":"e_1_3_4_101_2","volume-title":"I scanned 1000+ prompts so you don\u2019t have to: 10 need-to-know techniques","author":"Vogel Maximilian","year":"2023","unstructured":"Maximilian Vogel. 2023. I scanned 1000+ prompts so you don\u2019t have to: 10 need-to-know techniques. https:\/\/medium.com\/@maximilian.vogel\/i-scanned-1000-prompts-so-you-dont-have-to-10-need-to-know-techniques-a77bcd074d97"},{"key":"e_1_3_4_102_2","doi-asserted-by":"publisher","unstructured":"Bruce\u00a0E Wampold. 2015. How important are the common factors in psychotherapy? An update. World Psychiatry 14 3 (Oct. 2015) 270\u2013277. 10.1002\/wps.20238","DOI":"10.1002\/wps.20238"},{"key":"e_1_3_4_103_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203582015"},{"key":"e_1_3_4_104_2","doi-asserted-by":"publisher","DOI":"10.1037\/14775-005"},{"key":"e_1_3_4_105_2","doi-asserted-by":"publisher","unstructured":"Jeremy\u00a0J Webb. 2023. Proof of concept: Using ChatGPT to teach emergency physicians how to break bad news. Cureus 15 5 (May 2023) e38755. 10.7759\/cureus.38755","DOI":"10.7759\/cureus.38755"},{"key":"e_1_3_4_106_2","doi-asserted-by":"publisher","unstructured":"Anuradha Welivita and Pearl Pu. 2024. Is ChatGPT More Empathetic than Humans?10.48550\/arXiv.2403.05572","DOI":"10.48550\/arXiv.2403.05572"},{"key":"e_1_3_4_107_2","doi-asserted-by":"publisher","unstructured":"Philipp Wicke. 2024. Probing Language Models\u2019 Gesture Understanding for Enhanced Human-AI Interaction. 10.48550\/arXiv.2401.17858","DOI":"10.48550\/arXiv.2401.17858"},{"key":"e_1_3_4_108_2","doi-asserted-by":"publisher","unstructured":"Siyi Wu Feixue Han Bingsheng Yao Tianyi Xie Xuan Zhao and Dakuo Wang. 2024. Sunnie: An anthropomorphic LLM-based conversational agent for mental well-being activity recommendation. 10.48550\/arXiv.2405.13803","DOI":"10.48550\/arXiv.2405.13803"},{"key":"e_1_3_4_109_2","doi-asserted-by":"publisher","DOI":"10.1109\/RO-MAN60168.2024.10731248"},{"key":"e_1_3_4_110_2","doi-asserted-by":"publisher","unstructured":"Xiao Yan and Yi Ding. 2025. Are we there yet? A measurement study of efficiency for LLM applications on mobile devices. 10.48550\/arXiv.2504.00002","DOI":"10.48550\/arXiv.2504.00002"},{"key":"e_1_3_4_111_2","doi-asserted-by":"publisher","unstructured":"Xiyuan Zhang Ranak\u00a0Roy Chowdhury Rajesh\u00a0K Gupta and Jingbo Shang. 2024. Large Language Models for time series: A survey. 10.48550\/arXiv.2402.01801","DOI":"10.48550\/arXiv.2402.01801"},{"key":"e_1_3_4_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/3640794.3665550"},{"key":"e_1_3_4_113_2","doi-asserted-by":"publisher","unstructured":"Weixiang Zhao Yanyan Zhao Xin Lu Shilong Wang Yanpeng Tong and Bing Qin. 2023. Is ChatGPT equipped with emotional dialogue capabilities?10.48550\/arXiv.2304.09582","DOI":"10.48550\/arXiv.2304.09582"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3743724","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T16:08:23Z","timestamp":1757520503000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3743724"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,9]]},"references-count":112,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,9,30]]}},"alternative-id":["10.1145\/3743724"],"URL":"https:\/\/doi.org\/10.1145\/3743724","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,9]]},"assertion":[{"value":"2025-09-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}