{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T19:06:17Z","timestamp":1787166377057,"version":"build-2736575974"},"reference-count":92,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"name":"University of Virginia Engineering in Medicine Seed Pilot Program, the Gordon and Betty Moore Foundation, the National Institute of Mental Health of the National Institutes of Health","award":["R01MH132138"],"award-info":[{"award-number":["R01MH132138"]}]},{"name":"Commonwealth Cyber Initiative, an investment in the advancement of cyber R & D, innovation, and workforce development"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2026,7,30]]},"abstract":"<jats:p>Effective patient-provider communication is crucial in clinical care, directly impacting patient outcomes and quality of life. Traditional evaluation methods, such as human ratings, patient feedback, and provider self-assessments, are often limited by high costs and scalability issues. Although existing natural language processing (NLP) techniques show promise, they struggle with the nuances of clinical communication and require sensitive clinical data for training, reducing their effectiveness in real-world applications. Emerging large language models (LLMs) offer a new approach to assessing complex communication metrics, with the potential to advance the field through integration into passive sensing and just-in-time intervention systems. This study explores LLMs as evaluators of palliative care communication quality, leveraging their linguistic, in-context learning, and reasoning capabilities. Specifically, using simulated scripts crafted and labeled by healthcare professionals, we test proprietary models (e.g., GPT-4) and fine-tune open-source LLMs (e.g., LLaMA2) with a synthetic dataset generated by GPT-4 to evaluate clinical conversations, to identify key metrics such as \u201cunderstanding\u201d and \u201cempathy.\u201d Our findings demonstrated LLMs\u2019 superior performance in evaluating clinical communication, providing actionable feedback with reasoning, and demonstrating the feasibility and practical viability of developing in-house LLMs. This research highlights LLMs\u2019 potential to enhance patient-provider interactions and lays the groundwork for downstream steps in developing LLM-empowered clinical health systems.<\/jats:p>","DOI":"10.1145\/3712300","type":"journal-article","created":{"date-parts":[[2025,1,23]],"date-time":"2025-01-23T12:59:31Z","timestamp":1737637171000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["PALLM: Evaluating and Enhancing Palliative Care Conversations with Large Language Models"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1611-2053","authenticated-orcid":false,"given":"Zhiyuan","family":"Wang","sequence":"first","affiliation":[{"name":"Engineering Systems and Environment, University of Virginia, Charlottesville, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1203-1850","authenticated-orcid":false,"given":"Fangxu","family":"Yuan","sequence":"additional","affiliation":[{"name":"Department of Systems and Information Engineering, University of Virginia, Charlottesville, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1299-4730","authenticated-orcid":false,"given":"Virginia","family":"LeBaron","sequence":"additional","affiliation":[{"name":"School of Nursing, University of Virginia, Charlottesville, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8820-2536","authenticated-orcid":false,"given":"Tabor","family":"Flickinger","sequence":"additional","affiliation":[{"name":"School of Medicine, University of Virginia, Charlottesville, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8224-5164","authenticated-orcid":false,"given":"Laura E.","family":"Barnes","sequence":"additional","affiliation":[{"name":"Engineering Systems and Environment, University of Virginia, Charlottesville, Virginia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,22]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"191","article-title":"Health Insurance Portability and Accountability Act of 1996","volume":"104","author":"Act Accountability","year":"1996","unstructured":"Accountability Act. 1996. Health Insurance Portability and Accountability Act of 1996. Public Law 104 (1996), 191.","journal-title":"Public Law"},{"key":"e_1_3_2_3_2","first-page":"363","article-title":"The Oncotalk\/Vitaltalk model","author":"Arnold Robert M.","year":"2017","unstructured":"Robert M. Arnold, Anthony L. Back, Walter F. Baile, Kelly A. Edwards, and James A. Tulsky. 2017. The Oncotalk\/Vitaltalk model. In Oxford Textbook of Communication in Oncology and Palliative Care (nd ed.). Oxford Textbooks in Palliative Medicine, 363.","journal-title":"Oxford Textbook of Communication in Oncology and Palliative Care"},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","unstructured":"Annemieke P. Bikker Bridie Fitzpatrick Douglas Murphy and Stewart W. Mercer. 2015. Measuring empathic person-centred communication in primary care nurses: Validity and reliability of the consultation and relational empathy (CARE) measure. BMC Family Practice 16 (2015) 1\u20139.","DOI":"10.1186\/s12875-015-0374-y"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Lisa Jane Brighton Jonathan Koffman Amy Hawkins Christine McDonald Suzanne O\u2019Brien Vicky Robinson Shaheen A. Khan Rob George Irene J. Higginson and Lucy Ellen Selman. 2017. A systematic review of end-of-life care communication skills training for generalist palliative care providers: Research quality and reporting guidance. Journal of Pain and Symptom Management 54 3 (2017) 417\u2013425.","DOI":"10.1016\/j.jpainsymman.2017.04.008"},{"key":"e_1_3_2_6_2","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D. Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems Vol. 33 1877\u20131901."},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Elizabeth Lee Byerly. 1969. The nurse researcher as participant-observer in a nursing setting. Nursing Research 18 3 (1969) 230\u2013235.","DOI":"10.1097\/00006199-196905000-00007"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.nlpmc-1.9"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Natasha Clarke Peter Foltz and Peter Garrard. 2020. How to do things with (thousands of) words: Computational approaches to discourse analysis in Alzheimer\u2019s disease. Cortex 129 (2020) 446\u2013463.","DOI":"10.1016\/j.cortex.2020.05.001"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2013.282081"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"David A. Davis Paul E. Mazmanian Michael Fordis R. T. K. E. Van Harrison Kevin E. Thorpe and Laure Perrier. 2006. Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. JAMA 296 9 (2006) 1094\u20131102.","DOI":"10.1001\/jama.296.9.1094"},{"key":"e_1_3_2_12_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv.1810.04805. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1810.04805"},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","unstructured":"Ning Ding Yujia Qin Guang Yang Fuchao Wei Zonghan Yang Yusheng Su Shengding Hu Yulin Chen Chi-Min Chan Weize Chen et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence 5 3 (2023) 220\u2013235.","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Sammy Chouffani El Fassi Adonis Abdullah Ying Fang Sarabesh Natarajan Awab Bin Masroor Naya Kayali Simran Prakash and Gail E. Henderson. 2024. Not all AI health tools with regulatory authorization are clinically validated. Nature Medicine 30 (2024) 2718\u20132720.","DOI":"10.1038\/s41591-024-03203-3"},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Benjamin Ewert Farina Hodiamont Jeroen van Wijngaarden Sheila Payne Marieke Groot Jeroen Hasselaar Johann Menten and Lukas Radbruch. 2016. Building a taxonomy of integrated palliative care initiatives: Results from a focus group. BMJ Supportive & Palliative Care 6 1 (2016) 14\u201320.","DOI":"10.1136\/bmjspcare-2014-000841"},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Tabor Flickinger Virginia LeBaron Mehdi Boukhechba James Edwards Zhiyuan Wang David Ling Daniel Wilson and Laura Barnes. 2023. Evidence-based approach to designing wearable technology to improve patient-clinician communication. Patient Education and Counseling 109 (2023) 108\u2013109.","DOI":"10.1016\/j.pec.2022.10.251"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Cynthia Foronda Karina Gattamorta Kenya Snowden and Eric B. Bauman. 2014. Use of virtual clinical simulation to improve communication skills of baccalaureate nursing students: A pilot study. Nurse Education Today 34 6 (2014) e53\u2013e57.","DOI":"10.1016\/j.nedt.2013.10.007"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","unstructured":"Olaf P. Geerse Daniela J. Lamas Justin J. Sanders Joanna Paladino Jane Kavanagh Natalie J. Henrich Annette J. Berendsen Thijo J. N. Hiltermann Erik K. Fromme Rachelle E. Bernacki et al. 2019. A qualitative study of serious illness conversations in patients with advanced cancer. Journal of Palliative Medicine 22 7 (2019) 773\u2013781.","DOI":"10.1089\/jpm.2018.0487"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"Joy Goldsmith Betty Ferrell Elaine Wittenberg-Lyles and Sandra L. Ragan. 2013. Palliative care communication in oncology nursing. Clinical Journal of Oncology Nursing 17 2 (2013) 163\u2013167.","DOI":"10.1188\/13.CJON.163-167"},{"key":"e_1_3_2_20_2","unstructured":"Michael Hahn and Navin Goyal. 2023. A theory of emergent in-context learning as implicit structure induction. arXiv:2303.07971. Retrieved from https:\/\/arxiv.org\/abs\/2303.07971"},{"key":"e_1_3_2_21_2","doi-asserted-by":"crossref","unstructured":"Jodi Halpern. 2003. What is clinical empathy? Journal of General Internal Medicine 18 (2003) 670\u2013674.","DOI":"10.1046\/j.1525-1497.2003.21017.x"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580688"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2669-y"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s44159-023-00235-3"},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"S. C. Hawkins A. Osborne S. J. Schofield D. J. Pournaras and J. F. Chester. 2012. Improving the accuracy of self-assessment of practical clinical skills using video feedback\u2013The importance of including benchmarks. Medical Teacher 34 4 (2012) 279\u2013284.","DOI":"10.3109\/0142159X.2012.658897"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Ruining He and Julian McAuley. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web 507\u2013517.","DOI":"10.1145\/2872427.2883037"},{"issue":"2","key":"e_1_3_2_27_2","first-page":"2","article-title":"Experienced practitioners\u2019 views on interpersonal skills in telehealth delivery","volume":"16","author":"Henry Beverly W.","year":"2018","unstructured":"Beverly W. Henry, Leah J. Ames, Derryl E. Block, and John A. Vozenilek. 2018. Experienced practitioners\u2019 views on interpersonal skills in telehealth delivery. Internet Journal of Allied Health Sciences and Practice 16, 2 (2018), 2.","journal-title":"Internet Journal of Allied Health Sciences and Practice"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Beverly W. Henry Derryl E. Block James R. Ciesla Beth Ann McGowan and John A. Vozenilek. 2017. Clinician behaviors in telehealth care delivery: A systematic review. Advances in Health Sciences Education 22 (2017) 869\u2013888.","DOI":"10.1007\/s10459-016-9717-2"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","unstructured":"Mohammadreza Hojat. 2016. Empathy in health professions education and patient Care. DOI: 10.1007\/978-3-319-27625-0","DOI":"10.1007\/978-3-319-27625-0"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i14.17539"},{"key":"e_1_3_2_31_2","unstructured":"Edward J. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. arXiv.2106.09685. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2106.09685"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","unstructured":"Jessica Ji Josh Goldstein and Andrew Lohn. 2023. Controlling Large Language Models: A Primer. Technical Report. Center for Security and Emerging Technology. DOI: 10.51593\/2023CA009","DOI":"10.51593\/2023CA009"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2016.16840"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","unstructured":"Jan Koco\u0144 Igor Cichecki Oliwier Kaszyca Mateusz Kochanek Dominika Szyd\u0142o Joanna Baran Julita Bielaniewicz Marcin Gruza Arkadiusz Janz Kamil Kanclerz et al. 2023. ChatGPT: Jack of all trades master of none. Information Fusion 99 (Nov. 2023) 101861. DOI: 10.1016\/j.inffus.2023.101861","DOI":"10.1016\/j.inffus.2023.101861"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Takeshi Kojima Shixiang Shane Gu Machel Reid Yutaka Matsuo and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. In Advances in Neural Information Processing Systems Vol. 35 22199\u201322213.","DOI":"10.52202\/068431-1613"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2115730119"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Suzanne Kurtz Jonathan Silverman John Benson and Juliet Draper. 2003. Marrying content and process in clinical method teaching: Enhancing the Calgary\u2013Cambridge guides. Academic Medicine 78 8 (2003) 802\u2013809.","DOI":"10.1097\/00001888-200308000-00011"},{"key":"e_1_3_2_38_2","unstructured":"Bishal Lamichhane. 2023. Evaluation of ChatGPT for NLP-based mental health applications. arXiv.2303.15727. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2303.15727"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","unstructured":"Virginia LeBaron. 2024. Caring in Context: An Ethnography of Cancer Nursing in India. Taylor & Francis.","DOI":"10.4324\/9781003413158"},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","unstructured":"Virginia LeBaron Mehdi Boukhechba James Edwards Tabor Flickinger David Ling and Laura E. Barnes. 2022. Exploring the use of wearable sensors and natural language processing technology to improve patient-clinician communication: Protocol for a feasibility study. JMIR Research Protocols 11 5 (2022) e37975.","DOI":"10.2196\/37975"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231184991"},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","unstructured":"Nayeon Lee Wei Ping Peng Xu Mostofa Patwary Pascale N. Fung Mohammad Shoeybi and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. Advances in Neural Information Processing Systems Vol. 35 34586\u201334599.","DOI":"10.52202\/068431-2506"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2024.108013"},{"key":"e_1_3_2_44_2","unstructured":"Yuanchun Li Hao Wen Weijun Wang Xiangyu Li Yizhen Yuan Guohong Liu Jiacheng Liu Wenxing Xu Xiang Wang Yi Sun et al. 2024. Personal LLM Agents: Insights and survey about the capability efficiency and security. arXiv:2401.05459. Retrieved from https:\/\/arxiv.org\/abs\/2401.05459"},{"key":"e_1_3_2_45_2","unstructured":"Zaijing Li Gongwei Chen Rui Shao Dongmei Jiang and Liqiang Nie. 2024. Enhancing emotional generation capability of large language models via emotional chain-of-thought. arXiv.2401.06836. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2401.06836"},{"key":"e_1_3_2_46_2","unstructured":"Sourab Mangrulkar Sylvain Gugger Lysandre Debut Younes Belkada Sayak Paul and Benjamin Bossan. 2022. PEFT: State-of-the-art parameter-efficient fine-tuning methods. Retrieved from https:\/\/github.com\/huggingface\/peft"},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Edward Meinert Jessie Eerens Christina Banks Stephen Maloney George Rivers Dragan Ilic Kieran Walsh Azeem Majeed and Josip Car. 2021. Exploring the cost of eLearning in health professions education: Scoping review. JMIR Medical Education 7 1 (2021) e13681.","DOI":"10.2196\/13681"},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","unstructured":"Stewart W. Mercer Alex McConnachie Margaret Maxwell David Heaney and Graham C. M. Watt. 2005. Relevance and practical use of the consultation and relational empathy (CARE) measure in general practice. Family Practice 22 3 (2005) 328\u2013334.","DOI":"10.1093\/fampra\/cmh730"},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Bertalan Mesk\u00f3. 2023. The impact of multimodal large language models on health care\u2019s future. Journal of Medical Internet Research 25 (2023) e52865.","DOI":"10.2196\/52865"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMcp035232"},{"key":"e_1_3_2_51_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv.2303.08774. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2303.08774"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"Julie Parker and Enrico Coiera. 2000. Improving clinical communication: A view from psychology. Journal of the American Medical Informatics Association 7 5 (2000) 453\u2013461.","DOI":"10.1136\/jamia.2000.0070453"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","unstructured":"Cheng Peng Xi Yang Aokun Chen Kaleb E. Smith Nima PourNejatian Anthony B. Costa Cheryl Martin Mona G. Flores Ying Zhang Tanja Magoc et al. 2023. A study of generative large language model for medical research and healthcare. arXiv:2305.13523. Retrieved from https:\/\/arxiv.org\/abs\/2305.13523","DOI":"10.1038\/s41746-023-00958-w"},{"key":"e_1_3_2_54_2","unstructured":"Matthew E. Peters Sebastian Ruder and Noah A. Smith. 2019. To tune or not to tune? Adapting pretrained representations to diverse tasks. arXiv:1903.05987. Retrieved from https:\/\/arxiv.org\/abs\/1903.05987"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"Eboni G. Price Donna M. Windish Jeffrey Magaziner and Lisa A. Cooper. 2008. Assessing validity of standardized patient ratings of medical students\u2019 communication behavior using the roter interaction analysis system. Patient Education and Counseling 70 1 (2008) 3\u20139.","DOI":"10.1016\/j.pec.2007.10.002"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","unstructured":"Andy Pulman Janet Scammell and Michael Martin. 2009. Enabling interprofessional education: The role of technology to enhance learning. Nurse Education Today 29 2 (2009) 232\u2013239.","DOI":"10.1016\/j.nedt.2008.08.012"},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","unstructured":"Raul Puri Ryan Spring Mostofa Patwary Mohammad Shoeybi and Bryan Catanzaro. 2020. Training question answering models from synthetic data. arXiv:2002.09599. Retrieved from https:\/\/arxiv.org\/abs\/2002.09599","DOI":"10.18653\/v1\/2020.emnlp-main.468"},{"key":"e_1_3_2_58_2","unstructured":"Chengwei Qin Aston Zhang Zhuosheng Zhang Jiaao Chen Michihiro Yasunaga and Diyi Yang. 2023. Is ChatGPT a general-purpose natural language processing task solver? arXiv.2302.06476. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2302.06476"},{"key":"e_1_3_2_59_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.5195\/ijt.2017.6232"},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","unstructured":"Debra Roter and Susan Larson. 2002. The roter interaction analysis system (RIAS): Utility and flexibility for analysis of medical interactions. Patient Education and Counseling 46 4 (2002) 243\u2013251.","DOI":"10.1016\/S0738-3991(02)00012-5"},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","unstructured":"Justin J. Sanders Joanna Paladino Erica Reaves Hannah Luetke-Stahlman Rebecca Anhang Price Karl Lorenz Laura C. Hanson J. Randall Curtis Diane E. Meier Erik K. Fromme et al. 2020. Quality measurement of serious illness communication: Recommendations for health systems based on findings from a symposium of national experts. Journal of Palliative Medicine 23 1 (2020) 13\u201321.","DOI":"10.1089\/jpm.2019.0335"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.2196\/22421"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","unstructured":"Monika Sennekamp Katja Gilbert Ferdinand M. Gerlach and Corina Guethlin. 2012. Development and validation of the \u201cFrOCK\u201d: Frankfurt observer communication checklist. Zeitschrift f\u00fcr Evidenz Fortbildung und Qualit\u00e4t im Gesundheitswesen 106 8 (2012) 595\u2013601.","DOI":"10.1016\/j.zefq.2012.07.018"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.nedt.2020.104592"},{"key":"e_1_3_2_66_2","doi-asserted-by":"crossref","unstructured":"Linda Sweigart Marsha Burden Kay Hodson Carlton and John Fillwalk. 2014. Virtual simulations across curriculum prepare nursing students for patient interviews. Clinical Simulation in Nursing 10 3 (2014) e139\u2013e145.","DOI":"10.1016\/j.ecns.2013.10.003"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00896-7"},{"key":"e_1_3_2_68_2","unstructured":"Ruixiang Tang Xiaotian Han Xiaoqian Jiang and Xia Hu. 2023a. Does synthetic data generation of LLMS help clinical text mining? arXiv:2303.04360. Retrieved from https:\/\/arxiv.org\/abs\/2303.04360"},{"key":"e_1_3_2_69_2","doi-asserted-by":"crossref","unstructured":"Yla R. Tausczik and James W. Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology 29 1 (2010) 24\u201354.","DOI":"10.1177\/0261927X09351676"},{"key":"e_1_3_2_70_2","doi-asserted-by":"crossref","unstructured":"Arun James Thirunavukarasu Darren Shu Jeng Ting Kabilan Elangovan Laura Gutierrez Ting Fang Tan and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature Medicine 29 8 (2023) 1930\u20131940.","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_3_2_71_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv.2307.09288. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2307.09288"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","unstructured":"James A. Tulsky Mary Catherine Beach Phyllis N. Butow Susan E. Hickman Jennifer W. Mack R. Sean Morrison Richard L. Street Rebecca L. Sudore Douglas B. White and Kathryn I. Pollak. 2017. A research agenda for communication between health care professionals and patients living with serious illness. JAMA Internal Medicine 177 9 (2017) 1361\u20131366.","DOI":"10.1001\/jamainternmed.2017.2005"},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","unstructured":"Mengting Wan and Julian McAuley. 2016. Modeling ambiguity subjectivity and diverging viewpoints in opinion question answering systems. In Proceedings of the 2016 IEEE 16th International Conference on Data Mining (ICDM \u201916). IEEE 489\u2013498.","DOI":"10.1109\/ICDM.2016.0060"},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","unstructured":"Bailin Wang Zi Wang Xuezhi Wang Yuan Cao Rif A. Saurous and Yoon Kim. 2023. Grammar prompting for domain-specific language generation with large language models. arXiv.2305.19234. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2305.19234","DOI":"10.52202\/075280-2837"},{"key":"e_1_3_2_75_2","unstructured":"Cunxiang Wang Xiaoze Liu Yuanhao Yue Xiangru Tang Tianhang Zhang Cheng Jiayang Yunzhi Yao Wenyang Gao Xuming Hu Zehan Qi et al. 2023. Survey on factuality in large language models: Knowledge retrieval and domain-specificity. arXiv:2310.07521. Retrieved from https:\/\/arxiv.org\/abs\/2310.07521"},{"key":"e_1_3_2_76_2","unstructured":"Xuezhi Wang Jason Wei Dale Schuurmans Quoc Le Ed Chi Sharan Narang Aakanksha Chowdhery and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv:2203.11171. Retrieved from https:\/\/arxiv.org\/abs\/2203.11171"},{"key":"e_1_3_2_77_2","unstructured":"Xuezhi Wang Jason Wei Dale Schuurmans Quoc Le Ed Chi Sharan Narang Aakanksha Chowdhery and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. arXiv:2203.11171. Retrieved from https:\/\/arxiv.org\/abs\/2203.11171"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3686952"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610916"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613905.3651077"},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","unstructured":"Micki Washburn and Shu Zhou. 2018. Teaching note\u2014Technology-enhanced clinical simulations: Tools for practicing clinical skills in online social work programs. Journal of Social Work Education 54 3 (2018) 554\u2013560.","DOI":"10.1080\/10437797.2017.1404519"},{"key":"e_1_3_2_82_2","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter Fei Xia Ed Chi Quoc Le and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. arXiv.2201.11903. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2201.11903"},{"key":"e_1_3_2_83_2","doi-asserted-by":"crossref","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Fei Xia Ed Chi Quoc V. Le and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems Vol. 35 24824\u201324837.","DOI":"10.52202\/068431-1800"},{"key":"e_1_3_2_84_2","doi-asserted-by":"crossref","unstructured":"Julia Wikert Daniela Gesell Claudia Bausewein Maximiliane Jansky Friedemann Nauck Steven Kranz and Farina Hodiamont. 2022. Specialist palliative care classification: Typology development. BMJ Supportive & Palliative Care 14 (2022) e1488\u2013e1496.","DOI":"10.1136\/bmjspcare-2021-003435"},{"key":"e_1_3_2_85_2","doi-asserted-by":"crossref","unstructured":"Michael Woods Giselle Appel Aidana Daulbayeva Caleb Harris Rosalind Picard Julia Iyasere and Jonathan Avery. 2024. Integrating technology Into undergraduate medical education: Can affective computing help teach empathy? Academic Psychiatry 48 1 (2024) 110\u2013111.","DOI":"10.1007\/s40596-023-01900-8"},{"key":"e_1_3_2_86_2","doi-asserted-by":"crossref","unstructured":"Pranjul Yadav Michael Steinbach Vipin Kumar and Gyorgy Simon. 2018. Mining electronic health records (EHRs) a survey. ACM Computing Surveys 50 6 (2018) 1\u201340.","DOI":"10.1145\/3127881"},{"key":"e_1_3_2_87_2","doi-asserted-by":"crossref","unstructured":"Kailai Yang Shaoxiong Ji Tianlin Zhang Qianqian Xie Ziyan Kuang and Sophia Ananiadou. 2023. Towards interpretable mental health analysis with large language models. arXiv.2304.03347. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2304.03347","DOI":"10.18653\/v1\/2023.emnlp-main.370"},{"key":"e_1_3_2_88_2","unstructured":"Shunyu Yao Dian Yu Jeffrey Zhao Izhak Shafran Tom Griffiths Yuan Cao and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems Vol. 517 14."},{"key":"e_1_3_2_89_2","unstructured":"Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan Yiyang Zhou Junyang Wang Anwen Hu Pengcheng Shi Yaya Shi et al. 2023. Mplug-owl: Modularization empowers large language models with multimodality. arXiv:2304.14178. Retrieved from https:\/\/arxiv.org\/abs\/2304.14178"},{"key":"e_1_3_2_90_2","doi-asserted-by":"crossref","unstructured":"Wen-wai Yim Meliha Yetisgen William P. Harris and Sharon W. Kwan. 2016. Natural language processing in oncology: A review. JAMA Oncology 2 6 (2016) 797\u2013804.","DOI":"10.1001\/jamaoncol.2016.0213"},{"key":"e_1_3_2_91_2","doi-asserted-by":"crossref","unstructured":"Yue Yu Yuchen Zhuang Jieyu Zhang Yu Meng Alexander J. Ratner Ranjay Krishna Jiaming Shen and Chao Zhang. 2024. Large language model as attributed training data generator: A tale of diversity and bias. In Advances in Neural Information Processing Systems Vol. 36 55734\u201355784.","DOI":"10.52202\/075280-2433"},{"key":"e_1_3_2_92_2","doi-asserted-by":"crossref","unstructured":"Tianyi Zhang Faisal Ladhak Esin Durmus Percy Liang Kathleen McKeown and Tatsunori B. Hashimoto. 2024. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics 12 (2024) 39\u201357.","DOI":"10.1162\/tacl_a_00632"},{"key":"e_1_3_2_93_2","unstructured":"Qihuang Zhong Liang Ding Juhua Liu Bo Du and Dacheng Tao. 2023. Can ChatGPT understand too? A comparative study on ChatGPT and fine-tuned BERT. arXiv.2302.10198. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.2302.10198"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3712300","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T13:16:00Z","timestamp":1782134160000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3712300"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,22]]},"references-count":92,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,7,30]]}},"alternative-id":["10.1145\/3712300"],"URL":"https:\/\/doi.org\/10.1145\/3712300","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,22]]},"assertion":[{"value":"2024-03-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}