{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:57:18Z","timestamp":1791233838735,"version":"4.1.0"},"reference-count":106,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Conversational AI systems are increasingly being deployed in health care for clinical decision support, but their performance varies substantially across patient communication styles, health literacy levels, and behavioral patterns. Static benchmarks cannot capture multiturn dynamics through which this variation compounds, and no current evaluation framework implements structured AI risk management guidance for conversational health care AI. The result is a structural risk: AI systems may perform well in aggregate while failing disproportionately for the populations they are intended to help.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to develop and validate a patient simulation framework that aligns with the National Institute of Standards and Technology (NIST) AI Risk Management Framework (AI RMF) Map and Measure functions, providing an empirical basis for identifying and characterizing performance risks in conversational clinical AI across medical, linguistic, and behavioral patient variations. We applied the framework to a conversational decision aid for antidepressant selection in major depressive disorder (the AI decision aid).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The simulator integrated three profile dimensions: (1) medical profiles constructed from All of Us electronic health records using risk ratio gating; (2) linguistic profiles modeling a health literacy gradient and condition-specific communication; and (3) behavioral profiles representing cooperative, distracted, and adversarial engagement. We generated 500 simulated conversations and evaluated profile fidelity through human annotation and a large language model (LLM) judge, and then assessed downstream effects on the AI decision aid\u2019s concept retrieval and antidepressant recommendations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The patient simulator expressed medical concepts with high fidelity (96.6% accuracy across 8210 concepts), with substantial human interannotator agreement (\u03ba=0.73) and LLM-judge agreement against human annotators (\u03ba=0.78). Behavioral profiles were reliably distinguished (\u03ba=0.93; near perfect agreement), and linguistic profiles showed substantial agreement at the lower bound of the substantial range (\u03ba=0.61), which can be considered adequate to support profile-level analysis. The framework revealed monotonic degradation in AI decision aid performance across the health literacy gradient. Rank-1 concept retrieval increased from 47.6% for limited health literacy to 81.9% for proficient health literacy, with corresponding declines in antidepressant recommendation accuracy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Patient simulation grounded in the NIST AI RMF exposes measurable performance risks in conversational health care AI that static benchmarks miss, with direct equity implications. Health literacy operates as a structural risk factor, with degraded performance concentrated in patients carrying the greatest burden of psychiatric illness. The framework supports targeted risk-mitigation interventions before deployment. While we evaluated the framework only on antidepressant selection, extending it to other clinical decision-aid tasks remains an assignment for future work.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/100772","type":"journal-article","created":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:25:18Z","timestamp":1791231918000},"page":"e100772-e100772","source":"Crossref","is-referenced-by-count":0,"title":["A Patient Simulation Framework for Risk Assessment of Conversational Health Care AI: Development and Evaluation Study"],"prefix":"10.2196","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5571-8632","authenticated-orcid":false,"given":"Md Tanvir Rouf","family":"Shawon","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8629-107X","authenticated-orcid":false,"given":"Mohammad Sabik","family":"Irbaz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0540-8231","authenticated-orcid":false,"given":"Hadeel R A","family":"Elyazori","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9837-3115","authenticated-orcid":false,"given":"Keerti Reddy","family":"Resapu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-6771-5911","authenticated-orcid":false,"given":"Yili","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6731-5827","authenticated-orcid":false,"given":"Vladimir Franzuela","family":"Cardenas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5242-2991","authenticated-orcid":false,"given":"K Pierre","family":"Eklou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7402-9063","authenticated-orcid":false,"given":"Farrokh","family":"Alemi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5798-2664","authenticated-orcid":false,"given":"Kevin","family":"Lybarger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,5]]},"reference":[{"issue":"9","key":"R1","doi-asserted-by":"publisher","first-page":"1248","DOI":"10.1093\/jamia\/ocy072","article-title":"Conversational agents in healthcare: a systematic review","volume":"25","author":"Laranjo","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","DOI":"10.3390\/bioengineering12060631","article-title":"Large language models in healthcare and medical applications: a review","volume":"12","author":"Maity","journal-title":"Bioengineering (Basel)"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"issue":"8067","key":"R4","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1038\/s41586-025-08866-7","article-title":"Towards conversational diagnostic artificial intelligence","volume":"642","author":"Tu","journal-title":"Nature"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","DOI":"10.1186\/s12912-021-00684-2","article-title":"A literature-based study of patient-centered care and communication in nurse-patient interactions: barriers, facilitators, and the way forward","volume":"20","author":"Kwame","journal-title":"BMC Nurs"},{"issue":"Suppl 2","key":"R6","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1080\/10810730.2010.499981","article-title":"Health literacy and communication quality in health care organizations","volume":"15 Suppl 2","author":"Wynia","journal-title":"J Health Commun"},{"key":"R7","doi-asserted-by":"crossref","unstructured":"Gourabathina A Gerych W Pan E Ghassemi M . The medium is the message: how non-clinical information shapes clinical decisions in LLMs. Presented at: 2025 ACM Conference on Fairness, Accountability, and Transparency; Jun 23-26, 2025; Athens, Greece. [doi: 10.1145\/3715275.3732121]","DOI":"10.1145\/3715275.3732121"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.2196\/69006","article-title":"Standardizing and scaffolding health care AI-chatbot evaluation: systematic review","volume":"4","author":"Hua","journal-title":"JMIR AI"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01786-w","article-title":"Framework for bias evaluation in large language models in healthcare settings","volume":"8","author":"Templin","journal-title":"NPJ Digit Med"},{"issue":"5","key":"R10","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000492","article-title":"Achieving health equity through conversational AI: a roadmap for design and implementation of inclusive chatbots in healthcare","volume":"3","author":"Nadarzynski","journal-title":"PLOS Digit Health"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00879-8","article-title":"The shaky foundations of large language models and foundation models for electronic health records","volume":"6","author":"Wornow","journal-title":"NPJ Digit Med"},{"key":"R12","unstructured":"Guan S Wang J Bian J Zhu B Lou JG Xiong H . Evaluating LLM-based agents for multi-turn conversations: a survey. arXiv. Preprint posted online on Mar 28, 2025. [doi: 10.48550\/arXiv.2503.22458]"},{"key":"R13","doi-asserted-by":"crossref","unstructured":"Kwan WC Zeng X Jiang Y et al. MT-eval: a multi-turn capabilities evaluation benchmark for large language models. Presented at: 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, FL, USA. [doi: 10.18653\/v1\/2024.emnlp-main.1124]","DOI":"10.18653\/v1\/2024.emnlp-main.1124"},{"key":"R14","doi-asserted-by":"crossref","unstructured":"Balog K Zhai C . User simulation for evaluating information access systems. Presented at: 2023 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region; Nov 26-28, 2023; Beijing, China. [doi: 10.1145\/3624918.3629549]","DOI":"10.1145\/3624918.3629549"},{"key":"R15","doi-asserted-by":"crossref","unstructured":"Kalinich M Luccarelli J Moss F Torous J . Leveraging simulation to provide a practical framework for assessing the novel scope of risk of LLMs in healthcare. medRxiv. Preprint posted online on Nov 13, 2025. [doi: 10.1101\/2025.11.10.25339903]","DOI":"10.1101\/2025.11.10.25339903"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Tabassi E . Artificial intelligence risk management framework (AI RMF 1.0). NIST. 2023. URL: https:\/\/www.nist.gov\/publications\/artificial-intelligence-risk-management-framework-ai-rmf-10 [Accessed 28-10-2025]","DOI":"10.6028\/NIST.AI.100-1"},{"issue":"7","key":"R17","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1056\/NEJMsr1809937","article-title":"The \u201call of us\u201d research program","volume":"381","author":"All of Us Research Program Investigators","journal-title":"N Engl J Med"},{"key":"R18","unstructured":"Institute of Medicine (US) Committee on Health Literacy. Nielsen-Bohlman L Panzer AM Kindig DA , editors. Health Literacy: A Prescription to End Confusion. National Academies Press (US); 2004. URL: https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK216032\/ [Accessed 02-08-2026]"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1093\/heapro\/15.3.259","article-title":"Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century","volume":"15","author":"Nutbeam","journal-title":"Health Promot Int"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.psych.54.101601.145041","article-title":"Psychological aspects of natural language. use: our words, our selves","volume":"54","author":"Pennebaker","journal-title":"Annu Rev Psychol"},{"issue":"8","key":"R21","doi-asserted-by":"publisher","first-page":"1121","DOI":"10.1080\/02699930441000030","article-title":"Language use of depressed and depression-vulnerable college students","volume":"18","author":"Rude","journal-title":"Cogn Emot"},{"key":"R22","doi-asserted-by":"crossref","unstructured":"Roter D Hall JA . Doctors Talking with Patients\/Patients Talking with Doctors. Bloomsbury Publishing; 2006. ISBN: 9780275990145","DOI":"10.5040\/9798400641794"},{"issue":"3","key":"R23","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1016\/j.pec.2008.11.015","article-title":"How does communication heal? Pathways linking clinician-patient communication to health outcomes","volume":"74","author":"Street","journal-title":"Patient Educ Couns"},{"key":"R24","unstructured":"Lybargerlanguagelab\/Patient-Simulation. GitHub. URL: https:\/\/github.com\/lybargerlanguagelab\/Patient-Simulation [Accessed 19-08-2026]"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.1057\/s41599-024-03611-3","article-title":"Large language models empowered agent-based modeling and simulation: a survey and perspectives","volume":"11","author":"Gao","journal-title":"Humanit Soc Sci Commun"},{"key":"R26","doi-asserted-by":"crossref","unstructured":"Montaner M L\u00f3pez B de la Rosa JL . Evaluation of recommender systems through simulated users. Presented at: 6th International Conference on Enterprise Information Systems; Apr 14-17, 2004; Porto, Portugal. [doi: 10.5220\/0002622703030308]","DOI":"10.5220\/0002622703030308"},{"key":"R27","unstructured":"Yao S Halpern Y Thain N et al. Measuring recommender system effects with simulated users. arXiv. Preprint posted online on Jan 12, 2021. [doi: 10.48550\/arXiv.2101.04526]"},{"key":"R28","doi-asserted-by":"crossref","unstructured":"Luo X Tang Z Wang J Zhang X . DuetSim: building user simulator with dual large language models for task-oriented dialogues. Presented at: 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation; May 20-25, 2024; Turin, Italy. [doi: 10.63317\/5cp2zxmyham6]","DOI":"10.63317\/5cp2zxmyham6"},{"issue":"2","key":"R29","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1017\/S0269888906000944","article-title":"A survey of statistical user simulation techniques for reinforcement-learning of dialogue management strategies","volume":"21","author":"Schatzmann","journal-title":"Knowl Eng Rev"},{"key":"R30","unstructured":"Labhishetty S . Models and evaluation of user simulation in information retrieval. Illinois University Library. 2023. URL: https:\/\/www.ideals.illinois.edu\/items\/127324 [Accessed 29-11-2025]"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Zhang Y Liu X Zhai C . Information retrieval evaluation as search simulation: a general formal framework for IR evaluation. Presented at: 2017 ACM SIGIR International Conference on Theory of Information Retrieval; Oct 1-4, 2017; Amsterdam, The Netherlands. [doi: 10.1145\/3121050.3121070]","DOI":"10.1145\/3121050.3121070"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Yun T Yang E Safdari M et al. Sleepless nights, sugary days: creating synthetic users with health conditions for realistic coaching agent interactions. Presented at: Findings of the Association for Computational Linguistics: ACL 2025; Jul 27 to Aug 1, 2025; Vienna, Austria. [doi: 10.18653\/v1\/2025.findings-acl.729]","DOI":"10.18653\/v1\/2025.findings-acl.729"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.698","article-title":"An integrative survey on mental health conversational agents to bridge computer science and medical perspectives","volume":"2023","author":"Cho","journal-title":"Proc Conf Empir Methods Nat Lang Process"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/68486","article-title":"Virtual patients using large language models: scalable, contextualized simulation of clinician-patient dialogue with feedback","volume":"27","author":"Cook","journal-title":"J Med Internet Res"},{"key":"R35","doi-asserted-by":"crossref","unstructured":"Kyung D Chung H Bae S et al. PatientSim: a persona-driven simulator for realistic doctor-patient interactions. arXiv. Preprint posted online on May 23, 2025. [doi: 10.48550\/arXiv.2505.17818]","DOI":"10.52202\/085713-0861"},{"key":"R36","unstructured":"Liu L Yang X Lei J et al. A survey on medical large language models: technology, application, trustworthiness, and future directions. arXiv. Preprint posted online on Jun 6, 2024. [doi: 10.48550\/arXiv.2406.03712]"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.2196\/59213","article-title":"A language model-powered simulated patient with automated feedback for history taking: prospective study","volume":"10","author":"Holderried","journal-title":"JMIR Med Educ"},{"issue":"1","key":"R38","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01841-6","article-title":"A large language model digital patient system enhances ophthalmology history taking skills","volume":"8","author":"Luo","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R39","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1093\/jamia\/ocx079","article-title":"Synthea: an approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record","volume":"25","author":"Walonoski","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"R40","doi-asserted-by":"publisher","DOI":"10.3390\/s19051181","article-title":"SynSys: a synthetic data generation system for healthcare applications","volume":"19","author":"Dahmen","journal-title":"Sensors (Basel)"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.2196\/52615","article-title":"Generating synthetic electronic health record data using generative adversarial networks: tutorial","volume":"3","author":"Yan","journal-title":"JMIR AI"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00888-7","article-title":"EHR-Safe: generating high-fidelity and privacy-preserving synthetic electronic health records","volume":"6","author":"Yoon","journal-title":"NPJ Digit Med"},{"issue":"7","key":"R43","doi-asserted-by":"publisher","first-page":"1227","DOI":"10.1093\/jamia\/ocaf082","article-title":"Generating synthetic electronic health record data: a methodological scoping review with benchmarking on phenotype data and open-source software","volume":"32","author":"Chen","journal-title":"J Am Med Inform Assoc"},{"key":"R44","doi-asserted-by":"crossref","unstructured":"Rabaey P Heytens S Demeester T . SimSUM: simulated benchmark with structured and unstructured medical records. arXiv. Preprint posted online on Sep 13, 2024. [doi: 10.48550\/arXiv.2409.08936]","DOI":"10.1186\/s13326-025-00341-6"},{"key":"R45","unstructured":"Yu H Zhou J Li L et al. Simulated patient systems are intelligent when powered by large language model-based AI agents. arXiv. Preprint posted online on Sep 27, 2024. [doi: 10.48550\/arXiv.2409.18924]"},{"key":"R46","unstructured":"Bodonhelyi A Stegemann-Philipps C Sonanini A et al. Modeling challenging patient interactions: LLMs for medical communication training. arXiv. Preprint posted online on Mar 28, 2025. [doi: 10.48550\/arXiv.2503.22250]"},{"key":"R47","doi-asserted-by":"crossref","unstructured":"Wang R Milani S Chiu JC et al. PATIENT-\ud835\udf13: using large language models to simulate patients for training mental health professionals. Presented at: 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, FL, USA. [doi: 10.18653\/v1\/2024.emnlp-main.711]","DOI":"10.18653\/v1\/2024.emnlp-main.711"},{"key":"R48","doi-asserted-by":"crossref","unstructured":"Bao Z Liu Q Huang X Wei Z . SFMSS: service flow aware medical scenario simulation for conversational data generation. Presented at: Findings of the Association for Computational Linguistics: NAACL 2025; Apr 29 to May 4, 2025; Albuquerque, New Mexico. [doi: 10.18653\/v1\/2025.findings-naacl.259]","DOI":"10.18653\/v1\/2025.findings-naacl.259"},{"key":"R49","doi-asserted-by":"crossref","unstructured":"Sehgal NKR Kambhamettu H Chang A Zhu A Ungar L Guntuku SC . PAL: designing conversational agents as scalable, cooperative patient simulators for palliative\u2011care training. Presented at: 2025 Conference on Computer-Supported Cooperative Work and Social Computing; Oct 18-22, 2025; Bergen, Norway. [doi: 10.1145\/3715070.3749250]","DOI":"10.1145\/3715070.3749250"},{"key":"R50","unstructured":"Wojtusiak J . Towards intelligent patient data generator. Machine Learning and Inference Laboratory, George Mason University; 2016. URL: https:\/\/www.mli.gmu.edu\/papers\/2016\/16-9.pdf [Accessed 18-03-2026]"},{"issue":"3","key":"R51","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1002\/wps.21352","article-title":"Charting the evolution of artificial intelligence mental health chatbots from rule-based systems to large language models: a systematic review","volume":"24","author":"Hua","journal-title":"World Psychiatry"},{"issue":"1","key":"R52","doi-asserted-by":"publisher","DOI":"10.1186\/s12888-021-03133-1","article-title":"A surrogate measure for patient reported symptom remission in administrative data","volume":"21","author":"Alemi","journal-title":"BMC Psychiatry"},{"key":"R53","unstructured":"Most common antidepressants. Definitive Healthcare. URL: https:\/\/www.definitivehc.com\/resources\/healthcare-insights\/top-antidepressants-by-prescription-volume [Accessed 11-11-2025]"},{"issue":"10","key":"R54","doi-asserted-by":"publisher","first-page":"1730","DOI":"10.1093\/jamia\/ocad120","article-title":"Electronic health record data quality assessment and tools: a systematic review","volume":"30","author":"Lewis","journal-title":"J Am Med Inform Assoc"},{"key":"R55","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2020.103671","article-title":"Deep representation learning of patient data from electronic health records (EHR): a systematic review","volume":"115","author":"Si","journal-title":"J Biomed Inform"},{"key":"R56","doi-asserted-by":"publisher","DOI":"10.2196\/42615","article-title":"Digital health data quality issues: systematic review","volume":"25","author":"Syed","journal-title":"J Med Internet Res"},{"key":"R57","unstructured":"Lim E He YV Joselowitz J et al. MATRIX: multi-agent simulation framework for safe interactions and contextual clinical conversational evaluation. arXiv. Preprint posted online on Aug 26, 2025. [doi: 10.48550\/arXiv.2508.19163]"},{"issue":"1","key":"R58","doi-asserted-by":"publisher","DOI":"10.2196\/75849","article-title":"Harm reduction strategies for thoughtful use of large language models in the medical domain: perspectives for patients and clinicians","volume":"27","author":"Mo\u00ebll","journal-title":"J Med Internet Res"},{"key":"R59","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2458-12-80","article-title":"Health literacy and public health: a systematic review and integration of definitions and models","volume":"12","author":"S\u00f8rensen","journal-title":"BMC Public Health"},{"key":"R60","unstructured":"Zhou X Kim H Brahman F et al. HAICOSYSTEM: an ecosystem for sandboxing safety risks in human-AI interactions. arXiv. Preprint posted online on Sep 24, 2024. [doi: 10.48550\/arXiv.2409.16427]"},{"key":"R61","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944968","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","journal-title":"J Mach Learn Res"},{"issue":"8","key":"R62","doi-asserted-by":"publisher","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"R63","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-024-02059-x","article-title":"Patient engagement with conversational agents in health applications 2016-2022: a systematic review and meta-analysis","volume":"48","author":"Cevasco","journal-title":"J Med Syst"},{"key":"R64","unstructured":"Wolf MS Bailey SC . The role of health literacy in patient safety. PSNet. 2009. URL: https:\/\/psnet.ahrq.gov\/perspective\/role-health-literacy-patient-safety [Accessed 11-02-2026]"},{"key":"R65","unstructured":"Lee S Kim M Cherif L et al. Learning diverse attacks on large language models for robust red-teaming and safety tuning. arXiv. Preprint posted online on May 28, 2024. [doi: 10.48550\/arXiv.2405.18540]"},{"key":"R66","doi-asserted-by":"crossref","unstructured":"Perez E Huang S Song F et al. Red teaming language models with language models. Presented at: 2022 Conference on Empirical Methods in Natural Language Processing; Dec 7-11, 2022; Abu Dhabi, United Arab Emirates. [doi: 10.18653\/v1\/2022.emnlp-main.225]","DOI":"10.18653\/v1\/2022.emnlp-main.225"},{"key":"R67","unstructured":"National action plan to improve health literacy. CDC. 2024. URL: https:\/\/www.cdc.gov\/health-literacy\/php\/develop-plan\/national-action-plan.html [Accessed 04-11-2025]"},{"key":"R68","doi-asserted-by":"publisher","DOI":"10.5555\/ajhb.2007.31.supp.S19","article-title":"The causal pathways linking health literacy to health outcomes","volume":"31 Suppl 1","author":"Paasche-Orlow","journal-title":"Am J Health Behav"},{"issue":"1","key":"R69","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1177\/0261927X09351676","article-title":"The psychological meaning of words: LIWC and computerized text analysis methods","volume":"29","author":"Tausczik","journal-title":"J Lang Soc Psychol"},{"key":"R70","doi-asserted-by":"crossref","unstructured":"DeVault D Artstein R Benn G et al. SimSensei kiosk: a virtual human interviewer for healthcare decision support. Presented at: 2014 International Conference on Autonomous Agents and Multi-Agent Systems; May 5-9, 2014; Paris, France. [doi: 10.5555\/2615731.2617415]","DOI":"10.65109\/MXIV3169"},{"issue":"2","key":"R71","doi-asserted-by":"publisher","DOI":"10.2196\/mental.7785","article-title":"Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial","volume":"4","author":"Fitzpatrick","journal-title":"JMIR Ment Health"},{"key":"R72","doi-asserted-by":"crossref","unstructured":"Liao Y Meng Y Wang Y et al. Automatic interactive evaluation for large language models with state aware patient simulator. SSRN. Preprint posted online on Jul 15, 2024. [doi: 10.2139\/ssrn.4890649]","DOI":"10.2139\/ssrn.4890649"},{"issue":"1","key":"R73","doi-asserted-by":"publisher","DOI":"10.2196\/58974","article-title":"Empathic conversational agent platform designs and their evaluation in the context of mental health: systematic review","volume":"11","author":"Sanjeewa","journal-title":"JMIR Ment Health"},{"key":"R74","doi-asserted-by":"crossref","unstructured":"Simpson S McDowell A . The Clinical Interview: Skills for More Effective Patient Encounters. Routledge; 2019. [doi: 10.4324\/9780429437243]","DOI":"10.4324\/9780429437243"},{"key":"R75","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. Presented at: 36th International Conference on Neural Information Processing System; Nov 28 to Dec 9, 2022; New Orleans, LA, USA. [doi: 10.5555\/3600270.3602070]"},{"key":"R76","doi-asserted-by":"crossref","unstructured":"Alemi F Lin Y Elyazori HRA Cardenas VF Ramezani N Lybarger K . Causal AI reasoning: direct effects multiplicative inference algorithm. SSRN. Preprint posted online on Apr 21, 2026. [doi: 10.2139\/ssrn.6537875]","DOI":"10.2139\/ssrn.6537875"},{"key":"R77","doi-asserted-by":"crossref","unstructured":"Kincaid JP Fishburne RP Rogers RL Chissom BS . Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel. Institute for Simulation and Training; 1975. URL: https:\/\/stars.library.ucf.edu\/cgi\/viewcontent.cgi?article=1055&context=istlibrary [Accessed 10-02-2026]","DOI":"10.21236\/ADA006655"},{"key":"R78","doi-asserted-by":"crossref","unstructured":"Loper E Bird S . NLTK: the natural language toolkit. Presented at: ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics; Jul 7, 2022; Philadelphia, PA, USA. [doi: 10.3115\/1118108.1118117]","DOI":"10.3115\/1118108.1118117"},{"key":"R79","unstructured":"Salazar A . XLM-RoBERTa depression detection model. GitHub. URL: https:\/\/github.com\/malexandersalazar\/xlm-roberta-base-cls-depression [Accessed 18-09-2026]"},{"key":"R80","doi-asserted-by":"crossref","unstructured":"Bevendorff J Casals XB Chulvi B et al. Overview of PAN 2024: multi-author writing style analysis, multilingual text detoxification, oppositional thinking analysis, and generative AI authorship verification \u2014 extended abstract. Presented at: Advances in Information Retrieval: 46th European Conference on Information Retrieval; Mar 24-28, 2024; Glasgow, UK. [doi: 10.1007\/978-3-031-56072-9_1]","DOI":"10.1007\/978-3-031-56072-9_1"},{"key":"R81","doi-asserted-by":"crossref","unstructured":"Bhat S Varma V . Large language models as annotators: a preliminary evaluation for annotating low-resource language content. Presented at: 4th Workshop on Evaluation and Comparison of NLP Systems; Nov 1, 2023; Bali, Indonesia. [doi: 10.18653\/v1\/2023.eval4nlp-1.8]","DOI":"10.18653\/v1\/2023.eval4nlp-1.8"},{"key":"R82","doi-asserted-by":"crossref","unstructured":"Pavlovic M Poesio M . The effectiveness of LLMs as annotators: a comparative overview and empirical analysis of direct representation. Presented at: 3rd Workshop on Perspectivist Approaches to NLP; May 21, 2024; Turin, Italy. [doi: 10.63317\/5bf5rv86yt8a]","DOI":"10.63317\/5bf5rv86yt8a"},{"key":"R83","doi-asserted-by":"crossref","unstructured":"Zhang R Li Y Ma Y Zhou M Zou L . LLMaAA: making large language models as active annotators. arXiv. Preprint posted online on 2023. [doi: 10.48550\/arXiv.2310.19596]","DOI":"10.18653\/v1\/2023.findings-emnlp.872"},{"issue":"3","key":"R84","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1093\/jamia\/ocaf233","article-title":"Testing and evaluation of generative large language models in electronic health record applications: a systematic review","volume":"33","author":"Du","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"R85","doi-asserted-by":"publisher","first-page":"2058","DOI":"10.1007\/s11606-008-0805-8","article-title":"The increasing number of clinical items addressed during the time of adult primary care visits","volume":"23","author":"Abbo","journal-title":"J Gen Intern Med"},{"key":"R86","unstructured":"LangChain. GitHub. URL: https:\/\/github.com\/langchain-ai\/langchain [Accessed 07-05-2026]"},{"key":"R87","unstructured":"Jurafsky D Martin JH . Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models. 3rd ed. Stanford University; 2026. URL: https:\/\/web.stanford.edu\/~jurafsky\/slp3\/ [Accessed 18-09-2026]"},{"issue":"1","key":"R88","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1176\/appi.ajp.163.1.28","article-title":"Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice","volume":"163","author":"Trivedi","journal-title":"Am J Psychiatry"},{"issue":"3","key":"R89","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1002\/wps.21136","article-title":"Challenges of research on treatment-resistant depression: a clinician\u2019s perspective","volume":"22","author":"Rush","journal-title":"World Psychiatry"},{"issue":"6","key":"R90","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1111\/eip.13090","article-title":"Health literacy of people living with mental illness or substance use disorders: a systematic review","volume":"15","author":"Degan","journal-title":"Early Interv Psychiatry"},{"issue":"2","key":"R91","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1111\/j.1525-1497.2005.40245.x","article-title":"The prevalence of limited health literacy","volume":"20","author":"Paasche-Orlow","journal-title":"J Gen Intern Med"},{"issue":"2","key":"R92","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1093\/aje\/kwf182","article-title":"Socioeconomic inequalities in depression: a meta-analysis","volume":"157","author":"Lorant","journal-title":"Am J Epidemiol"},{"issue":"4","key":"R93","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1093\/eurpub\/ckz127","article-title":"Inequalities in the economic consequences of depression and anxiety in Europe: a systematic scoping review","volume":"30","author":"Linder","journal-title":"Eur J Public Health"},{"key":"R94","doi-asserted-by":"publisher","DOI":"10.1155\/2013\/158746","article-title":"Psychomotor retardation in depression: a systematic review of diagnostic, pathophysiologic, and therapeutic implications","volume":"2013","author":"Bennabi","journal-title":"Biomed Res Int"},{"issue":"1","key":"R95","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1017\/S1092852918001207","article-title":"Cognitive impairment in major depressive disorder","volume":"24","author":"Pan","journal-title":"CNS Spectr"},{"issue":"3","key":"R96","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1097\/FCH.0000000000000071","article-title":"Trauma informed care in medicine: current knowledge and future research directions","volume":"38","author":"Raja","journal-title":"Fam Community Health"},{"issue":"1-2","key":"R97","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1002\/ajcp.12217","article-title":"Trauma-informed care for individuals with serious mental illness: an avenue for community psychology\u2019s involvement in community mental health","volume":"61","author":"Mihelicova","journal-title":"Am J Community Psychol"},{"issue":"11","key":"R98","doi-asserted-by":"publisher","first-page":"1905","DOI":"10.1176\/ajp.2006.163.11.1905","article-title":"Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report","volume":"163","author":"Rush","journal-title":"AJP"},{"issue":"6464","key":"R99","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","article-title":"Dissecting racial bias in an algorithm used to manage the health of populations","volume":"366","author":"Obermeyer","journal-title":"Science"},{"issue":"7","key":"R100","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1136\/bmjqs-2025-018723","article-title":"Bridging diagnostic safety and mental health: a systematic review highlighting inequities in autism spectrum disorder diagnosis","volume":"35","author":"Srivarathan","journal-title":"BMJ Qual Saf"},{"issue":"9","key":"R101","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1097\/NMD.0b013e31817d0181","article-title":"Limited literacy and psychiatric disorders among users of an urban safety-net hospital\u2019s mental health outpatient clinic","volume":"196","author":"Lincoln","journal-title":"J Nerv Ment Dis"},{"issue":"6","key":"R102","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1017\/S0033291713001785","article-title":"The methods and outcomes of cultural adaptations of psychological treatments for depressive disorders: a systematic review","volume":"44","author":"Chowdhary","journal-title":"Psychol Med"},{"issue":"9","key":"R103","doi-asserted-by":"publisher","first-page":"954","DOI":"10.1111\/j.1742-1241.2011.02746.x","article-title":"Effectiveness of interventions to improve antidepressant medication adherence: a systematic review","volume":"65","author":"Chong","journal-title":"Int J Clin Pract"},{"issue":"5","key":"R104","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1056\/NEJMra050100","article-title":"Adherence to medication","volume":"353","author":"Osterberg","journal-title":"N Engl J Med"},{"issue":"9","key":"R105","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The PHQ-9: validity of a brief depression severity measure","volume":"16","author":"Kroenke","journal-title":"J Gen Intern Med"},{"issue":"1","key":"R106","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1136\/jnnp.23.1.56","article-title":"A rating scale for depression","volume":"23","author":"Hamilton","journal-title":"J Neurol Neurosurg Psychiatry"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T20:25:26Z","timestamp":1791231926000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e100772"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,5]]},"references-count":106,"URL":"https:\/\/doi.org\/10.2196\/100772","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,5]]},"article-number":"v5i9e100772"}}