{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T15:38:20Z","timestamp":1782229100063,"version":"3.54.5"},"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T00:00:00Z","timestamp":1777248000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006390","name":"University of Lausanne","doi-asserted-by":"crossref","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100006390","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>Psychosocial risk assessment is a cornerstone of mental health care, yet remains resource-intensive and inconsistently delivered across domains such as suicide, intimate partner violence (IPV), and substance misuse. Recent advances in large language models (LLMs) raise the possibility of scalable, conversational agents capable of detecting and evaluating psychosocial risk. Across three interlinked studies, we evaluated the performance of LLMs in this context. Study 1 benchmarked GPT-4 and Claude 3 Sonnet against vignettes constructed from participants\u2019 lived-experience, finding high accuracy in detecting risk domains and substantial agreement with participant-rated severity, though suicidality proved more challenging than IPV or substance misuse. Study 2 examined participants\u2019 perceptions of LLM-generated responses, revealing that most judged them accurate, empathic, and clinically useful, with no differences across models or domains. Study 3 implemented a supervised, three-agent GPT-4o-based chatbot system integrating one chatbot as a therapeutic agent, a supervisor for risk detection, and a JSON-based assessor for structured evaluation. The therapeutic agent chatbot was successfully completed full risk assessments most of the time while maintaining therapeutic quality. Together, these studies suggest that LLMs can contribute to psychosocial risk detection and structured assessment under controlled conditions, while underscoring the need for careful supervision, rigorous validation, and clearly defined boundaries before consideration of real-world clinical deployment.<\/jats:p>","DOI":"10.1371\/journal.pdig.0001352","type":"journal-article","created":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T17:45:07Z","timestamp":1777311907000},"page":"e0001352","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse"],"prefix":"10.1371","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5594-2095","authenticated-orcid":true,"given":"Laura M.","family":"Vowels","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pranika","family":"Vohra","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Danyang","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pegah","family":"Zeinoddin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alex","family":"Elswick","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tiffany","family":"Marcantonio","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathan D.","family":"Wood","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew J.","family":"Vowels","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,4,27]]},"reference":[{"issue":"2","key":"pdig.0001352.ref001","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1111\/acps.13306","volume":"144","author":"S Lagan","year":"2021","journal-title":"Acta Psychiatr Scand"},{"key":"pdig.0001352.ref002","unstructured":"Substance Abuse and Mental Health Services Administration. Key Substance Use and Mental Health Indicators in the United States: Results from the 2024 National Survey on Drug Use and Health. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. 2025. https:\/\/www.samhsa.gov\/data\/data-we-collect\/nsduh-national-survey-drug-use-and-health\/national-releases"},{"issue":"1","key":"pdig.0001352.ref003","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1891\/1946-6560.4.1.76","article-title":"Risk Assessment in Intimate Partner Violence: A Systematic Review of Contemporary Approaches","volume":"4","author":"TL Nicholls","year":"2013","journal-title":"Partner Abuse"},{"key":"pdig.0001352.ref004","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.avb.2019.03.008","article-title":"Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools","volume":"47","author":"CE van der Put","year":"2019","journal-title":"Aggression and Violent Behavior"},{"issue":"2","key":"pdig.0001352.ref005","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1192\/bjp.bp.107.040113","article-title":"Cross-national prevalence and risk factors for suicidal ideation, plans and attempts","volume":"192","author":"MK Nock","year":"2008","journal-title":"Br J Psychiatry"},{"key":"pdig.0001352.ref006","unstructured":"Ahuja-Kohli S. EU mental health apps. World Health Expo Insights. cited 2026 February 17. https:\/\/www.worldhealthexpo.com\/insights\/medical-technology\/are-mental-health-and-wellness-apps-really-helping-consumers"},{"issue":"9","key":"pdig.0001352.ref007","doi-asserted-by":"crossref","DOI":"10.2196\/14567","article-title":"Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis","volume":"21","author":"A Baumel","year":"2019","journal-title":"J Med Internet Res"},{"key":"pdig.0001352.ref008","doi-asserted-by":"crossref","DOI":"10.2196\/44548","article-title":"Evaluating Conversational Agents for Mental Health: Scoping Review of Outcomes and Outcome Measurement Instruments","volume":"25","author":"AI Jabir","year":"2023","journal-title":"J Med Internet Res"},{"issue":"6","key":"pdig.0001352.ref009","doi-asserted-by":"crossref","DOI":"10.2196\/18301","article-title":"Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review","volume":"22","author":"A Abd-Alrazaq","year":"2020","journal-title":"J Med Internet Res"},{"key":"pdig.0001352.ref010","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1080\/17434440.2021.2013200","article-title":"Artificially intelligent chatbots in digital mental health interventions: a review","volume":"18","author":"EM Boucher","year":"2021","journal-title":"Expert Rev Med Devices"},{"key":"pdig.0001352.ref011","doi-asserted-by":"crossref","unstructured":"He Y, Yang L, Qian C, Li T, Su Z, Zhang Q. Conversational agent interventions for mental health problems: A systematic review and meta-analysis of randomized controlled trials. 2022. https:\/\/doi.org\/10.2196\/preprints.43862","DOI":"10.2196\/preprints.43862"},{"issue":"3","key":"pdig.0001352.ref012","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1080\/17483107.2022.2146768","article-title":"Dialogue agents for artificial intelligence-based conversational systems for cognitively disabled: a systematic review","volume":"19","author":"SM Huq","year":"2024","journal-title":"Disabil Rehabil Assist Technol"},{"issue":"9","key":"pdig.0001352.ref013","doi-asserted-by":"crossref","first-page":"1248","DOI":"10.1093\/jamia\/ocy072","article-title":"Conversational agents in healthcare: a systematic review","volume":"25","author":"L Laranjo","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"pdig.0001352.ref014","doi-asserted-by":"crossref","first-page":"103978","DOI":"10.1016\/j.ijmedinf.2019.103978","article-title":"An overview of the features of chatbots in mental health: A scoping review","volume":"132","author":"AA Abd-Alrazaq","year":"2019","journal-title":"Int J Med Inform"},{"key":"pdig.0001352.ref015","doi-asserted-by":"crossref","first-page":"1199058","DOI":"10.3389\/fpsyg.2023.1199058","article-title":"ChatGPT outperforms humans in emotional awareness evaluations","volume":"14","author":"Z Elyoseph","year":"2023","journal-title":"Front Psychol"},{"key":"pdig.0001352.ref016","article-title":"We spoke to people who started using ChatGPT as their therapist","author":"E Pirnay","year":"2023","journal-title":"Vice"},{"issue":"6","key":"pdig.0001352.ref017","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum","volume":"183","author":"JW Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"issue":"2","key":"pdig.0001352.ref018","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmen.0000145","article-title":"When ELIZA meets therapists: A Turing test for the heart and mind","volume":"2","author":"SG Hatch","year":"2025","journal-title":"PLOS Ment Health"},{"issue":"2","key":"pdig.0001352.ref019","doi-asserted-by":"crossref","first-page":"100077","DOI":"10.1016\/j.chbah.2024.100077","article-title":"Are chatbots the new relationship experts? Insights from three studies","volume":"2","author":"LM Vowels","year":"2024","journal-title":"Computers in Human Behavior: Artificial Humans"},{"issue":"2","key":"pdig.0001352.ref020","doi-asserted-by":"crossref","DOI":"10.55922\/001c.92367","article-title":"ChatGPT: A Pilot Study on a Promising Tool for Mental Health Support in Psychiatric Inpatient Care","volume":"2","author":"A Melo","year":"2024","journal-title":"International Journal of Psychiatric Trainees"},{"key":"pdig.0001352.ref021","first-page":"137","article-title":"MyListener: An AI-Mediated Journaling Mobile Application for Alleviating Depression and Loneliness Using Contextual Data","author":"G Jung","year":"2024","journal-title":"Companion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing"},{"key":"pdig.0001352.ref022","article-title":"MindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients\u2019 Journaling","author":"T Kim","year":"2024","journal-title":"arXiv"},{"key":"pdig.0001352.ref023","article-title":"AI in relationship counselling: Evaluating ChatGPT\u2019s therapeutic efficacy in providing relationship advice","author":"LM Vowels","year":"2023","journal-title":"PsyArXiv"},{"key":"pdig.0001352.ref024","doi-asserted-by":"crossref","first-page":"100141","DOI":"10.1016\/j.chbah.2025.100141","article-title":"Evaluating the efficacy of Amanda: A voice-based large language model chatbot for relationship challenges","volume":"4","author":"LM Vowels","year":"2025","journal-title":"Computers in Human Behavior: Artificial Humans"},{"issue":"9","key":"pdig.0001352.ref025","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pmen.0000411","article-title":"The efficacy, feasibility, and technical outcomes of a GPT-4o-based chatbot Amanda for relationship support: A randomized controlled trial","volume":"2","author":"LM Vowels","year":"2025","journal-title":"PLOS Ment Health"},{"key":"pdig.0001352.ref026","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1145\/191666.191703","article-title":"Computers are social actors","author":"C Nass","year":"1994","journal-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems"},{"key":"pdig.0001352.ref027","doi-asserted-by":"crossref","first-page":"1213141","DOI":"10.3389\/fpsyt.2023.1213141","article-title":"Beyond human expertise: the promise and limitations of ChatGPT in suicide risk assessment","volume":"14","author":"Z Elyoseph","year":"2023","journal-title":"Front Psychiatry"},{"key":"pdig.0001352.ref028","article-title":"ChatGPT for Suicide Risk Assessment on Social Media: Quantitative Evaluation of Model Performance, Potentials and Limitations","author":"H Ghanadian","year":"2023","journal-title":"arXiv"},{"issue":"4","key":"pdig.0001352.ref029","doi-asserted-by":"crossref","first-page":"354","DOI":"10.3390\/psychiatryint4040032","article-title":"Suicide Risk Assessment and Prevention Tools in the UK: Current Landscape and Future Directions","volume":"4","author":"A Arowosegbe","year":"2023","journal-title":"Psychiatry International"},{"key":"pdig.0001352.ref030","doi-asserted-by":"crossref","first-page":"706933","DOI":"10.3389\/fpsyt.2021.706933","article-title":"Impulsivity as a Risk Factor for Suicide in Bipolar Disorder","volume":"12","author":"P Zakowicz","year":"2021","journal-title":"Front Psychiatry"},{"key":"pdig.0001352.ref031","unstructured":"National Coalition Against Domestic Violence. Statistics. 2023. https:\/\/ncadv.org\/STATISTICS"},{"issue":"4","key":"pdig.0001352.ref032","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1177\/0886260508317180","article-title":"The danger assessment: validation of a lethality risk assessment instrument for intimate partner femicide","volume":"24","author":"JC Campbell","year":"2009","journal-title":"J Interpers Violence"},{"issue":"7","key":"pdig.0001352.ref033","doi-asserted-by":"crossref","first-page":"1537","DOI":"10.1177\/0886260512468250","article-title":"The average predictive validity of intimate partner violence risk assessment instruments","volume":"28","author":"JT Messing","year":"2013","journal-title":"J Interpers Violence"},{"issue":"1","key":"pdig.0001352.ref034","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1111\/jels.12276","article-title":"Comparing Conventional and Machine\u2010Learning Approaches to Risk Assessment in Domestic Abuse Cases","volume":"18","author":"J Grogger","year":"2021","journal-title":"J Empirical Legal Studies"},{"issue":"5","key":"pdig.0001352.ref035","doi-asserted-by":"crossref","DOI":"10.1016\/j.amepre.2015.05.031","article-title":"2010 National and State Costs of Excessive Alcohol Consumption","volume":"49","author":"JJ Sacks","year":"2015","journal-title":"American Journal of Preventive Medicine"},{"issue":"1","key":"pdig.0001352.ref036","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1186\/s12916-019-1461-z","article-title":"Suicide prevention and depression apps\u2019 suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines","volume":"17","author":"L Martinengo","year":"2019","journal-title":"BMC Med"},{"key":"pdig.0001352.ref037","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1016\/j.jad.2022.09.028","article-title":"Evaluation of chatbot-delivered interventions for self-management of depression: Content analysis","volume":"319","author":"L Martinengo","year":"2022","journal-title":"J Affect Disord"},{"issue":"7","key":"pdig.0001352.ref038","doi-asserted-by":"crossref","first-page":"5246","DOI":"10.3390\/ijerph20075246","article-title":"Smartphone Apps for Domestic Violence Prevention: A Systematic Review","volume":"20","author":"M Sumra","year":"2023","journal-title":"Int J Environ Res Public Health"},{"issue":"11","key":"pdig.0001352.ref039","doi-asserted-by":"crossref","DOI":"10.2196\/17156","article-title":"Mobile Apps to Reduce Tobacco, Alcohol, and Illicit Drug Use: Systematic Review of the First Decade","volume":"22","author":"PK Staiger","year":"2020","journal-title":"J Med Internet Res"},{"issue":"4","key":"pdig.0001352.ref040","doi-asserted-by":"crossref","DOI":"10.2196\/11831","article-title":"Smartphone Apps Targeting Alcohol and Illicit Substance Use: Systematic Search in in Commercial App Stores and Critical Content Analysis","volume":"7","author":"B Tofighi","year":"2019","journal-title":"JMIR Mhealth Uhealth"},{"key":"pdig.0001352.ref041","article-title":"Scope, Characteristics, Behavior Change Techniques, and Quality of Conversational Agents for Mental Health and Well-Being: Systematic Assessment of Apps","volume":"25","author":"X Lin","year":"2023","journal-title":"J Med Internet Res"},{"issue":"2","key":"pdig.0001352.ref042","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1001\/jamapsychiatry.2017.3838","article-title":"A Solution-Focused Research Approach to Achieve an Implementable Revolution in Digital Mental Health","volume":"75","author":"DC Mohr","year":"2018","journal-title":"JAMA Psychiatry"},{"key":"pdig.0001352.ref043","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/00224499.2025.2547814","article-title":"Exploring How Large Language Models Understand Sexual Communication in Hypothetical Sexual Situations","author":"LM Vowels","year":"2025","journal-title":"J Sex Res"},{"issue":"5","key":"pdig.0001352.ref044","article-title":"Columbia-suicide severity rating scale (C-SSRS)","volume":"21","author":"J Salvi","year":"2019","journal-title":"Emergency Medicine Practice"},{"key":"pdig.0001352.ref045","article-title":"PsychBench: A comprehensive and professional benchmark for evaluating the performance of LLM-assisted psychiatric clinical practice","author":"S Liu","year":"2025","journal-title":"arXiv"},{"issue":"1","key":"pdig.0001352.ref046","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"TYC Tam","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"1","key":"pdig.0001352.ref047","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1038\/s44277-025-00049-6","article-title":"Mindbench.ai: an actionable platform to evaluate the profile and performance of large language models in a mental healthcare context","volume":"3","author":"B Dwyer","year":"2025","journal-title":"NPP Digit Psychiatry Neurosci"},{"key":"pdig.0001352.ref048","article-title":"VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health (arXiv:2602.05088)","author":"KH Bentley","year":"2026","journal-title":"arXiv"},{"key":"pdig.0001352.ref049","article-title":"The Effect of Artificial Intelligence on Patient-Physician Trust: Cross-Sectional Vignette Study","volume":"26","author":"AGM Zondag","year":"2024","journal-title":"J Med Internet Res"},{"issue":"1","key":"pdig.0001352.ref050","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1186\/s12911-024-02459-6","article-title":"Assessing the research landscape and clinical utility of large language models: a scoping review","volume":"24","author":"Y-J Park","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"pdig.0001352.ref051","article-title":"Ethical and social risks of harm from language models","author":"L Weidinger","year":"2021","journal-title":"ArXiv"},{"issue":"2","key":"pdig.0001352.ref052","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1037\/bul0000084","article-title":"Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research","volume":"143","author":"JC Franklin","year":"2017","journal-title":"Psychol Bull"},{"issue":"1","key":"pdig.0001352.ref053","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1111\/sltb.12395","article-title":"Accuracy of Clinician Predictions of Future Self-Harm: A Systematic Review and Meta-Analysis of Predictive Studies","volume":"49","author":"R Woodford","year":"2019","journal-title":"Suicide Life Threat Behav"},{"issue":"7","key":"pdig.0001352.ref054","first-page":"508","article-title":"HITS: a short domestic violence screening tool for use in a family practice setting","volume":"30","author":"KM Sherin","year":"1998","journal-title":"Fam Med"},{"key":"pdig.0001352.ref055","unstructured":"Richards L. Domestic Abuse, Stalking and Harassment and Honour Based Violence (DASH, 2009) Risk Identification and Assessment and Management Model. 2009. https:\/\/www.buckssafeguarding.org.uk\/childrenpartnership\/wp-content\/uploads\/sites\/2\/2020\/10\/DASH-November-2015.pdf"},{"key":"pdig.0001352.ref056","unstructured":"Babor TF, Higgins-Biddle JC, Saunders JB, Monteiro MG. Department of Mental Health and Substance Dependence. 2001."},{"issue":"4","key":"pdig.0001352.ref057","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1016\/0306-4603(82)90005-3","article-title":"The drug abuse screening test","volume":"7","author":"HA Skinner","year":"1982","journal-title":"Addict Behav"},{"key":"pdig.0001352.ref058","doi-asserted-by":"crossref","first-page":"100183","DOI":"10.1016\/j.chbah.2025.100183","article-title":"Navigating relationships with GenAI chatbots: User attitudes, acceptability, and potential","volume":"5","author":"LM Vowels","year":"2025","journal-title":"Computers in Human Behavior: Artificial Humans"},{"issue":"4","key":"pdig.0001352.ref059","doi-asserted-by":"crossref","DOI":"10.2196\/25847","article-title":"Evidence on Digital Mental Health Interventions for Adolescents and Young People: Systematic Overview","volume":"8","author":"S Lehtimaki","year":"2021","journal-title":"JMIR Ment Health"},{"issue":"1","key":"pdig.0001352.ref060","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1055\/s-0039-1677903","article-title":"Artificial Intelligence in Clinical Decision Support: Challenges for Evaluating AI and Practical Implications","volume":"28","author":"F Magrabi","year":"2019","journal-title":"Yearbook of Medical Informatics"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001352","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T17:45:18Z","timestamp":1777311918000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001352"}},"subtitle":[],"editor":[{"given":"Xiaoxia","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,27]]},"references-count":60,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,27]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001352","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,27]]}}}