{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T13:49:54Z","timestamp":1782136194931,"version":"3.54.5"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T00:00:00Z","timestamp":1782086400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"National Science Foundation","award":["2335967"],"award-info":[{"award-number":["2335967"]}]},{"name":"EAGER: Knowledge-guided neurosymbolic AI with guardrails for safe virtual health assistants () [\u2013]."}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2026,7,30]]},"abstract":"<jats:p>Large language models (LLMs) are increasingly attracting the attention of healthcare professionals for their potential to assist in diagnostic assessments, which could alleviate the strain on the healthcare system caused by a high patient load and a shortage of providers. For LLMs to be effective in supporting diagnostic assessments, it is essential that they closely replicate the standard diagnostic procedures used by clinicians. In this paper, we specifically examine the diagnostic assessment processes described in the Patient Health Questionnaire-9 (PHQ-9) for major depressive disorder (MDD) and the Generalized Anxiety Disorder-7 (GAD-7) questionnaire for generalized anxiety disorder (GAD). We investigate various prompting and fine-tuning techniques to guide both proprietary and open source LLMs in adhering to these processes, and we evaluate the agreement between LLM-generated diagnostic outcomes and expert-validated ground truth. For fine-tuning, we utilize the MentaLLaMa and Llama models, while for prompting, we experiment with proprietary models like GPT-3.5 and GPT-4o, as well as open source models such as llama-3.1-8b and mixtral-8x7b.<\/jats:p>\n                  <jats:p>\n                    <jats:italic toggle=\"yes\">Software Availability<\/jats:italic>\n                    . We make all software artifacts available at this GitHub link (\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/kauroy1994\/Large-Language-Models-for-Assisting-with-Mental-Health-Diagnostic-Assessments\">https:\/\/github.com\/kauroy1994\/Large-Language-Models-for-Assisting-with-Mental-Health-Diagnostic-Assessments<\/jats:ext-link>\n                    ).\n                  <\/jats:p>\n                  <jats:p>\n                    <jats:italic toggle=\"yes\">Institutional Review Board (IRB)<\/jats:italic>\n                    . This study does not require approval from the IRB. It involves using clinician-annotated social media posts, authorized for research purposes. The primary objective is to evaluate the effectiveness of LLMs that incorporate diagnostic criteria for major depressive disorder and general anxiety disorder for assisting with mental health assessments.\n                  <\/jats:p>","DOI":"10.1145\/3805697","type":"journal-article","created":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T14:37:43Z","timestamp":1776436663000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring the Potential of Large Language Models for Assisting with\u00a0Mental Health Diagnostic Assessments"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6610-7845","authenticated-orcid":false,"given":"Kaushik","family":"Roy","sequence":"first","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9379-1087","authenticated-orcid":false,"given":"Harshul","family":"Surana","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0964-4185","authenticated-orcid":false,"given":"Darssan","family":"Eswaramoorthi","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9694-1359","authenticated-orcid":false,"given":"Yuxin","family":"Zi","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5202-8140","authenticated-orcid":false,"given":"Vedant","family":"Palit","sequence":"additional","affiliation":[{"name":"Indian Institute of Technology, Kharagpur, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-7472-4690","authenticated-orcid":false,"given":"Ritvik","family":"Garimella","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0021-5293","authenticated-orcid":false,"given":"Amit","family":"Sheth","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,22]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","unstructured":"Nitin Rane. 2023. ChatGPT and similar generative artificial intelligence (AI) for building and construction industry: Contribution opportunities and challenges of large language models for industry 4.0 industry 5.0 and society 5.0. DOI: 10.2139\/ssrn.4603221","DOI":"10.2139\/ssrn.4603221"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsr2214184"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1177\/1049732320924625"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2019.01713"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/3600270.3602281"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3495883"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.5555\/3600270.3602070"},{"key":"e_1_3_2_10_2","first-page":"53728","volume-title":"Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923)","author":"Rafailov Rafael","year":"2024","unstructured":"Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. In Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923), 53728\u201353741."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3648137"},{"key":"e_1_3_2_12_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_2_13_2","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Antoine Roux Arthur Mensch Blanche Savary Chris Bamford Devendra Singh Chaplot Diego de las Casas Emma Bou Hanna Florian Bressand et al. 2024. Mixtral of experts. arXiv:2401.04088. Retrieved from https:\/\/arxiv.org\/abs\/2401.04088"},{"key":"e_1_3_2_14_2","unstructured":"Anxo P\u00e9rez Marcos Fern\u00e1ndez-Pichel Javier Parapar and David E. Losada. 2023. DepreSym: A depression symptom annotated corpus and the role of LLMs as assessors of psychological markers. arXiv:2308.10758. Retrieved from https:\/\/arxiv.org\/abs\/2308.10758"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.clpsych-1.3"},{"key":"e_1_3_2_16_2","volume-title":"Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych \u201924)","author":"Yates Andrew","year":"2024","unstructured":"Andrew Yates, Bart Desmet, Emily Prud\u2019Hommeaux, Ayah Zirikly, Steven Bedrick, Sean MacAvaney, Kfir Bar, Molly Ireland, and Yaakov Ophir. 2024. Proceedings of the 9th Workshop on Computational Linguistics and Clinical Psychology (CLPsych \u201924). Association for Computational Linguistics."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.clpsych-1.12"},{"key":"e_1_3_2_18_2","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier et al. 2023. Mistral 7b. arXiv:2310.06825. Retrieved from https:\/\/arxiv.org\/abs\/2310.06825"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"Sumit Dalal Deepa Tilwani Manas Gaur Sarika Jain Valerie Shalin and Amit Seth. 2023. A cross attention approach to diagnostic explainability using clinical practice guidelines for depression. arXiv:2311.13852. Retrieved from https:\/\/arxiv.org\/abs\/2311.13852","DOI":"10.36227\/techrxiv.170723261.14729994\/v1"},{"key":"e_1_3_2_20_2","unstructured":"Jacob Devlin. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.63317\/2d7umxifj8wz"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaaiss.v1i1.27494"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.clpsych-1.16"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2022.1056728"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2021.3101919"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2022.3182349"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2023.3268724"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2023.3344353"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2024.3444928"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2024.3441128"}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3805697","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T13:10:11Z","timestamp":1782133811000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3805697"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,22]]},"references-count":29,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,7,30]]}},"alternative-id":["10.1145\/3805697"],"URL":"https:\/\/doi.org\/10.1145\/3805697","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"value":"2691-1957","type":"print"},{"value":"2637-8051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,22]]},"assertion":[{"value":"2024-03-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}