{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T22:37:47Z","timestamp":1792708667143,"version":"4.5.0"},"reference-count":31,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"\n Background<\/jats:title>\n Large language models (LLMs) show promise for mental health counseling applications. However, their deployment faces significant challenges, including data privacy concerns requiring on-premise solutions, high operational costs, and Western-centric biases that limit cultural appropriateness for non\u2013English-speaking populations.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and evaluate culturally competent small language models (SLMs) with approximately 7 to 8 billion parameters for Korean mental health counseling and to validate a 2-stage hybrid evaluation framework combining LLM-as-a-Judge screening with expert clinical validation.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We constructed a dataset of 1558 diary-based counseling interactions from a Korean mental health mobile app (Maeumpyeonuijeom). Three Korean-optimized base models (SKT A.X 4.0 Light {7B}, LG EXAONE 3.5 {7.8B}, and Meta Llama 3.1 {8B}) were fine-tuned using an aggressive fine-tuning strategy with high-rank low-rank adaptation (r=256) over 20 epochs. Inference parameters were optimized using GPT-5 (gpt-5-2025-08-07) as an LLM-as-a-Judge. The optimized SLMs and GPT-4o (gpt-4o-2024-08-06) were evaluated by 10 board-certified psychiatrists (mean clinical experience of approximately 10 years) on 50 counseling scenarios across 4 dimensions (individualization, supportiveness, cultural appropriateness, and safety) using 5-point Likert scales. Wilcoxon signed-rank tests with Holm correction were used for statistical comparison, and intraclass correlation coefficients (ICCs) assessed interrater reliability.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n No statistically significant difference from GPT-4o was detected for SKT A.X 4.0 Light in supportiveness (mean 3.39, SD 0.74 vs mean 3.45, SD 0.90; P=.26) and cultural appropriateness (mean 3.30, SD 0.77 vs mean 3.29, SD 0.79; P=.58). SKT A.X 4.0 Light ranked first in cultural appropriateness among all models. Interrater reliability analysis revealed higher agreement among psychiatrists for local SLMs (ICC=0.575 for SKT A.X 4.0 Light in cultural appropriateness) compared with GPT-4o (ICC=0.211). GPT-4o maintained significantly higher safety scores than all SLMs (P<.001). The LLM-as-a-Judge demonstrated moderate-to-good correlation with human evaluation but exhibited a ceiling effect in safety scoring.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n A 7 to 8-billion-parameter SLM fine-tuned on culturally specific data can achieve performance comparable to GPT-4o in supportiveness and cultural appropriateness for mental health counseling. These findings support the feasibility of locally deployable, privacy-preserving mental health AI systems based on culturally adapted SLMs. However, the safety gap necessitates human expert oversight, and success depends on the availability of high-quality language-specific base models.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/92470","type":"journal-article","created":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T16:55:09Z","timestamp":1789664109000},"page":"e92470","source":"Crossref","is-referenced-by-count":0,"title":["Optimizing Small Local Language Models for Culturally Competent Mental Health Counseling: Comparative Evaluation With GPT-4o by Psychiatrists"],"prefix":"10.2196","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6116-598X","authenticated-orcid":false,"given":"Jaewook","family":"Han","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2522-4329","authenticated-orcid":false,"given":"Beomgi","family":"So","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0205-0637","authenticated-orcid":false,"given":"Whanbo","family":"Jung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9405-6069","authenticated-orcid":false,"given":"Minwoo","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8982-6314","authenticated-orcid":false,"given":"Hoon","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0953-6245","authenticated-orcid":false,"given":"Daun","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,22]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01611-4"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.2196\/59479"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.13614"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-018-0272-7"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.25122\/jml-2021-0100"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00700-y"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae346"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.04088"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1145\/3768165"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.09685"},{"key":"ref13","doi-asserted-by":"crossref","first-page":"3405","DOI":"10.18653\/v1\/2021.emnlp-main.274","author":"Kim, B","year":"2021","journal-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.2196\/54617"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1017\/s1351324922000213"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.06.029"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.2196\/78082"},{"key":"ref18","first-page":"10088","author":"Dettmers, T","year":"2023","journal-title":"NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems"},{"key":"ref19","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.18653\/v1\/2024.acl-long.77","author":"Huang, J","year":"2024","journal-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.10313"},{"key":"ref21","first-page":"55006","author":"Zhou, C","year":"2023","journal-title":"NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1904.09751"},{"key":"ref23","first-page":"46595","author":"Zheng, L","year":"2023","journal-title":"NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems"},{"key":"ref24","doi-asserted-by":"crossref","first-page":"16366","DOI":"10.18653\/v1\/2024.acl-long.862","author":"Naous, T","year":"2024","journal-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2018.5630"},{"key":"ref26","author":"Sue, DW","year":"1999","journal-title":"Counseling the Culturally Different: Theory and Practice"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1001\/jamapsychiatry.2026.0032"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2112.04359"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.2196\/57400"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.08774"},{"key":"ref31","doi-asserted-by":"crossref","first-page":"6282","DOI":"10.18653\/v1\/2020.acl-main.560","author":"Joshi, P","year":"2020","journal-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e92470","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T21:55:09Z","timestamp":1792706109000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2026\/1\/e92470"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,22]]},"references-count":31,"URL":"https:\/\/doi.org\/10.2196\/92470","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.92470","asserted-by":"object"}]},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,22]]}}}