{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T20:38:44Z","timestamp":1792183124746,"version":"4.4.0"},"reference-count":20,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T00:00:00Z","timestamp":1751760000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T00:00:00Z","timestamp":1751760000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003945","name":"Link\u00f6ping University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003945","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>The substantial increase in mental health disorders globally necessitates scalable, accurate tools for detecting and classifying these conditions in digital environments. This study addresses the critical challenge of automated mental health classification by comparing three distinct computational approaches: (1) Traditional Natural Language Processing (NLP) with advanced feature engineering, (2) Prompt-engineered large language models (LLMs), and (3) Fine-tuned LLMs. The dataset consisted of over 51,000 publicly available text statements from social media platforms, tagged with seven mental health conditions: Normal, Depression, Suicidal, Anxiety, Stress, Bipolar Disorder, and Personality Disorder. The dataset was stratified into training, validation, and test sets for model evaluation. The primary outcome was classification accuracy across these seven mental health conditions. Additional metrics like precision, recall, and F1-score were analyzed. We compared the results of the three computational approaches and overfitting was monitored through validation loss across epochs for the fine-tuned LLM. The NLP model with advanced feature engineering achieved an overall accuracy of 95%, surpassing both the prompt-engineered LLM (65%) and the fine-tuned LLM (91%). This model performed exceptionally well in terms of accuracy and precision. While fine-tuning for three epochs yielded optimal results, further training led to overfitting and decreased performance. This study demonstrates the significant benefits of applying advanced text preprocessing and feature engineering techniques to traditional NLP models, alongside fine-tuning LLMs, such as GPT-4o-mini, for mental health classification tasks. The results clearly indicate that off-the-shelf LLM chatbots using prompt engineering are inadequate for mental health classification, performing 30% points below specialized NLP approaches. Despite the popularity of general-purpose LLMs, specialized approaches remain superior for critical healthcare applications like mental health classification.<\/jats:p>","DOI":"10.1038\/s41598-025-08031-0","type":"journal-article","created":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T06:41:59Z","timestamp":1751784119000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["Comparing traditional natural language processing and large language models for mental health status classification: a multi-model evaluation"],"prefix":"10.1038","volume":"15","author":[{"given":"Thomas","family":"Kallstenius","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea Johansson","family":"Capusan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gerhard","family":"Andersson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adam","family":"Williamson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,6]]},"reference":[{"issue":"9904","key":"8031_CR1","doi-asserted-by":"publisher","first-page":"1575","DOI":"10.1016\/S0140-6736(13)61611-6","volume":"382","author":"HA Whiteford","year":"2013","unstructured":"Whiteford, H. A. et al. Global burden of disease attributable to mental and substance use disorders: findings from the global burden of disease study 2010. Lancet. 382 (9904), 1575\u20131586 (2013).","journal-title":"Lancet"},{"issue":"2","key":"8031_CR2","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/S2215-0366(15)00505-2","volume":"3","author":"D Vigo","year":"2016","unstructured":"Vigo, D., Thornicroft, G. & Atun, R. Estimating the true global burden of mental illness. Lancet Psychiatry. 3 (2), 171\u2013178 (2016).","journal-title":"Lancet Psychiatry"},{"issue":"1","key":"8031_CR3","doi-asserted-by":"publisher","first-page":"11846","DOI":"10.1038\/s41598-020-68764-y","volume":"10","author":"J Kim","year":"2020","unstructured":"Kim, J., Lee, J., Park, E. & Han, J. A deep learning model for detecting mental illness from user content on social media. Sci. Rep. 10 (1), 11846 (2020).","journal-title":"Sci. Rep."},{"key":"8031_CR4","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.cobeha.2017.07.005","volume":"18","author":"SC Guntuku","year":"2017","unstructured":"Guntuku, S. C., Yaden, D. B., Kern, M. L., Ungar, L. H. & Eichstaedt, J. C. Detecting depression and mental illness on social media: an integrative review. Curr. Opin. Behav. Sci. 18, 43\u201349 (2017).","journal-title":"Curr. Opin. Behav. Sci."},{"key":"8031_CR5","doi-asserted-by":"publisher","unstructured":"Abd-alrazaq, A. et al. The performance of artificial intelligence-driven technologies in diagnosing mental disorders: an umbrella review. NPJ Digit. Med. 5(1), 87. https:\/\/doi.org\/10.1038\/s41746-022-00631-8 (2022).","DOI":"10.1038\/s41746-022-00631-8"},{"key":"8031_CR6","doi-asserted-by":"publisher","first-page":"e51712","DOI":"10.2196\/51712","volume":"20","author":"H Chin","year":"2023","unstructured":"Chin, H. et al. The potential of chatbots for emotional support and promoting mental well-being in different cultures: mixed methods study. J Med. Internet Res. 25, e51712. https:\/\/doi.org\/10.2196\/51712 (2023).","journal-title":"J Med. Internet Res."},{"issue":"1","key":"8031_CR7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-016-0028-x","volume":"7","author":"G Gkotsis","year":"2017","unstructured":"Gkotsis, G. et al. Characterisation of mental health conditions in social media using informed deep learning. Sci. Rep. 7 (1), 1\u201311 (2017).","journal-title":"Sci. Rep."},{"issue":"3","key":"8031_CR8","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1136\/jamia.2009.002733","volume":"17","author":"AR Aronson","year":"2010","unstructured":"Aronson, A. R. & Lang, F-M. An overview of metamap: historical perspective and recent advances. J. Am. Med. Inf. Assoc. 17 (3), 229\u2013236 (2010).","journal-title":"J. Am. Med. Inf. Assoc."},{"issue":"5","key":"8031_CR9","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1136\/jamia.2009.001560","volume":"17","author":"GK Savova","year":"2010","unstructured":"Savova, G. K. et al. Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications. J. Am. Med. Inf. Assoc. 17 (5), 507\u2013513 (2010).","journal-title":"J. Am. Med. Inf. Assoc."},{"key":"8031_CR10","doi-asserted-by":"crossref","unstructured":"Dalal, S. et al. A cross attention approach to diagnostic explainability using clinical practice guidelines for depression. IEEE J. Biomed. Health Inform. (2024).","DOI":"10.36227\/techrxiv.170723261.14729994\/v1"},{"key":"8031_CR11","unstructured":"Dalal, S., Jain, S. & Dave, M. Deep knowledge-infusion for explainable depression detection. arXiv preprint arXiv:240902122 (2024)."},{"issue":"4","key":"8031_CR12","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee, J. et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 36 (4), 1234\u20131240 (2020).","journal-title":"Bioinformatics"},{"key":"8031_CR13","doi-asserted-by":"crossref","unstructured":"Alsentzer, E. et al. Publicly available clinical BERT embeddings. arXiv preprint arXiv:190403323 (2019).","DOI":"10.18653\/v1\/W19-1909"},{"key":"8031_CR14","doi-asserted-by":"crossref","unstructured":"Howard, J. & Ruder, S. Universal language model fine-tuning for text classification. arXiv preprint arXiv:180106146 (2018).","DOI":"10.18653\/v1\/P18-1031"},{"key":"8031_CR15","unstructured":"PowersDM Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. ArXiv Preprint. arXiv:201016061 (2020)."},{"issue":"8","key":"8031_CR16","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1037\/abn0000850","volume":"132","author":"EC Stade","year":"2023","unstructured":"Stade, E. C., Ungar, L., Eichstaedt, J. C., Sherman, G. & Ruscio, A. M. Depression and anxiety have distinct and overlapping language patterns: results from a clinical interview. J. Psychopathol. Clin. Sci. 132 (8), 972\u2013983. https:\/\/doi.org\/10.1037\/abn0000850 (2023).","journal-title":"J. Psychopathol. Clin. Sci."},{"issue":"12","key":"8031_CR17","doi-asserted-by":"publisher","first-page":"2605","DOI":"10.1017\/S0033291716001215","volume":"46","author":"SK Fineberg","year":"2016","unstructured":"Fineberg, S. K. et al. Self-reference in psychosis and depression: a language marker of illness. Psychol. Med. 46 (12), 2605\u20132615. https:\/\/doi.org\/10.1017\/S0033291716001215 (2016).","journal-title":"Psychol. Med."},{"issue":"3","key":"8031_CR18","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1145\/3446776","volume":"64","author":"C Zhang","year":"2021","unstructured":"Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. Understanding deep learning (still) requires rethinking generalization. Commun. ACM. 64 (3), 107\u2013115 (2021).","journal-title":"Commun. ACM"},{"issue":"4","key":"8031_CR19","doi-asserted-by":"publisher","first-page":"null","DOI":"10.1136\/eb-2015-102228","volume":"18","author":"T Yemisi","year":"2015","unstructured":"Yemisi, T., Richard, D. R. & Jonathan, J. D. Meta-analysis of diagnostic accuracy studies in mental health. Evid. Based Ment. Health. 18 (4), null. https:\/\/doi.org\/10.1136\/eb-2015-102228 (2015).","journal-title":"Evid. Based Ment. Health"},{"issue":"4","key":"8031_CR20","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1176\/appi.ps.202100692","volume":"74","author":"JS Tauscher","year":"2023","unstructured":"Tauscher, J. S. et al. Automated detection of cognitive distortions in text exchanges between clinicians and people with serious mental illness. Psychiatr. Serv. 74 (4), 407\u2013410 (2023).","journal-title":"Psychiatr. Serv."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-08031-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-08031-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-08031-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T06:42:05Z","timestamp":1751784125000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-08031-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,6]]},"references-count":20,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["8031"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-08031-0","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,6]]},"assertion":[{"value":"16 February 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"24102"}}