{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T16:57:57Z","timestamp":1783702677295,"version":"3.55.0"},"reference-count":44,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T00:00:00Z","timestamp":1755820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Depression is a growing concern gaining attention in both public discourse and AI research. While deep neural networks (DNNs) have been used for its recognition, they still lack real-world effectiveness. Large language models (LLMs) show strong potential but require domain-specific fine-tuning and struggle with non-textual cues. Since depression is often expressed through vocal tone and behavior rather than explicit text, relying on language alone is insufficient. Diagnostic accuracy also suffers without incorporating psychological expertise. To address these limitations, we present, to the best of our knowledge, the first application of LLMs to multimodal depression detection using the DAIC-WOZ dataset. We extract the audio features using the pre-trained model Wav2Vec, and map them to text-based LLMs for further processing. We also propose a novel strategy for incorporating psychological knowledge into LLMs to enhance diagnostic performance, specifically using a question and answer set to grant authorized knowledge to LLMs. Our approach yields a notable improvement in both Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) compared to a base score proposed by the related original paper. The codes are available in <jats:ext-link>Github<\/jats:ext-link>.<\/jats:p>","DOI":"10.3389\/fcomp.2025.1629725","type":"journal-article","created":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T08:49:08Z","timestamp":1755852548000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Large language models for depression recognition in spoken language integrating psychological knowledge"],"prefix":"10.3389","volume":"7","author":[{"given":"Yupei","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuaijie","family":"Shao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Milling","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bj\u00f6rn W.","family":"Schuller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,8,22]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-4920871\/v1","article-title":"Psycholex: Unveiling the psychological mind of large language models","author":"Abbasi","year":"2024","journal-title":"arXiv"},{"key":"B2","doi-asserted-by":"publisher","first-page":"45","DOI":"10.48308\/CMCMA.2.1.45","article-title":"Automated depression recognition using multimodal machine learning: a study on the daic-woz dataset","volume":"2","author":"Afzal Aghaei","year":"2023","journal-title":"Comp. Mathem. Comp. Model. Appl"},{"key":"B3","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1109\/MIS.2023.3254179","article-title":"Will affective computing emerge from foundation models and general artificial intelligence? A first evaluation of chatGPT","volume":"38","author":"Amin","year":"2023","journal-title":"IEEE Intellig. Syst"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1677","DOI":"10.11591\/ijeecs.v19.i3.pp1677-1684","article-title":"On the review of image and video-based depression detection using machine learning","volume":"19","author":"Ashraf","year":"2020","journal-title":"Indon J. Elect. Eng. Comp. Sci"},{"key":"B5","doi-asserted-by":"publisher","first-page":"12449","DOI":"10.48550\/arXiv.2006.11477","article-title":"wav2vec 2.0: A framework for self-supervised learning of speech representations","volume":"33","author":"Baevski","year":"2020","journal-title":"Adv Neural Inf Process Syst"},{"key":"B6","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1038\/s41398-023-02594-0","article-title":"Relative importance of speech and voice features in the classification of schizophrenia and depression","volume":"13","author":"Berardi","year":"2023","journal-title":"Transl Psychiatry"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2107.03374","article-title":"Evaluating large language models trained on code","author":"Chen","year":"2021","journal-title":"arXiv"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1007\/s10579-024-09720-4","article-title":"Depression symptoms modelling from social media text: an LLM driven semi-supervised learning approach","volume":"58","author":"Farruque","year":"2024","journal-title":"Lang. Resources Eval"},{"key":"B9","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/BF02596483","article-title":"Recognition of depression by internists in primary care: a comparison of internist and \u201cgold standard\u201d psychiatric assessments","volume":"4","author":"Gerber","year":"1989","journal-title":"J. Gen. Intern. Med"},{"key":"B10","first-page":"3123","article-title":"\u201cThe distress analysis interview corpus of human and computer interviews,\u201d","author":"Gratch","year":"2014","journal-title":"Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC\u201814)"},{"key":"B11","first-page":"4679","article-title":"\u201cDepression diagnosis and forecast based on mobile phone sensor data,\u201d","author":"He","year":"2022","journal-title":"2022 44th Annual International Conference of the IEEE Engineering in Medicine"},{"key":"B12","first-page":"3","article-title":"Lora: Low-rank adaptation of large language models","volume":"1","author":"Hu","year":"2022","journal-title":"ICLR"},{"key":"B13","doi-asserted-by":"publisher","first-page":"12734","DOI":"10.1038\/s41598-024-63556-0","article-title":"Depression recognition using voice-based pre-training model","volume":"14","author":"Huang","year":"2024","journal-title":"Sci Rep"},{"key":"B14","doi-asserted-by":"publisher","first-page":"964582","DOI":"10.3389\/fdgth.2022.964582","article-title":"Personalised depression forecasting using mobile sensor data and ecological momentary assessment","volume":"4","author":"Kathan","year":"","journal-title":"Front. Digital Health"},{"key":"B15","first-page":"2627","author":"Kathan","year":""},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.10750","article-title":"Depression detection on social media with large language models","author":"Lan","year":"2024","journal-title":"arXiv"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2503.21419","article-title":"Neuroplasticity in artificial intelligence-an overview and inspirations on drop in & out learning","author":"Li","year":"2025","journal-title":"arXiv"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2412.19437","article-title":"Deepseek-v3 technical report","author":"Liu","year":"2024","journal-title":"arXiv"},{"key":"B19","doi-asserted-by":"crossref","first-page":"5487","DOI":"10.1145\/3637528.3671552","article-title":"\u201cEmollms: A series of emotional large language models and annotation tools for comprehensive affective analysis,\u201d","volume-title":"Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Liu","year":"2024"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1121583","DOI":"10.3389\/fpsyt.2023.1121583","article-title":"Detecting depression of chinese microblog users via text analysis: Combining linguistic inquiry word count (liwc) with culture and suicide related lexicons","volume":"14","author":"Lyu","year":"2023","journal-title":"Front. Psychiat"},{"key":"B21","doi-asserted-by":"publisher","first-page":"31","DOI":"10.21437\/IberSPEECH.2024-7","article-title":"\u201cMulti-triplet loss-based models for categorical depression recognition from speech,\u201d","volume":"2024","author":"Mallol-Ragolta","year":"2024","journal-title":"Proc. IberSPEECH"},{"key":"B22","first-page":"182","article-title":"\u201cKnowledge injection to counter large language model (LLM) hallucination,\u201d","volume-title":"European Semantic Web Conference","author":"Martino","year":"2023"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1057\/s41599-023-02313-6","article-title":"Detecting depression on video logs using audiovisual features","volume":"10","author":"Min","year":"2023","journal-title":"Humanit. Soc. Sci. Commun"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1954","DOI":"10.1109\/TAFFC.2022.3177737","article-title":"Dual attention and element recalibration networks for automatic depression level prediction","volume":"14","author":"Niu","year":"2022","journal-title":"IEEE Trans. Affect. Comp"},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139173469","volume-title":"Learning How to Learn","author":"Novak","year":"1984"},{"key":"B26","doi-asserted-by":"publisher","first-page":"112","DOI":"10.3390\/bdcc8090112","article-title":"Sentiment informed sentence bert-ensemble algorithm for depression detection","volume":"8","author":"Ogunleye","year":"2024","journal-title":"Big Data Cognit. Com"},{"key":"B27","doi-asserted-by":"publisher","first-page":"100677","DOI":"10.1016\/j.jadr.2023.100677","article-title":"A mixed-methods feasibility study of a novel ai-enabled, web-based, clinical decision support system for the treatment of major depression in adults","volume":"14","author":"Qassim","year":"2023","journal-title":"J. Affect. Disord. Reports"},{"key":"B28","first-page":"28492","article-title":"\u201cRobust speech recognition via large-scale weak supervision,\u201d","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2023"},{"key":"B29","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/3347320.3357688","article-title":"\u201cAVEC 2019 workshop and challenge: state-of-mind, detecting depression with AI, and cross-cultural affect recognition,\u201d","author":"Ringeval","year":"2019","journal-title":"Proceedings of the 9th International on Audio\/visual Emotion Challenge and Workshop"},{"key":"B30","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/3133944.3133953","article-title":"\u201cAVEC 2017: real-life depression, and affect recognition workshop and challenge,\u201d","volume-title":"Proceedings of the 7th Annual Workshop on Audio\/Visual Emotion Challenge, AVEC '17","author":"Ringeval","year":"2017"},{"key":"B31","doi-asserted-by":"publisher","first-page":"e0000237","DOI":"10.1371\/journal.pdig.0000237","article-title":"Diverse patients' attitudes towards artificial intelligence (ai) in diagnosis","volume":"2","author":"Robertson","year":"2023","journal-title":"PLOS Digital Health"},{"key":"B32","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1038\/s44184-024-00112-8","article-title":"Harnessing multimodal approaches for depression detection using large language models and facial expressions","volume":"3","author":"Sadeghi","year":"2024","journal-title":"NPJ Mental Health Res"},{"key":"B33","doi-asserted-by":"publisher","first-page":"116076","DOI":"10.1016\/j.eswa.2021.116076","article-title":"Audio based depression detection using convolutional autoencoder","volume":"189","author":"Sardari","year":"2022","journal-title":"Expert Syst Appl"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.08907","article-title":"Affective computing has changed: the foundation model disruption","author":"Schuller","year":"2024","journal-title":"arXiv"},{"key":"B35","doi-asserted-by":"publisher","first-page":"e54617","DOI":"10.2196\/54617","article-title":"Using large language models to detect depression from user-generated diary text data as a novel approach in digital mental health screening: Instrument validation study","volume":"26","author":"Shin","year":"2024","journal-title":"J. Med. Internet Res"},{"key":"B36","first-page":"212","article-title":"\u201cscubeMSEC@LT-EDI-ACL2022: Detection of depression using transformer models,\u201d","author":"Sivamanikandan","year":"2022","journal-title":"Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion"},{"key":"B37","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s40708-023-00188-6","article-title":"Deep learning and machine learning in psychiatry: a survey of current progress in depression detection, diagnosis and treatment","volume":"10","author":"Squires","year":"2023","journal-title":"Brain Inform"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.13289","article-title":"Salmonn: towards generic hearing abilities for large language models","author":"Tang","year":"2023","journal-title":"arXiv"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2407.06125","article-title":"Depression detection and analysis using large language models on textual and audio-visual modalities","author":"Tank","year":"2024","journal-title":"arXiv"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.13971","article-title":"Llama: Open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"arXiv"},{"key":"B41","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/2988257.2988258","article-title":"\u201cAVEC 2016: Depression, mood, and emotion recognition workshop and challenge,\u201d","author":"Valstar","year":"2016","journal-title":"Proceedings of the 6th International Workshop on Audio\/Visual Emotion Challenge"},{"key":"B42","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/2512530.2512533","article-title":"\u201cAVEC 2013: the continuous audio\/visual emotion and depression recognition challenge,\u201d","volume-title":"Proceedings of the 3rd ACM International Workshop on Audio\/Visual Emotion Challenge","author":"Valstar","year":"2013"},{"key":"B43","doi-asserted-by":"publisher","first-page":"56472","DOI":"10.7759\/cureus.56472","article-title":"The role of artificial intelligence in identifying depression and anxiety: a comprehensive literature review","volume":"16","author":"Zafar","year":"2024","journal-title":"Cureus"},{"key":"B44","doi-asserted-by":"publisher","first-page":"6690","DOI":"10.1109\/TCSS.2024.3396345","article-title":"Refashioning emotion recognition modelling: the advent of generalised large models","volume":"11","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Comp. Soc. Syst"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2025.1629725\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T08:49:12Z","timestamp":1755852552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2025.1629725\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,22]]},"references-count":44,"alternative-id":["10.3389\/fcomp.2025.1629725"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2025.1629725","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,22]]},"article-number":"1629725"}}