{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T12:13:19Z","timestamp":1770984799172,"version":"3.50.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T00:00:00Z","timestamp":1770940800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T00:00:00Z","timestamp":1770940800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007065","name":"Universit\u00e0 degli Studi di Salerno","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007065","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    The assessment of disorganized thought and language is essential for diagnosing schizophrenia (SZ), still remaining a subjective and labor-intensive clinical process. We present\n                    <jats:italic>ClearThought<\/jats:italic>\n                    , an automated framework that uses large language models (LLMs) to evaluate the Thought and Language Disorder (TALD) scale through few-shot prompting on transcribed psychiatric interviews. The model generates 0\u20134 item-level severity scores and structured justifications aligned with the TALD rubric. We evaluated ClearThought on a dataset of 33 SZ patient interviews, comparing model outputs to clinician ratings using both ordinal and binary performance metrics. For ordinal scoring, the system achieved macro F1\n                    <jats:inline-formula>\n                      <jats:tex-math>$$\\ge$$<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    0.80 on 11 items and very strong Spearman correlations (\n                    <jats:inline-formula>\n                      <jats:tex-math>$$\\varvec{\\rho \\ge 0.80}$$<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    ) on 15 items, particularly for disorders that show clear and noticeable patterns in spoken language, such as\n                    <jats:italic>Blocking<\/jats:italic>\n                    and\n                    <jats:italic>Restricted Thinking<\/jats:italic>\n                    . Entropy-aware analysis revealed that high performance was most meaningful when accompanied by sufficient label variability. For binary detection, the model accurately identified disorder presence in 26 of 30 items (F1 &gt; 0.80), with over 11 items exceeding F1 = 0.90. However, high scores on low-entropy items (e.g.,\n                    <jats:italic>Clanging<\/jats:italic>\n                    ,\n                    <jats:italic>Neologisms<\/jats:italic>\n                    ) were often driven by consistent disorder absence, highlighting the need for caution when interpreting results from skewed label distributions. Clinicians rated the model\u2019s justifications as clinically sound and interpretable, with average scores above 4.0\/5.0 for scoring accuracy, clarity, and overall trust. These results suggest that LLMs, guided by prompt-based scoring and structured justification, can support objective, interpretable, and scalable TALD assessments. The framework performs best for linguistically relevant disorders and provides a transparent interface to assist clinical reasoning. Future work will target rare disorders, enhance justification, and extend validation across broader clinical cohorts.\n                  <\/jats:p>","DOI":"10.1007\/s11042-026-21401-8","type":"journal-article","created":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T11:24:14Z","timestamp":1770981854000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Prompt-based justification and scoring with large language models for thought and language disorder assessment"],"prefix":"10.1007","volume":"85","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6929-0056","authenticated-orcid":false,"given":"Rita","family":"Francese","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Laura","family":"De Santis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Federica","family":"Iannotta","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felice","family":"Iasevoli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,2,13]]},"reference":[{"key":"21401_CR1","unstructured":"World Health Organization Schizophrenia (2025). https:\/\/www.who.int\/news-room\/fact-sheets\/detail\/schizophrenia. Accessed 28 Jun 2025"},{"key":"21401_CR2","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1016\/j.schres.2020.01.002","volume":"218","author":"BB Ortiz","year":"2020","unstructured":"Ortiz BB, Higuchi CH, Noto C, Joyce DW, Correll CU, Bressan RA (2020) Gadelha A A symptom combination predicting treatment-resistant schizophrenia-a strategy for real-world clinical practice. Schizophr Res 218:195\u2013200","journal-title":"Schizophr Res"},{"issue":"1\u20133","key":"21401_CR3","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.schres.2016.09.017","volume":"178","author":"E Roche","year":"2016","unstructured":"Roche E, Lyne J, O\u2019Donoghue B, Segurado R, Behan C, Renwick L, Fanning F, Madigan K (2016) Clarke M The prognostic value of formal thought disorder following first episode psychosis. Schizophr Res 178(1\u20133):29\u201334","journal-title":"Schizophr Res"},{"issue":"3","key":"21401_CR4","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1093\/schbul\/12.3.373","volume":"12","author":"M Harrow","year":"1986","unstructured":"Harrow M, Marengo JT (1986) Schizophrenic thought disorder at followup: its persistence and prognostic significance. Schizophr Bull 12(3):373\u2013393","journal-title":"Schizophr Bull"},{"issue":"1","key":"21401_CR5","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1016\/j.schres.2014.10.024","volume":"160","author":"T Kircher","year":"2014","unstructured":"Kircher T, Krug A, Stratmann M, Ghazi S, Schales C, Frauenheim M, Turner L, F\u00e4hrmann P, Hornig T, Katzev M, Grosvald M, M\u00fcller-Isberner R (2014) Nagels A A rating scale for the assessment of objective and subjective formal thought and language disorder (tald). Schizophr Res 160(1):216\u2013221","journal-title":"Schizophr Res"},{"issue":"7","key":"21401_CR6","doi-asserted-by":"publisher","first-page":"36417","DOI":"10.2196\/36417","volume":"11","author":"RO Cotes","year":"2022","unstructured":"Cotes RO, Boazak M, Griner E, Jiang Z, Kim B, Bremer W, Seyedi S, Rad AB, Clifford GD (2022) Multimodal assessment of schizophrenia and depression utilizing video, acoustic, locomotor, electroencephalographic, and heart rate technology: Protocol for an observational study. JMIR Res Prot 11(7):36417","journal-title":"JMIR Res Prot"},{"issue":"1","key":"21401_CR7","first-page":"1","volume":"8","author":"X Xu","year":"2024","unstructured":"Xu X, Yao B, Dong Y, Gabriel S, Yu H, Hendler J, Ghassemi M, Dey AK (2024) Wang D Mental-llm: Leveraging large language models for mental health prediction via online text data. Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies 8(1):1\u201332","journal-title":"Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies"},{"key":"21401_CR8","doi-asserted-by":"crossref","unstructured":"Li J, Zhou Z, Lyu H, Wang Z (2025) Large language models-powered clinical decision support: enhancing or replacing human expertise? Elsevier","DOI":"10.1016\/j.imed.2025.01.001"},{"key":"21401_CR9","unstructured":"Rawte V, Sheth A, Das A (2023) A survey of hallucination in large foundation models. arXiv preprint arXiv:2309.05922"},{"key":"21401_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-025-20984-y","author":"R Francese","year":"2025","unstructured":"Francese R, Delle Cave M, Barra P, Ciccarelli M, De Simone G, Iannotta F (2025) Iasevoli F DeepTald: a system for supporting schizophrenia-related language and thought disorders detection with nlp models and explanations. Multimedia Tools and Applications. https:\/\/doi.org\/10.1007\/s11042-025-20984-y","journal-title":"Multimedia Tools and Applications"},{"issue":"1","key":"21401_CR11","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1038\/s41537-019-0077-9","volume":"5","author":"N Rezaii","year":"2019","unstructured":"Rezaii N, Walker E (2019) Wolff P A machine learning approach to predicting psychosis using semantic density and latent content analysis. NPJ Schizophr 5(1):9","journal-title":"NPJ Schizophr"},{"key":"21401_CR12","doi-asserted-by":"crossref","unstructured":"Nettekoven CR, Diederen K, Giles O, Duncan H, Stenson I, Olah J, Gibbs-Dean T, Collier N, V\u00e9rtes PE, Spencer TJ et al (2023) Semantic speech networks linked to formal thought disorder in early psychosis. Schizophrenia bulletin 49(Supplement_2), 142\u2013152","DOI":"10.1093\/schbul\/sbac056"},{"key":"21401_CR13","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1109\/TNSRE.2022.3163777","volume":"30","author":"Y-J Huang","year":"2022","unstructured":"Huang Y-J, Lin Y-T, Liu C-C, Lee L-E, Hung S-H, Lo J-K, Fu L-C (2022) Assessing schizophrenia patients through linguistic and acoustic features using deep learning techniques. IEEE Trans Neural Syst Rehab Eng 30:947\u2013956","journal-title":"IEEE Trans Neural Syst Rehab Eng"},{"key":"21401_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2021.114135","volume":"304","author":"J Sarzynska-Wawer","year":"2021","unstructured":"Sarzynska-Wawer J, Wawer A, Pawlak A, Szymanowska J, Stefaniak I, Jarkiewicz M (2021) Okruszek L Detecting formal thought disorder by deep contextualized word representations. Psychiatry Res 304:114135","journal-title":"Psychiatry Res"},{"key":"21401_CR15","doi-asserted-by":"crossref","unstructured":"Iter D, Yoon J, Jurafsky D (2018) Automatic detection of incoherent speech for diagnosing schizophrenia. In: Proceedings of the fifth workshop on computational linguistics and clinical psychology: from keyboard to clinic, pp 136\u2013146","DOI":"10.18653\/v1\/W18-0615"},{"issue":"3","key":"21401_CR16","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1176\/appi.prcp.20230003","volume":"5","author":"L Jeong","year":"2023","unstructured":"Jeong L, Lee M, Eyre B, Balagopalan A, Rudzicz F (2023) Gabilondo C Exploring the use of natural language processing for objective assessment of disorganized speech in schizophrenia. Psychiatr Res Clin Pract 5(3):84\u201392","journal-title":"Psychiatr Res Clin Pract"},{"key":"21401_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2024.116119","volume":"341","author":"SL Pugh","year":"2024","unstructured":"Pugh SL, Chandler C, Cohen AS, Diaz-Asper C, Elvev\u00e5g B, Foltz PW (2024) Assessing dimensions of thought disorder with large language models: The tradeoff of accuracy and consistency. Psychiatry Res 341:116119","journal-title":"Psychiatry Res"},{"issue":"10","key":"21401_CR18","first-page":"1013","volume":"8","author":"I Fradkin","year":"2023","unstructured":"Fradkin I, Nour MM, Dolan RJ (2023) Theory-driven analysis of natural language processing measures of thought disorder using generative language modeling. Biol Psychiatry Cogn Neurosci Neuroimaging 8(10):1013\u20131023","journal-title":"Biol Psychiatry Cogn Neurosci Neuroimaging"},{"key":"21401_CR19","doi-asserted-by":"crossref","unstructured":"Amaro I, Francese R, Tortora G, Tucci C, D\u2019Errico L, Staffa M (2023) Supporting schizophrenia patients\u2019 care with robotics and artificial intelligence. In: International conference on human-computer interaction, pp 482\u2013495. Springer","DOI":"10.1007\/978-3-031-48041-6_32"},{"issue":"1\u20133","key":"21401_CR20","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1016\/j.schres.2007.03.001","volume":"93","author":"B Elvev\u00e5g","year":"2007","unstructured":"Elvev\u00e5g B, Foltz PW, Weinberger DR, Goldberg TE (2007) Quantifying incoherence in speech: an automated methodology and novel application to schizophrenia. Schizophrenia Res 93(1\u20133):304\u2013316","journal-title":"Schizophrenia Res"},{"issue":"4","key":"21401_CR21","doi-asserted-by":"publisher","first-page":"34928","DOI":"10.1371\/journal.pone.0034928","volume":"7","author":"NB Mota","year":"2012","unstructured":"Mota NB, Vasconcelos NA, Lemos N, Pieretti AC, Kinouchi O, Cecchi GA, Copelli M (2012) Ribeiro S Speech graphs provide a quantitative measure of thought disorder in psychosis. PLoS ONE 7(4):34928","journal-title":"PLoS ONE"},{"issue":"1","key":"21401_CR22","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1002\/wps.20491","volume":"17","author":"CM Corcoran","year":"2018","unstructured":"Corcoran CM, Carrillo F, Fern\u00e1ndez-Slezak D, Bedi G, Klim C, Javitt DC, Bearden CE, Cecchi GA (2018) Prediction of psychosis across protocols and risk cohorts using automated language analysis. World Psychiatry 17(1):67\u201375","journal-title":"World Psychiatry"},{"issue":"9","key":"21401_CR23","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1016\/j.medengphy.2010.07.013","volume":"32","author":"V Rapcan","year":"2010","unstructured":"Rapcan V, D\u2019Arcy S, Yeap S, Afzal N, Thakore J, Reilly RB (2010) Acoustic and temporal analysis of speech: A potential biomarker for schizophrenia. Medical Engineering & Physics 32(9):1074\u20131079. https:\/\/doi.org\/10.1016\/j.medengphy.2010.07.013","journal-title":"Medical Engineering & Physics"},{"issue":"1","key":"21401_CR24","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1002\/lio2.354","volume":"5","author":"DM Low","year":"2020","unstructured":"Low DM, Bentley KH, Ghosh SS (2020) Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope investigative otolaryngology 5(1):96\u2013116","journal-title":"Laryngoscope investigative otolaryngology"},{"issue":"3","key":"21401_CR25","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1093\/schbul\/12.3.473","volume":"12","author":"NC Andreasen","year":"1986","unstructured":"Andreasen NC (1986) Scale for the assessment of thought, language, and communication (tlc). Schizophr Bull 12(3):473","journal-title":"Schizophr Bull"},{"key":"21401_CR26","doi-asserted-by":"crossref","unstructured":"Xiao H, Zhou F, Liu X, Liu T, Li Z, Liu X, Huang X (2024) A comprehensive survey of large language models and multimodal large language models in medicine. Information Fusion, 102888","DOI":"10.2139\/ssrn.5031720"},{"issue":"7","key":"21401_CR27","doi-asserted-by":"publisher","first-page":"744","DOI":"10.3390\/diagnostics14070744","volume":"14","author":"X Xu","year":"2024","unstructured":"Xu X, Chen Z, Li J (2024) colleagues Natural language processing and schizophrenia: A scoping review. Diagnostics 14(7):744. https:\/\/doi.org\/10.3390\/diagnostics14070744","journal-title":"Diagnostics"},{"key":"21401_CR28","doi-asserted-by":"crossref","unstructured":"Bredin H, Yin R, Coria JM, Gelly G, Korshunov P, Lavechin M, Fustes D, Titeux H, Bouaziz W, Gill M-P (2020) Pyannote. audio: neural building blocks for speaker diarization. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 7124\u20137128. IEEE","DOI":"10.1109\/ICASSP40776.2020.9052974"},{"key":"21401_CR29","unstructured":"Radford A, Kim JW, Xu T, Brockman G, McLeavey C, Sutskever I (2023) Robust speech recognition via large-scale weak supervision. In: International conference on machine learning, pp 28492\u201328518. PMLR"},{"key":"21401_CR30","unstructured":"Jiang AQ, Sablayrolles A, Roux A, Mensch A, Savary B, Bamford C, Chaplot DS, Casas Ddl, Hanna EB, Bressand F et al (2024) Mixtral of experts. arXiv preprint arXiv:2401.04088"},{"key":"21401_CR31","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M et al (2020) Transformers: State-of-the-art natural language processing. In: Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp 38\u201345","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"21401_CR32","unstructured":"Cleveland Clinic (2022) DSM-5: What It Is & What It Diagnoses. https:\/\/my.clevelandclinic.org\/health\/articles\/24291-diagnostic-and-statistical-manual-dsm-5. Accessed: June 22, 2025"},{"issue":"3","key":"21401_CR33","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","volume":"27","author":"CE Shannon","year":"1948","unstructured":"Shannon CE (1948) A mathematical theory of communication. The Bell system technical journal 27(3):379\u2013423","journal-title":"The Bell system technical journal"},{"key":"21401_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2022.102941","volume":"169","author":"M Naiseh","year":"2023","unstructured":"Naiseh M, Al-Thani D, Jiang N (2023) Ali R How the different explanation classes impact trust calibration: The case of clinical decision support systems. Int J Hum Comput Stud 169:102941","journal-title":"Int J Hum Comput Stud"},{"issue":"4","key":"21401_CR35","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","volume":"45","author":"M Sokolova","year":"2009","unstructured":"Sokolova M (2009) Lapalme G A systematic analysis of performance measures for classification tasks. Information processing & management 45(4):427\u2013437","journal-title":"Information processing & management"},{"key":"21401_CR36","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1023\/A:1010920819831","volume":"45","author":"DJ Hand","year":"2004","unstructured":"Hand DJ, Till RJ (2004) A simple generalisation of the area under the roc curve for multiple class classification problems. Mach Learn 45:171\u2013186","journal-title":"Mach Learn"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-026-21401-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-026-21401-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-026-21401-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T11:24:17Z","timestamp":1770981857000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-026-21401-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,13]]},"references-count":36,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["21401"],"URL":"https:\/\/doi.org\/10.1007\/s11042-026-21401-8","relation":{},"ISSN":["1573-7721"],"issn-type":[{"value":"1573-7721","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,13]]},"assertion":[{"value":"3 July 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 October 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 February 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"All experimental procedures involving human participants were reviewed and approved by the Ethics Committee of the Department of Computer Science at the University of Salerno (Approval Record No.\u00a01, 2025).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Informed consent was obtained from all individual participants included in the study. Participants were informed about the purpose of the research, the procedures involved, and their right to withdraw at any time without consequence.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"Participants provided explicit consent for their anonymized data to be used in publications arising from this research. No personally identifiable information is included in the published materials.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Publish"}}],"article-number":"170"}}