{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T20:20:29Z","timestamp":1783455629185,"version":"3.55.0"},"reference-count":71,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T00:00:00Z","timestamp":1783382400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T00:00:00Z","timestamp":1783382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100022790","name":"Hector Stiftung II","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100022790","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Psychiatry\u2019s reliance on language makes LLMs a natural tool for psychopathological assessment, yet structured, item-level assessments from psychiatric clinical interviews remain under-researched. In this proof-of-concept study, 10 LLMs assessed transcripts of three simulated psychiatric interviews across all 100 items of the Association for Methodology and Documentation in Psychiatry (AMDP) system, benchmarked against 108 early-career clinicians rating full audiovisual recordings, using an expert consensus panel as reference. GPT-5.1 and Gemini-3-Pro-Preview achieved the highest accuracy (0.72; 64th percentile of the clinician distribution) using majority voting across three runs with AMDP definitions as context. GPT-5.1, selected for a marginal advantage, showed per-scenario accuracies of 0.81 (depression), 0.76 (mania), and 0.60 (schizophrenia) versus clinician means of 0.79, 0.68, and 0.58. Clinicians and LLMs showed distinct error profiles: clinicians tended to over-infer symptom presence, whereas LLMs more conservatively flagged items as \u201cnot assessable\u201d \u2014 most pronounced for observation-dependent items but present even for text-assessable items (19.4% vs. 11.4%,\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001). In post hoc simulated disagreement resolutions (2091 clinician pairs; 35.5% disagreements), LLM and board-certified supervision were associated with more accurate resolutions than unsupervised random clinician selection (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.0002). These proof-of-concept findings require validation in real patient interviews, larger samples, and prospective studies integrating multimodal input.\n                  <\/jats:p>","DOI":"10.1038\/s41746-026-02852-7","type":"journal-article","created":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T18:03:50Z","timestamp":1783447430000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Benchmarking large language models against practicing clinicians on psychopathological assessment"],"prefix":"10.1038","volume":"9","author":[{"given":"Esra","family":"Lenz","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joonas","family":"Naamanka","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolfgang","family":"Trabert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald","family":"Bottlender","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Berend","family":"Malchow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Meyer-Lindenberg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Gradinger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emanuel","family":"Schwarz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,7,7]]},"reference":[{"key":"2852_CR1","doi-asserted-by":"publisher","first-page":"e0331459","DOI":"10.1371\/journal.pone.0331459","volume":"20","author":"BE Bunnell","year":"2025","unstructured":"Bunnell, B. E. et al. Automated detection and prediction of suicidal behavior from clinical notes using deep learning. PLOS ONE 20, e0331459 (2025).","journal-title":"PLOS ONE"},{"key":"2852_CR2","first-page":"770","volume":"5","author":"CM Corcoran","year":"2020","unstructured":"Corcoran, C. M. & Cecchi, G. A. Using Language Processing and Speech Analysis for the Identification of Psychosis and Other Disorders. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 5, 770\u2013779 (2020).","journal-title":"Biol. Psychiatry Cogn. Neurosci. Neuroimaging"},{"key":"2852_CR3","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1080\/01638539809545028","volume":"25","author":"TK Landauer","year":"1998","unstructured":"Landauer, T. K., Foltz, P. W. & Laham, D. An introduction to latent semantic analysis. Discourse Process 25, 259\u2013284 (1998).","journal-title":"Discourse Process"},{"key":"2852_CR4","doi-asserted-by":"publisher","first-page":"19","DOI":"10.31887\/DCNS.2015.17.1\/hhaefner","volume":"17","author":"H H\u00e4fner","year":"2015","unstructured":"H\u00e4fner, H. Descriptive psychopathology, phenomenology, and the legacy of Karl Jaspers. Dialogues Clin. Neurosci. 17, 19\u201329 (2015).","journal-title":"Dialogues Clin. Neurosci."},{"key":"2852_CR5","doi-asserted-by":"publisher","first-page":"282","DOI":"10.3390\/diagnostics15030282","volume":"15","author":"R Obuchowicz","year":"2025","unstructured":"Obuchowicz, R. et al. Artificial intelligence-empowered radiology\u2014current status and critical review. Diagnostics 15, 282 (2025).","journal-title":"Diagnostics"},{"key":"2852_CR6","doi-asserted-by":"publisher","DOI":"10.1186\/1755-7682-3-1","volume":"3","author":"SE Lakhan","year":"2010","unstructured":"Lakhan, S. E., Vieira, K. & Hamlat, E. Biomarkers in psychiatry: drawbacks and potential for misuse. Int. Arch. Med. 3, 1 (2010).","journal-title":"Int. Arch. Med."},{"key":"2852_CR7","first-page":"62","volume":"94","author":"C Armstrong","year":"2016","unstructured":"Armstrong, C. APA Updates Guidelines on Psychiatric Evaluation in Adults. Am. Fam. Phys. 94, 62\u201364 (2016).","journal-title":"Am. Fam. Phys."},{"key":"2852_CR8","doi-asserted-by":"publisher","first-page":"946387","DOI":"10.3389\/fpsyt.2022.946387","volume":"13","author":"C Crema","year":"2022","unstructured":"Crema, C., Attardi, G., Sartiano, D. & Redolfi, A. Natural language processing in clinical neuroscience and psychiatry: A review. Front. Psychiatry 13, 946387 (2022).","journal-title":"Front. Psychiatry"},{"key":"2852_CR9","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1609\/icwsm.v7i1.14432","volume":"7","author":"M De Choudhury","year":"2013","unstructured":"De Choudhury, M., Gamon, M., Counts, S. & Horvitz, E. Predicting depression via social media. Proc. Int. AAAI Conf. Web Soc. Media 7, 128\u2013137 (2013).","journal-title":"Proc. Int. AAAI Conf. Web Soc. Media"},{"key":"2852_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-024-02134-3","volume":"48","author":"I Levkovich","year":"2024","unstructured":"Levkovich, I. & Omar, M. Evaluating of BERT-based and large language models for suicide detection, prevention, and risk assessment: a systematic review. J. Med. Syst. 48, 113 (2024).","journal-title":"J. Med. Syst."},{"key":"2852_CR11","doi-asserted-by":"publisher","first-page":"e55067","DOI":"10.2196\/55067","volume":"13","author":"BG Teferra","year":"2024","unstructured":"Teferra, B. G. et al. Screening for depression using natural language processing: literature review. Interact. J. Med. Res. 13, e55067 (2024).","journal-title":"Interact. J. Med. Res."},{"key":"2852_CR12","doi-asserted-by":"crossref","unstructured":"Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171\u20134186 (2019).","DOI":"10.18653\/v1\/N19-1423"},{"key":"2852_CR13","doi-asserted-by":"crossref","unstructured":"Ji, S. et al. MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. In Proceedings of the 13th Language Resources and Evaluation Conference (eds Calzolari, N. et al.) 7184\u20137190 (2022).","DOI":"10.63317\/2d7umxifj8wz"},{"key":"2852_CR14","doi-asserted-by":"publisher","first-page":"e0332207","DOI":"10.1371\/journal.pone.0332207","volume":"20","author":"A Humayun","year":"2025","unstructured":"Humayun, A. et al. Artificial intelligence as a predictive tool for mental health status: Insights from a systematic review and meta-analysis. PLOS ONE 20, e0332207 (2025).","journal-title":"PLOS ONE"},{"key":"2852_CR15","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyt.2024.1422807","volume":"15","author":"M Omar","year":"2024","unstructured":"Omar, M. et al. Applications of large language models in psychiatry: a systematic review. Front. Psychiatry 15, 1422807 (2024).","journal-title":"Front. Psychiatry"},{"key":"2852_CR16","doi-asserted-by":"crossref","unstructured":"Rosenman, G., Hendler, T. & Wolf, L. LLM Questionnaire Completion for Automatic Psychiatric Assessment. In Findings of the Association for Computational Linguistics: EMNLP 2024 (eds Al-Onaizan, Y., Bansal, M. & Chen, Y.-N.) 403\u2013415 (2024).","DOI":"10.18653\/v1\/2024.findings-emnlp.23"},{"key":"2852_CR17","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1038\/s41746-020-0233-7","volume":"3","author":"S Chancellor","year":"2020","unstructured":"Chancellor, S. & De Choudhury, M. Methods in predictive techniques for mental health status on social media: a critical review. npj Digit. Med. 3, 43 (2020).","journal-title":"npj Digit. Med."},{"key":"2852_CR18","doi-asserted-by":"publisher","first-page":"100321","DOI":"10.1016\/j.osnem.2025.100321","volume":"48","author":"L Belcastro","year":"2025","unstructured":"Belcastro, L., Cantini, R., Marozzo, F., Talia, D. & Trunfio, P. Detecting mental disorder on social media: A ChatGPT-augmented explainable approach. Online Soc. Netw. Media 48, 100321 (2025).","journal-title":"Online Soc. Netw. Media"},{"key":"2852_CR19","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1038\/s41746-024-01181-x","volume":"7","author":"J Kim","year":"2024","unstructured":"Kim, J. et al. Large language models outperform mental and medical health care professionals in identifying obsessive-compulsive disorder. npj Digit. Med. 7, 193 (2024).","journal-title":"npj Digit. Med."},{"key":"2852_CR20","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1038\/s41537-025-00601-5","volume":"11","author":"S Volkmer","year":"2025","unstructured":"Volkmer, S., Gl\u00fcck, A., Meyer-Lindenberg, A., Schwarz, E. & Hirjak, D. Validating large language models against manual information extraction from case reports of drug-induced parkinsonism in patients with schizophrenia spectrum and mood disorders: a proof of concept study. Schizophrenia 11, 47 (2025).","journal-title":"Schizophrenia"},{"key":"2852_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/S0033291724002642","volume":"54","author":"E Perfalk","year":"2024","unstructured":"Perfalk, E. et al. Predicting involuntary admission following inpatient psychiatric treatment using machine learning trained on electronic health record data. Psychol. Med. 54, 1\u201314 (2024).","journal-title":"Psychol. Med."},{"key":"2852_CR22","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1017\/neu.2021.22","volume":"33","author":"L Hansen","year":"2021","unstructured":"Hansen, L. et al. The PSYchiatric clinical outcome prediction (PSYCOP) cohort: leveraging the potential of electronic health records in the treatment of mental disorders. Acta Neuropsychiatr. 33, 323\u2013330 (2021).","journal-title":"Acta Neuropsychiatr."},{"key":"2852_CR23","doi-asserted-by":"publisher","unstructured":"Li, L. et al. A scoping review of using Large Language Models (LLMs) to investigate Electronic Health Records (EHRs). Preprint at https:\/\/doi.org\/10.48550\/arXiv.2405.03066 (2024).","DOI":"10.48550\/arXiv.2405.03066"},{"key":"2852_CR24","doi-asserted-by":"publisher","first-page":"104724","DOI":"10.1016\/j.ijmedinf.2022.104724","volume":"161","author":"S Madan","year":"2022","unstructured":"Madan, S. et al. Deep Learning-based detection of psychiatric attributes from German mental health records. Int. J. Med. Inform. 161, 104724 (2022).","journal-title":"Int. J. Med. Inform."},{"key":"2852_CR25","doi-asserted-by":"publisher","first-page":"446","DOI":"10.3389\/fpsyt.2018.00446","volume":"9","author":"F Schultze-Lutter","year":"2018","unstructured":"Schultze-Lutter, F., Schmidt, S. J. & Theodoridou, A. Psychopathology\u2014a precision tool in need of re-sharpening. Front. Psychiatry 9, 446 (2018).","journal-title":"Front. Psychiatry"},{"key":"2852_CR26","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1111\/pcn.13864","volume":"79","author":"A Raballo","year":"2025","unstructured":"Raballo, A., Ravenda, F. & Mira, A. Diagnosing schizophrenia spectrum disorders: Large language models (LLMs) vs. leading international psychiatrists (LIPs). Psychiatry Clin. Neurosci. 79, 599\u2013600 (2025).","journal-title":"Psychiatry Clin. Neurosci."},{"key":"2852_CR27","first-page":"9","volume":"15","author":"I Levkovich","year":"2025","unstructured":"Levkovich, I. Evaluating diagnostic accuracy and treatment efficacy in mental health: a comparative analysis of large language model tools and mental health professionals. Eur. J. Investig. Health Psychol. Educ. 15, 9 (2025).","journal-title":"Eur. J. Investig. Health Psychol. Educ."},{"key":"2852_CR28","doi-asserted-by":"publisher","first-page":"e250040","DOI":"10.1001\/jamahealthforum.2025.0040","volume":"6","author":"TA Buckley","year":"2025","unstructured":"Buckley, T. A., Crowe, B., Abdulnour, R.-E. E., Rodman, A. & Manrai, A. K. Comparison of frontier open-source and proprietary large language models for complex diagnoses. JAMA Health Forum 6, e250040 (2025).","journal-title":"JAMA Health Forum"},{"key":"2852_CR29","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1038\/s41746-025-01543-z","volume":"8","author":"H Takita","year":"2025","unstructured":"Takita, H. et al. A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians. npj Digit. Med. 8, 175 (2025).","journal-title":"npj Digit. Med."},{"key":"2852_CR30","doi-asserted-by":"publisher","first-page":"701","DOI":"10.3390\/healthcare13070701","volume":"13","author":"RA El Arab","year":"2025","unstructured":"El Arab, R. A. et al. Bridging the gap: from AI success in clinical trials to real-world healthcare implementation\u2014a narrative review. Healthcare 13, 701 (2025).","journal-title":"Healthcare"},{"key":"2852_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajp.2024.104168","volume":"100","author":"OK Gargari","year":"2024","unstructured":"Gargari, O. K. et al. Diagnostic accuracy of large language models in psychiatry. Asian J. Psychiatry 100, 104168 (2024).","journal-title":"Asian J. Psychiatry"},{"key":"2852_CR32","doi-asserted-by":"publisher","DOI":"10.1136\/bmjhci-2025-101470","volume":"32","author":"SB Egli","year":"2025","unstructured":"Egli, S. B., Arpagaus, A., Amacher, S. A., Hunziker, S. & Bassetti, S. Use, knowledge and perception of large language models in clinical practice: a cross-sectional mixed-methods survey among clinicians in Switzerland. BMJ Health Care Inf. 32, e101470 (2025).","journal-title":"BMJ Health Care Inf."},{"key":"2852_CR33","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1038\/s41746-025-01476-7","volume":"8","author":"F Dennst\u00e4dt","year":"2025","unstructured":"Dennst\u00e4dt, F., Hastings, J., Putora, P. M., Schmerder, M. & Cihoric, N. Implementing large language models in healthcare while balancing control, collaboration, costs and security. npj Digit. Med. 8, 143 (2025).","journal-title":"npj Digit. Med."},{"key":"2852_CR34","doi-asserted-by":"publisher","first-page":"34","DOI":"10.3928\/02793695-20110329-02","volume":"49","author":"NL Keltner","year":"2011","unstructured":"Keltner, N. L., Grant, J. S. & McLernon, D. Use of actors as standardized psychiatric patients. J. Psychosoc. Nurs. Ment. Health Serv. 49, 34\u201340 (2011).","journal-title":"J. Psychosoc. Nurs. Ment. Health Serv."},{"key":"2852_CR35","doi-asserted-by":"publisher","first-page":"910","DOI":"10.1097\/00001888-199808000-00019","volume":"73","author":"B Hodges","year":"1998","unstructured":"Hodges, B., Regehr, G., Hanson, M. & McNaughton, N. Validation of an objective structured clinical examination in psychiatry. Acad. Med. 73, 910\u2013912 (1998).","journal-title":"Acad. Med."},{"key":"2852_CR36","doi-asserted-by":"publisher","first-page":"696","DOI":"10.1111\/medu.14166","volume":"54","author":"M-A Piot","year":"2020","unstructured":"Piot, M.-A. et al. Simulation in psychiatry for medical doctors: A systematic review and meta-analysis. Med. Educ. 54, 696\u2013708 (2020).","journal-title":"Med. Educ."},{"key":"2852_CR37","doi-asserted-by":"publisher","unstructured":"Das AMDP-System: Manual zur Dokumentation des psychischen Befundes in Psychiatrie, Psychotherapie und Psychosomatik. (Hogrefe, G\u00f6ttingen, 2023). https:\/\/doi.org\/10.1026\/03157-000.","DOI":"10.1026\/03157-000"},{"key":"2852_CR38","unstructured":"The AMDP System: Manual for Assessment and Documentation of Psychopathology in Psychiatry. (Hogrefe, Boston, 2018)."},{"key":"2852_CR39","doi-asserted-by":"publisher","first-page":"45","DOI":"10.3389\/fpsyt.2017.00045","volume":"8","author":"R-D Stieglitz","year":"2017","unstructured":"Stieglitz, R.-D., Haug, A., F\u00e4hndrich, E., R\u00f6sler, M. & Trabert, W. Comprehensive psychopathological assessment based on the Association for Methodology and Documentation in Psychiatry (AMDP) System: development, methodological foundation, application in clinical routine, and research. Front. Psychiatry 8, 45 (2017).","journal-title":"Front. Psychiatry"},{"key":"2852_CR40","doi-asserted-by":"publisher","first-page":"e1001531","DOI":"10.1371\/journal.pmed.1001531","volume":"10","author":"LCM Bertens","year":"2013","unstructured":"Bertens, L. C. M. et al. Use of expert panels to define the reference standard in diagnostic research: a systematic review of published methods and reporting. PLOS Med. 10, e1001531 (2013).","journal-title":"PLOS Med."},{"key":"2852_CR41","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1093\/cid\/ciac829","volume":"76","author":"R Patel","year":"2022","unstructured":"Patel, R., Tsalik, E. L., Evans, S., Fowler, V. G. & Doernberg, S. B. Clinically adjudicated reference standards for evaluation of infectious diseases diagnostics. Clin. Infect. Dis. 76, 938\u2013943 (2022).","journal-title":"Clin. Infect. Dis."},{"key":"2852_CR42","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1097\/00004714-200206000-00013","volume":"22","author":"MJ M\u00fcller","year":"2002","unstructured":"M\u00fcller, M. J. & Szegedi, A. Effects of interrater reliability of psychopathologic assessment on power and sample size calculations in clinical trials. J. Clin. Psychopharmacol. 22, 318\u2013325 (2002).","journal-title":"J. Clin. Psychopharmacol."},{"key":"2852_CR43","doi-asserted-by":"publisher","first-page":"526","DOI":"10.3390\/diagnostics13030526","volume":"13","author":"H Rocha Neto","year":"2023","unstructured":"Rocha Neto, H. et al. Inter-rater reliability between structured and non-structured interviews is fair in schizophrenia and bipolar disorders\u2014a systematic review and meta-analysis. Diagnostics 13, 526 (2023).","journal-title":"Diagnostics"},{"key":"2852_CR44","doi-asserted-by":"publisher","unstructured":"Imam, M. F., Lyu, C. & Aji, A. F. Can Multimodal LLMs do visual temporal understanding and reasoning? The answer is No! Preprint at https:\/\/doi.org\/10.48550\/arXiv.2501.10674 (2025).","DOI":"10.48550\/arXiv.2501.10674"},{"key":"2852_CR45","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2426153122","volume":"122","author":"N Z\u00f6ller","year":"2025","unstructured":"Z\u00f6ller, N. et al. Human\u2013AI collectives most accurately diagnose clinical vignettes. Proc. Natl. Acad. Sci. USA. 122, e2426153122 (2025).","journal-title":"Proc. Natl. Acad. Sci. USA."},{"key":"2852_CR46","doi-asserted-by":"publisher","first-page":"e2111547119","DOI":"10.1073\/pnas.2111547119","volume":"119","author":"M Steyvers","year":"2022","unstructured":"Steyvers, M., Tejeda, H., Kerrigan, G. & Smyth, P. Bayesian modeling of human\u2013AI complementarity. Proc. Natl. Acad. Sci. USA. 119, e2111547119 (2022).","journal-title":"Proc. Natl. Acad. Sci. USA."},{"key":"2852_CR47","doi-asserted-by":"publisher","first-page":"2293","DOI":"10.1038\/s41562-024-02024-1","volume":"8","author":"M Vaccaro","year":"2024","unstructured":"Vaccaro, M., Almaatouq, A. & Malone, T. When combinations of humans and AI are useful: A systematic review and meta-analysis. Nat. Hum. Behav. 8, 2293\u20132303 (2024).","journal-title":"Nat. Hum. Behav."},{"key":"2852_CR48","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-18751-2","volume":"12","author":"C Reverberi","year":"2022","unstructured":"Reverberi, C. et al. Experimental evidence of effective human-AI collaboration in medical decision-making. Sci. Rep. 12, 14952 (2022).","journal-title":"Sci. Rep."},{"key":"2852_CR49","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1001\/jama.2024.21700","volume":"333","author":"S Bedi","year":"2025","unstructured":"Bedi, S. et al. Testing and evaluation of health care applications of large language models: a systematic review. JAMA 333, 319\u2013328 (2025).","journal-title":"JAMA"},{"key":"2852_CR50","doi-asserted-by":"crossref","unstructured":"Kasagga, A. et al. Performance of ChatGPT and Large Language Models on Medical Licensing Exams Worldwide: A Systematic Review and Network Meta-Analysis With Meta-Regression. Cureus 17, e94300, (2025).","DOI":"10.7759\/cureus.94300"},{"key":"2852_CR51","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1162\/tacl_a_00638","volume":"12","author":"NF Liu","year":"2024","unstructured":"Liu, N. F. et al. Lost in the middle: how language models use long contexts. Trans. Assoc. Comput. Linguist. 12, 157\u2013173 (2024).","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"2852_CR52","unstructured":"Xie, J., Zhang, K., Chen, J., Lou, R. & Su, Y. Adaptive Chameleon or Stubborn Sloth: revealing the behavior of large language models in knowledge conflicts. In the 12th International Conference on Learning Representations https:\/\/openreview.net\/forum?id=auKAUJZMO6 (2024)."},{"key":"2852_CR53","doi-asserted-by":"publisher","unstructured":"Du, Y. et al. Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2025 23281\u201323298 (Association for Computational Linguistics, 2025). https:\/\/doi.org\/10.18653\/v1\/2025.findings-emnlp.1264.","DOI":"10.18653\/v1\/2025.findings-emnlp.1264"},{"key":"2852_CR54","doi-asserted-by":"publisher","first-page":"101891","DOI":"10.1016\/j.ijlp.2023.101891","volume":"88","author":"\u00c1 Escol\u00e0-Gasc\u00f3n","year":"2023","unstructured":"Escol\u00e0-Gasc\u00f3n, \u00c1, Dagnall, N. & Drinkwater, K. Bias analysis in forensic and non-forensic psychiatric assessments. Int. J. Law Psychiatry 88, 101891 (2023).","journal-title":"Int. J. Law Psychiatry"},{"key":"2852_CR55","doi-asserted-by":"publisher","first-page":"2651","DOI":"10.1017\/S0033291711000808","volume":"41","author":"R Mendel","year":"2011","unstructured":"Mendel, R. et al. Confirmation bias: why psychiatrists stick to wrong preliminary diagnoses. Psychol. Med. 41, 2651\u20132659 (2011).","journal-title":"Psychol. Med."},{"key":"2852_CR56","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1038\/s41746-025-01985-5","volume":"8","author":"BW Nelson","year":"2025","unstructured":"Nelson, B. W. et al. Evaluating the performance of general purpose large language models in identifying human facial emotions. npj Digit. Med. 8, 615 (2025).","journal-title":"npj Digit. Med."},{"key":"2852_CR57","doi-asserted-by":"publisher","first-page":"e54369","DOI":"10.2196\/54369","volume":"11","author":"Z Elyoseph","year":"2024","unstructured":"Elyoseph, Z. et al. Capacity of Generative AI to interpret human emotions from visual and textual data: pilot evaluation study. JMIR Ment. Health 11, e54369 (2024).","journal-title":"JMIR Ment. Health"},{"key":"2852_CR58","doi-asserted-by":"publisher","unstructured":"Xu, C., Guan, S., Greene, D. & Kechadi, M.-T. Benchmark Data Contamination of Large Language Models: A Survey. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2406.04244 (2024).","DOI":"10.48550\/arXiv.2406.04244"},{"key":"2852_CR59","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-34591-0","volume":"13","author":"S Ott","year":"2022","unstructured":"Ott, S., Barbosa-Silva, A., Blagec, K., Brauner, J. & Samwald, M. Mapping global dynamics of benchmark creation and saturation in artificial intelligence. Nat. Commun. 13, 6793 (2022).","journal-title":"Nat. Commun."},{"key":"2852_CR60","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1007\/s00406-011-0271-x","volume":"262","author":"D L\u00e4ge","year":"2012","unstructured":"L\u00e4ge, D., Egli, S., Riedel, M. & M\u00f6ller, H.-J. Exploring the structure of psychopathological symptoms: a re-analysis of AMDP data by robust nonmetric multidimensional scaling. Eur. Arch. Psychiatry Clin. Neurosci. 262, 227\u2013238 (2012).","journal-title":"Eur. Arch. Psychiatry Clin. Neurosci."},{"key":"2852_CR61","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1007\/BF00342790","volume":"233","author":"R Gebhardt","year":"1983","unstructured":"Gebhardt, R. & Pietzcker, A. Zur Validierung der AMDP-Syndromskalen. Arch. Psychiatr. Nervenkr. 233, 509\u2013523 (1983).","journal-title":"Arch. Psychiatr. Nervenkr."},{"key":"2852_CR62","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1055\/s-2007-1017381","volume":"18","author":"E F\u00e4hndrich","year":"1985","unstructured":"F\u00e4hndrich, E. & Renfordt, E. The AMDP system for the documentation of psychiatric symptoms: course and effectivity of a training seminar. Pharmacopsychiatry 18, 278\u2013281 (1985).","journal-title":"Pharmacopsychiatry"},{"key":"2852_CR63","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.04132","author":"W-L Chiang","year":"2024","unstructured":"Chiang, W.-L. et al. Chatbot Arena.: Open Platform for Evaluating LLMs by Human Preference https:\/\/doi.org\/10.48550\/arXiv.2403.04132 (2024).","journal-title":"Chatbot Arena.: Open Platform for Evaluating LLMs by Human Preference"},{"key":"2852_CR64","doi-asserted-by":"publisher","unstructured":"Dannhoferov\u00e1, J. & Jedli\u010dka, P. Optimizing prompts through AI frameworks: a path to more relevant responses. In Proc. 27th Annual Int. Conference Economic Competitiveness and Sustainability 2025 29\u201338. https:\/\/doi.org\/10.11118\/978-80-7701-047-4-0029 (2025).","DOI":"10.11118\/978-80-7701-047-4-0029"},{"key":"2852_CR65","doi-asserted-by":"crossref","unstructured":"Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. In Proc. 36th Int. Conference on Neural Information Processing Systems 24824\u201324837 (Curran Associates, 2022).","DOI":"10.52202\/068431-1800"},{"key":"2852_CR66","doi-asserted-by":"publisher","first-page":"95574","DOI":"10.1109\/ACCESS.2025.3575303","volume":"13","author":"A Mohamed","year":"2025","unstructured":"Mohamed, A., Rashid, M. E. & Shaalan, K. In-Context Learning in Large Language Models (LLMs): mechanisms, capabilities, and implications for advanced knowledge representation and reasoning. IEEE Access 13, 95574\u201395593 (2025).","journal-title":"IEEE Access"},{"key":"2852_CR67","doi-asserted-by":"publisher","unstructured":"Wang, X. et al. Self-consistency improves chain of thought reasoning in language models. https:\/\/doi.org\/10.48550\/arXiv.2203.11171 (2023).","DOI":"10.48550\/arXiv.2203.11171"},{"key":"2852_CR68","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s40596-019-01140-9","volume":"44","author":"A Martin","year":"2020","unstructured":"Martin, A., Krause, R., Jacobs, A., Chilton, J. & Amsalem, D. The mental status exam through video-clips of simulated psychiatric patients: an online educational resource. Acad. Psychiatry 44, 179\u2013183 (2020).","journal-title":"Acad. Psychiatry"},{"key":"2852_CR69","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1080\/08039488.2019.1703038","volume":"74","author":"ES Blaabjerg","year":"2020","unstructured":"Blaabjerg, E. S. et al. Variability between psychiatrists on domains of the mental status examination. Nord. J. Psychiatry 74, 287\u2013292 (2020).","journal-title":"Nord. J. Psychiatry"},{"key":"2852_CR70","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1176\/ps.50.5.695","volume":"50","author":"SM Bhaskara","year":"1999","unstructured":"Bhaskara, S. M. Setting Benchmarks and Determining Psychiatric Workloads in Community Mental Health Programs. Psychiatr. Serv. 50, 695\u2013696 (1999).","journal-title":"Psychiatr. Serv."},{"key":"2852_CR71","doi-asserted-by":"publisher","first-page":"S85","DOI":"10.1016\/j.mjafi.2020.11.007","volume":"77","author":"D Kumar","year":"2021","unstructured":"Kumar, D., Jaipurkar, R., Shekhar, A., Sikri, G. & Srinivas, V. Item analysis of multiple choice questions: A quality assurance test for an assessment tool. Med. J. Armed Forces India 77, S85\u2013S89 (2021).","journal-title":"Med. J. Armed Forces India"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-02852-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-02852-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-02852-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T20:02:30Z","timestamp":1783454550000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-02852-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,7]]},"references-count":71,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["2852"],"URL":"https:\/\/doi.org\/10.1038\/s41746-026-02852-7","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,7]]},"assertion":[{"value":"14 March 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 July 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A.M.L. has received editorial fees from the American Association for the Advancement of Science, Elsevier, and Thieme Verlag; consultant fees from Agence Nationale de la Recherche, Brain Mind Institute, Brainsway, CISSN (Catania International Summer School of Neuroscience), Daimler und Benz Stiftung, Fondation FondaMental, Hector Stiftung II, Janssen-Cilag GmbH, Lundbeck A\/S, Lundbeckfonden, Lundbeck International Neuroscience Foundation, MedinCell, Sage Therapeutics, Techspert.io, The LOOP Z\u00fcrich, University Medical Center Utrecht, and von Behring R\u00f6ntgen Stiftung; and speaker fees from \u00c4rztekammer Nordrhein, BAG Psychiatrie Oberbayern, Biotest AG, Forum Werkstatt Karlsruhe, International Society of Psychiatric Genetics, Brentwood, Klinik f\u00fcr Psychiatrie und Psychotherapie Ingolstadt, Lundbeck SAS France, med Update GmbH, Merz-Stiftung, Siemens Healthineers, and Society of Biological Psychiatry. E.S. received speaker fees from bfd buchholz-fachinformationsdienst GmbH and Lundbeckfonden as well as editorial fees from Lundbeckfonden and the Wellcome Trust. B.M., W.T., and R.B. are members of AMDP e.V. and are current or former members of its executive board\/leadership body. The other authors do not have a competing interest.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"518"}}