{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T20:58:50Z","timestamp":1792616330071,"version":"4.5.0"},"reference-count":27,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Psychiatry Clin Neurosci"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:sec><jats:title>Aim<\/jats:title><jats:p>Large language models (LLMs) have been suggested to play a role in medical education and medical practice. However, the potential of their application in the psychiatric domain has not been well\u2010studied.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>In the first step, we compared the performance of ChatGPT GPT\u20104, Bard, and Llama\u20102 in the 2022 Taiwan Psychiatric Licensing Examination conducted in traditional Mandarin. In the second step, we compared the scores of these three LLMs with those of 24 experienced psychiatrists in 10 advanced clinical scenario questions designed for psychiatric differential diagnosis.<\/jats:p><\/jats:sec><jats:sec><jats:title>Result<\/jats:title><jats:p>Only GPT\u20104 passed the 2022 Taiwan Psychiatric Licensing Examination (scoring 69 and \u2265\u200960 being considered a passing grade), while Bard scored 36 and Llama\u20102 scored 25. GPT\u20104 outperformed Bard and Llama\u20102, especially in the areas of \u2018Pathophysiology &amp; Epidemiology\u2019 (\u03c7<jats:sup>2<\/jats:sup>\u2009=\u200922.4, <jats:italic>P<\/jats:italic>\u2009&lt;\u20090.001) and \u2018Psychopharmacology &amp; Other therapies\u2019 (\u03c7<jats:sup>2<\/jats:sup>\u2009=\u200915.8, <jats:italic>P<\/jats:italic>\u2009&lt;\u20090.001). In the differential diagnosis, the mean score of the 24 experienced psychiatrists (mean 6.1, standard deviation 1.9) was higher than that of GPT\u20104 (5), Bard (3), and Llama\u20102 (1).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Compared to Bard and Llama\u20102, GPT\u20104 demonstrated superior abilities in identifying psychiatric symptoms and making clinical judgments. Besides, GPT\u20104's ability for differential diagnosis closely approached that of the experienced psychiatrists. GPT\u20104 revealed a promising potential as a valuable tool in psychiatric practice among the three LLMs.<\/jats:p><\/jats:sec>","DOI":"10.1111\/pcn.13656","type":"journal-article","created":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T09:05:04Z","timestamp":1708938304000},"page":"347-352","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Comparing the performance of <scp>ChatGPT GPT<\/scp>\u20104, Bard, and Llama\u20102 in the Taiwan Psychiatric Licensing Examination and in differential diagnosis with multi\u2010center psychiatrists"],"prefix":"10.1111","volume":"78","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6036-045X","authenticated-orcid":false,"given":"Dian\u2010Jeng","family":"Li","sequence":"first","affiliation":[{"name":"Department of Addiction Science Kaohsiung Municipal Kai\u2010Syuan Psychiatric Hospital  Kaohsiung Taiwan"},{"name":"Department of Nursing Meiho University  Pingtung Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu\u2010Chen","family":"Kao","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Tri\u2010Service General Hospital, National Defense Medical Center  Taipei Taiwan"},{"name":"Department of Psychiatry Tri\u2010Service General Hospital, Beitou branch  Taipei Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9987-022X","authenticated-orcid":false,"given":"Shih\u2010Jen","family":"Tsai","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Taipei Veterans General Hospital  Taipei Taiwan"},{"name":"Department of Psychiatry, College of Medicine National Yang Ming Chiao Tung University  Taipei Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3779-9074","authenticated-orcid":false,"given":"Ya\u2010Mei","family":"Bai","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Taipei Veterans General Hospital  Taipei Taiwan"},{"name":"Department of Psychiatry, College of Medicine National Yang Ming Chiao Tung University  Taipei Taiwan"},{"name":"Institute of Brain Science National Yang Ming Chiao Tung University  Taipei Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1016-3634","authenticated-orcid":false,"given":"Ta\u2010Chuan","family":"Yeh","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Tri\u2010Service General Hospital, National Defense Medical Center  Taipei Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6911-1839","authenticated-orcid":false,"given":"Che\u2010Sheng","family":"Chu","sequence":"additional","affiliation":[{"name":"Center for Geriatric and Gerontology Kaohsiung Veterans General Hospital  Kaohsiung Taiwan"},{"name":"Non\u2010invasive Neuromodulation Consortium for Mental Disorders Society of Psychophysiology  Taipei Taiwan"},{"name":"Graduate Institute of Medicine, College of Medicine Kaohsiung Medical University  Kaohsiung Taiwan"},{"name":"Department of Psychiatry Kaohsiung Veterans General Hospital  Kaohsiung Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8650-4060","authenticated-orcid":false,"given":"Chih\u2010Wei","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Kaohsiung Chang Gung Memorial Hospital  Kaohsiung Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4460-5517","authenticated-orcid":false,"given":"Szu\u2010Wei","family":"Cheng","sequence":"additional","affiliation":[{"name":"Department of General Medicine Chi Mei Medical Center  Tainan Taiwan"},{"name":"Mind\u2010Body Interface Laboratory (MBI\u2010Lab) and Department of Psychiatry China Medical University Hospital  Taichung Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4136-1251","authenticated-orcid":false,"given":"Tien\u2010Wei","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Psychiatry E\u2010DA Dachang Hospital, I\u2010Shou University  Kaohsiung Taiwan"},{"name":"Department of Psychiatry E\u2010DA Hospital, I\u2010Shou University  Kaohsiung Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1138-5586","authenticated-orcid":false,"given":"Chih\u2010Sung","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Psychiatry Tri\u2010Service General Hospital, National Defense Medical Center  Taipei Taiwan"},{"name":"Department of Psychiatry Tri\u2010Service General Hospital, Beitou branch  Taipei Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4501-2502","authenticated-orcid":false,"given":"Kuan\u2010Pin","family":"Su","sequence":"additional","affiliation":[{"name":"Mind\u2010Body Interface Laboratory (MBI\u2010Lab) and Department of Psychiatry China Medical University Hospital  Taichung Taiwan"},{"name":"College of Medicine China Medical University  Taichung Taiwan"},{"name":"An\u2010Nan Hospital China Medical University  Tainan Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2024,2,26]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.metrad.2023.100017"},{"key":"e_1_2_9_3_1","first-page":"11","article-title":"ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns","volume":"11","author":"Sallam M","year":"2023","journal-title":"Healthcare (Basel)"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_2_9_5_1","first-page":"01764","article-title":"Q\u2010pain: A question answering dataset to measure social bias in pain management","volume":"2108","author":"Log\u00e9 C","year":"2021","journal-title":"arXiv preprint arXiv:2108.01764"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-00667-z"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/pcn.13588"},{"key":"e_1_2_9_8_1","first-page":"13375","article-title":"Capabilities of gpt\u20104 on medical challenge problems","volume":"2303","author":"Nori H","year":"2023","journal-title":"arXiv preprint arXiv:2303.13375"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2023.1219326"},{"key":"e_1_2_9_10_1","first-page":"13.23292418","article-title":"GPT\u20104, an artificial intelligence large language model, exhibits high levels of accuracy on dermatology specialty certificate exam questions","volume":"2023","author":"Shetty M","year":"2023","journal-title":"medRxiv"},{"key":"e_1_2_9_11_1","first-page":"04709","article-title":"A comparative study of open\u2010source large language models, GPT\u20104 and Claude 2: Multiple\u2010choice test taking in nephrology","volume":"2023","author":"Wu S","year":"2023","journal-title":"arXiv preprint arXiv:2308.04709"},{"key":"e_1_2_9_12_1","volume-title":"When Panic Attacks: The New, Drug\u2010Free Anxiety Therapy that Can Change your Life","author":"Burns DD","year":"2007"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.2196\/mental.7785"},{"key":"e_1_2_9_14_1","volume-title":"The Anti\u2010Depressant Book: A Practical Guide for Teens and Young Adults to Overcome Depression and Stay Healthy","author":"Towery J","year":"2016"},{"key":"e_1_2_9_15_1","doi-asserted-by":"publisher","DOI":"10.2196\/mental.9782"},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1176\/appi.books.9780890425596"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40596-016-0627-7"},{"key":"e_1_2_9_18_1","unstructured":"LinY.Taiwan LLaMa."},{"key":"e_1_2_9_19_1","unstructured":"ShevchukV.GPT\u20104 Parameters Explained: Everything you Need to Know.2023."},{"key":"e_1_2_9_20_1","unstructured":"HitakshiT.Google Bard Training Data Parameters \u2013 Is it 1.56 trillion?2023."},{"key":"e_1_2_9_21_1","unstructured":"MetaAI.Introducing LLaMA: A foundational 65\u2010billion\u2010parameter large language model.2023."},{"key":"e_1_2_9_22_1","unstructured":"NostaJ.The Left Brain Right Brain Dynamics of LLMs.2023."},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.1097\/JCMA.0000000000000946"},{"key":"e_1_2_9_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03308-9"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.2196\/48002"},{"key":"e_1_2_9_26_1","first-page":"23536290.v2","volume-title":"Chatgpt vs. Bard: A Comparative Study","author":"Ahmed I","year":"2023"},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(23)01129-7"},{"key":"e_1_2_9_28_1","first-page":"07257","article-title":"Chatcad: Interactive computer\u2010aided diagnosis on medical image using large language models","volume":"2302","author":"Wang S","year":"2023","journal-title":"arXiv preprint arXiv:2302.07257"}],"container-title":["Psychiatry and Clinical Neurosciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/pcn.13656","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,3]],"date-time":"2024-06-03T08:35:52Z","timestamp":1717403752000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/pcn.13656"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,26]]},"references-count":27,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["10.1111\/pcn.13656"],"URL":"https:\/\/doi.org\/10.1111\/pcn.13656","archive":["Portico"],"relation":{},"ISSN":["1323-1316","1440-1819"],"issn-type":[{"value":"1323-1316","type":"print"},{"value":"1440-1819","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,26]]},"assertion":[{"value":"2023-09-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-05","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}