{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T17:30:47Z","timestamp":1762363847863,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T00:00:00Z","timestamp":1762300800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background: Large Language Models (LLMs), such as ChatGPT, are increasingly applied in medicine for summarization, clinical decision support, and diagnostic assistance, including recent work in dermatology. Previous AI and NLP models in dermatology have mainly focused on lesion classification, diagnostic support, and patient education, while extracting structured treatment information from unstructured dermatology records remains underexplored. We evaluated ChatGPT-4o\u2019s ability to identify psoriasis treatments from free-text documentation, compared with expert annotations. Methods: In total, 94 electronic medical records (EMRs) of patients diagnosed with psoriasis were analyzed. ChatGPT-4o extracted treatments used for psoriasis from each unstructured clinical note. Its output was compared to manually curated reference annotations by expert dermatologists. A total of 83 treatments, including topical agents, systemic medications, biologics, phototherapy, and procedural interventions, were evaluated. Performance metrics included recall, precision, F1-score, specificity, accuracy, Cohen\u2019s Kappa, and Area Under the Curve (AUC). Analyses were conducted at the individual-treatment level and grouped into pharmacologic categories. Results: ChatGPT-4o demonstrated strong performance, with recall of 0.91, precision of 0.96, F1-score of 0.94, specificity of 0.99, and accuracy of 0.99. Agreement with expert annotations was high (Cohen\u2019s Kappa = 0.93; AUC = 0.98). Group-level analysis confirmed these results, with the highest performance in biologics and methotrexate (F1 = 1.00) and lower recall in categories with vague documentation, such as systemic corticosteroids and antihistamines. Conclusions: Our study highlights the potential of LLMs to extract psoriasis treatment information from unstructured clinical documentation and structure it for research and decision support. The model performed best with well-defined, commonly used treatments.<\/jats:p>","DOI":"10.3390\/jcm14217845","type":"journal-article","created":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T17:06:06Z","timestamp":1762362366000},"page":"7845","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["One Step Closer to Conversational Medical Records: ChatGPT Parses Psoriasis Treatments from EMRs"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5154-7984","authenticated-orcid":false,"given":"Jonathan","family":"Shapiro","sequence":"first","affiliation":[{"name":"Maccabi Healthcare Services, Ramat Hasharon 4702633, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7020-7224","authenticated-orcid":false,"given":"Mor","family":"Atlas","sequence":"additional","affiliation":[{"name":"Ono Academic, Business Administration Faculty, One Academic Boulevard, Kiryat Ono 5510701, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sharon","family":"Baum","sequence":"additional","affiliation":[{"name":"Dermatology Department, Sheba Medical Center, Ramat-Gan 5262000, Israel"},{"name":"Gray Faculty of Medicine, Tel-Aviv University, Tel-Aviv 6997801, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Felix","family":"Pavlotsky","sequence":"additional","affiliation":[{"name":"Dermatology Department, Sheba Medical Center, Ramat-Gan 5262000, Israel"},{"name":"Gray Faculty of Medicine, Tel-Aviv University, Tel-Aviv 6997801, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aviv","family":"Barzilai","sequence":"additional","affiliation":[{"name":"Dermatology Department, Sheba Medical Center, Ramat-Gan 5262000, Israel"},{"name":"Gray Faculty of Medicine, Tel-Aviv University, Tel-Aviv 6997801, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4206-7117","authenticated-orcid":false,"given":"Rotem","family":"Gershon","sequence":"additional","affiliation":[{"name":"Dermatology Department, Sheba Medical Center, Ramat-Gan 5262000, Israel"},{"name":"Gray Faculty of Medicine, Tel-Aviv University, Tel-Aviv 6997801, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-8777-8316","authenticated-orcid":false,"given":"Romi","family":"Gleicher","sequence":"additional","affiliation":[{"name":"Ruth and Bruce Rappaport Faculty of Medicine, Technion\u2014Institute of Technology, Haifa 3200003, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4827-7075","authenticated-orcid":false,"given":"Itay","family":"Cohen","sequence":"additional","affiliation":[{"name":"Rutgers School of Public Health, Rutgers University, Piscataway, NJ 08854, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"870","DOI":"10.1016\/j.jaad.2023.05.081","article-title":"ChatGPT for healthcare providers and patients: Practical implications within dermatology","volume":"89","author":"Jin","year":"2023","journal-title":"J. Am. Acad. Dermatol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1016\/j.clindermatol.2024.06.019","article-title":"The state of artificial intelligence for systemic dermatoses: Background and applications for psoriasis, systemic sclerosis, and much more","volume":"42","author":"Ravipati","year":"2024","journal-title":"Clin. Dermatol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Goktas, P., and Grzybowski, A. (2024). Assessing the Impact of ChatGPT in Dermatology: A Comprehensive Rapid Review. J. Clin. Med., 13.","DOI":"10.3390\/jcm13195909"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Khamaysi, Z., Awwad, M., Jiryis, B., Bathish, N., and Shapiro, J. (2025). The Role of ChatGPT in Dermatology Diagnostics. Diagnostics, 15.","DOI":"10.3390\/diagnostics15121529"},{"key":"ref_5","first-page":"311","article-title":"The use of a ChatGPT-4-based chatbot in teledermatology: A retrospective exploratory study","volume":"23","author":"Shapiro","year":"2025","journal-title":"J. Dtsch. Dermatol. Ges."},{"key":"ref_6","first-page":"e43812","article-title":"ChatGPT: A Supplemental Tool for Efficiency and Improved Communication in Rural Dermatology","volume":"15","author":"Baker","year":"2023","journal-title":"Cureus"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101","DOI":"10.3390\/dermatopathology11010009","article-title":"Skin and Syntax: Large Language Models in Dermatopathology","volume":"11","author":"Shah","year":"2024","journal-title":"Dermatopathology"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2225","DOI":"10.1111\/jdv.20286","article-title":"Natural language processing in dermatology: A systematic literature review and state of the art","volume":"38","author":"Paganelli","year":"2024","journal-title":"J. Eur. Acad. Dermatol. Venereol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.semarthrit.2010.05.002","article-title":"Validation of psoriatic arthritis diagnoses in electronic medical records using natural language processing","volume":"40","author":"Love","year":"2011","journal-title":"Semin. Arthritis Rheum."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1007","DOI":"10.1093\/jamia\/ocv180","article-title":"Extracting information from the text of electronic medical records to improve case detection: A systematic review","volume":"23","author":"Ford","year":"2016","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/j.clindermatol.2024.06.017","article-title":"The potential role and restrictions of artificial intelligence in medical school dermatology education","volume":"42","author":"Perrin","year":"2024","journal-title":"Clin. Dermatol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1016\/j.clindermatol.2024.06.018","article-title":"Application of an NLP AI Tool in Psoriasis: A Cross-Sectional Comparative Study on Identifying Affected Areas in Patients\u2019 Data","volume":"42","author":"Shapiro","year":"2024","journal-title":"Clin. Dermatol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1055\/s-0038-1638585","article-title":"Biomedical ontologies in action: Role in knowledge management, data integration and decision support","volume":"17","author":"Bodenreider","year":"2008","journal-title":"Yearb. Med. Inform."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.jbi.2017.11.011","article-title":"Clinical information extraction applications: A literature review","volume":"77","author":"Wang","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1006\/jbin.2001.1029","article-title":"A simple algorithm for identifying negated findings and diseases in discharge summaries","volume":"34","author":"WChapman","year":"2001","journal-title":"J. Biomed. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1136\/amiajnl-2013-001622","article-title":"Comprehensive temporal information detection from clinical text: Medical events, time, and TLINK identification","volume":"20","author":"Sohn","year":"2013","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3566","DOI":"10.1007\/s00330-023-10384-x","article-title":"Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports","volume":"34","author":"Hasani","year":"2024","journal-title":"Eur. Radiol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3852","DOI":"10.1038\/s41598-025-86890-3","article-title":"Evaluating GPT models for clinical note de-identification","volume":"15","author":"Altalla","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1038\/s41746-024-01079-8","article-title":"A critical assessment of using ChatGPT for extracting structured data from clinical notes","volume":"7","author":"Huang","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"ooae060","DOI":"10.1093\/jamiaopen\/ooae060","article-title":"Leveraging GPT-4 for identifying cancer phenotypes in electronic health records: A performance comparison between GPT-4, GPT-3.5-turbo, Flan-T5, Llama-3-8B, and spaCy\u2019s rule-based and machine learning-based methods","volume":"7","author":"Bhattarai","year":"2024","journal-title":"JAMIA Open"},{"key":"ref_21","unstructured":"Nori, H., McKinney, S.M., Carignan, D., and Horvitz, E. (2023). Capabilities of GPT-4 on Medical Challenge Problems. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e35","DOI":"10.1016\/S2589-7500(24)00246-2","article-title":"The potential of Generative Pre-trained Transformer 4 (GPT-4) to analyse medical notes in three different languages: A retrospective model-evaluation study","volume":"7","author":"Menezes","year":"2025","journal-title":"Lancet Digit. Health"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1001\/jamainternmed.2024.8138","article-title":"Trends in Physician Electronic Health Record Time and Message Volume","volume":"185","author":"Holmgren","year":"2025","journal-title":"JAMA Intern. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e65656","DOI":"10.2196\/65656","article-title":"Assessing the Impact on Electronic Health Record Burden After Five Years of Physician Engagement in a Canadian Mental Health Organization: Mixed-Methods Study","volume":"12","author":"Tajirian","year":"2025","journal-title":"JMIR Hum. Factors"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Blankemeier","year":"2024","journal-title":"Nat. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e68998","DOI":"10.2196\/68998","article-title":"Scientific Evidence for Clinical Text Summarization Using Large Language Models: Scoping Review","volume":"27","author":"Bednarczyk","year":"2025","journal-title":"J. Med. Internet. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1038\/s44401-024-00011-2","article-title":"Current and future state of evaluation of large language models for medical summarization tasks","volume":"2","author":"Croxford","year":"2025","journal-title":"npj Health Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e70093","DOI":"10.1111\/cts.70093","article-title":"With big data comes big responsibility: Strategies for utilizing aggregated, standardized, de-identified electronic health record data for research","volume":"18","author":"Olaker","year":"2025","journal-title":"Clin. Transl. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1089\/tmj.2020.0069","article-title":"Implementation and Impact of a Store-and-Forward Teledermatology Platform in an Urban Academic Safety-Net Health Care System","volume":"27","author":"Dobry","year":"2021","journal-title":"Telemed. e-Health"},{"key":"ref_30","unstructured":"Newswire, P. (2025, September 25). New Statcounter AI Data Finds ChatGPT Sends 79.8% of All Chatbot Referrals to Websites. PR Newswire. Available online: https:\/\/www.prnewswire.com\/news-releases\/new-statcounter-ai-data-finds-chatgpt-sends-798-of-all-chatbot-referrals-to-websites-301965483.html."},{"key":"ref_31","unstructured":"(2025, September 25). Statcounter, AI Chatbot Market Share Worldwide, Statcounter Global Stats. Available online: https:\/\/gs.statcounter.com\/ai-chatbot-market-share."},{"key":"ref_32","unstructured":"Jiang, A.Q.S., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D.S., de las Casas, D., Hanna, E.B., and Bressand, F. (2024). Mixtral of experts. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"bvae163.1038","DOI":"10.1210\/jendso\/bvae163.1038","article-title":"7543 Comparative Analysis of Language Model Systems In Endocrinology: Performance And Human Acceptability Assessment","volume":"8","author":"Randhawa","year":"2024","journal-title":"J. Endocr. Soc."}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/21\/7845\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T17:25:51Z","timestamp":1762363551000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/21\/7845"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,5]]},"references-count":33,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["jcm14217845"],"URL":"https:\/\/doi.org\/10.3390\/jcm14217845","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,5]]}}}