{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T15:23:32Z","timestamp":1786116212713,"version":"build-2736575974"},"reference-count":19,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T00:00:00Z","timestamp":1785456000000},"content-version":"vor","delay-in-days":24,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Columbia Irving Institute for Clinical and Translational Research Training KL2 Mentored Career Development","award":["KL2TR001874"],"award-info":[{"award-number":["KL2TR001874"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,7,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>We extracted a validated disease activity measure in rheumatoid arthritis (RA), the Clinical Disease Activity Index (CDAI), from a large tertiary academic medical center electronic health record (EHR) using an automated large language model (LLM)-based approach without requiring model pretraining.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>The New York Presbyterian\/Columbia University Medical Center Clinical Data Warehouse contains EHR data for over 4.5 million patients. RA patients were identified using International Classification of Disease-9 (ICD-9) and ICD-10 codes. Expert-curated CDAI keywords were extracted from unstructured notes using an automated natural language processing (NLP) pipeline leveraging GPT-4o API, a HIPAA-compliant, institutionally approved LLM platform. Performance was evaluated against expert chart review.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Among 2756 RA patients with notes, 1038 (37.7%) were seropositive, 796 (28.9%) were seronegative, and 922 (33.4%) had unknown serostatus. Clinical Disease Activity Index and its components were extracted in 15.4% (160\/1038) of seropositive patients indicating remission or low disease activity. Clinical Disease Activity Index documentation was more frequent among patients with multiple notes and among faculty, with high extraction accuracy (precision\/recall\/F1\u2009=\u20090.97).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>This represents the first attempt to employ a zero-shot, ChatGPT-powered LLM platform to extract RA disease activity measures from real-world EHR data. Although a low prevalence of documentation was noted, important distinctions were observed when patients were subgrouped by serostatus, level of training, and number of visits.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>An LLM-based pipeline accurately extracted CDAI from a single large academic EHR, revealing infrequent real-world documentation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooag152","type":"journal-article","created":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T11:48:29Z","timestamp":1784116109000},"source":"Crossref","is-referenced-by-count":0,"title":["Large language model-based extraction of rheumatoid arthritis clinical disease activity index from clinical notes"],"prefix":"10.1093","volume":"9","author":[{"given":"Reid","family":"Weisberg","sequence":"first","affiliation":[{"name":"Division of Rheumatology, Department of Medicine, UT Southwestern Medical Center , Dallas, TX 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruoqi","family":"Yang","sequence":"additional","affiliation":[{"name":"Data Science Institute, Columbia University , New York, NY 10027,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Iram","family":"Kamdar","sequence":"additional","affiliation":[{"name":"Data Science Institute, Columbia University , New York, NY 10027,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Justin","family":"Chestang","sequence":"additional","affiliation":[{"name":"Columbia University Vagelos College of Physicians & Surgeons , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joan","family":"Bathon","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Department of Medicine, Columbia University Medical Center , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8792-0402","authenticated-orcid":false,"given":"Jon T","family":"Giles","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Kao Autoimmunity Institute, Cedars-Sinai Medical Center , Los Angeles, CA 90048,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9624-0214","authenticated-orcid":false,"given":"Chunhua","family":"Weng","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Columbia University , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5277-349X","authenticated-orcid":false,"given":"Elizabeth","family":"Park","sequence":"additional","affiliation":[{"name":"Division of Rheumatology, Department of Medicine, Columbia University Medical Center , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,30]]},"reference":[{"key":"2026080604265243400_ooag152-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/acr.22783","article-title":"2015 American College of Rheumatology guideline for the treatment of rheumatoid arthritis: ACR RA treatment recommendations","volume":"68","author":"Singh","year":"2016","journal-title":"Arthritis Care & Research"},{"key":"2026080604265243400_ooag152-B2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1136\/annrheumdis-2015-207524","article-title":"Treating rheumatoid arthritis to target: 2014 update of the recommendations of an international task force","volume":"75","author":"Smolen","year":"2016","journal-title":"Ann Rheum Dis"},{"issue":"Suppl. 3","key":"2026080604265243400_ooag152-B3","doi-asserted-by":"crossref","first-page":"iii56","DOI":"10.1136\/ard.2007.078360","article-title":"Tight control in the treatment of rheumatoid arthritis: efficacy and feasibility","volume":"66","author":"Bakker","year":"2007","journal-title":"Annals of the Rheumatic Diseases"},{"key":"2026080604265243400_ooag152-B4","doi-asserted-by":"crossref","first-page":"1540","DOI":"10.1002\/acr.24042","article-title":"2019 Update of the American College of Rheumatology recommended rheumatoid arthritis disease activity measures","volume":"71","author":"England","year":"2019","journal-title":"Arthritis Care Res (Hoboken)"},{"key":"2026080604265243400_ooag152-B5","doi-asserted-by":"publisher","first-page":"2969","DOI":"10.1038\/s41591-023-02700-1","article-title":"Six ways large language models are changing healthcare","volume":"29","author":"Webster","year":"2023","journal-title":"Nat Med"},{"key":"2026080604265243400_ooag152-B6","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: a systematic review","volume":"333","author":"Bedi","year":"2025","journal-title":"JAMA"},{"key":"2026080604265243400_ooag152-B7","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1001\/jama.2023.14217","article-title":"Creation and adoption of large language models in medicine","volume":"330","author":"Shah","year":"2023","journal-title":"JAMA"},{"key":"2026080604265243400_ooag152-B8","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1002\/acr.24869","article-title":"Development of a natural language processing system for extracting rheumatoid arthritis outcomes from clinical notes using the National Rheumatology Informatics System for Effectiveness Registry","volume":"75","author":"Humbert-Droz","year":"2023","journal-title":"Arthritis Care & Research"},{"key":"2026080604265243400_ooag152-B9","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1002\/acr2.11089","article-title":"Extraction of rheumatoid arthritis disease activity measures from electronic health records using automated processing algorithms","volume":"1","author":"Cannon","year":"2019","journal-title":"ACR Open Rheumatol"},{"key":"2026080604265243400_ooag152-B10","first-page":"75","article-title":"Scores for all seasons: SDAI and CDAI","volume":"32","author":"Smolen","year":"2014","journal-title":"Clin Exp Rheumatol"},{"key":"2026080604265243400_ooag152-B11","doi-asserted-by":"crossref","first-page":"40","DOI":"10.3389\/fmed.2018.00040","article-title":"Common evaluations of disease activity in rheumatoid arthritis reach discordant classifications across different populations","volume":"5","author":"Canh\u00e3o","year":"2018","journal-title":"Front Med (Lausanne)"},{"key":"2026080604265243400_ooag152-B12","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1136\/annrheumdis-2020-eular.1377","article-title":"SAT0093 CDAI and DAS28 in the management of rheumatoid arthritis in clinical practice","volume":"79","author":"Takanashi","year":"2020","journal-title":"Ann Rheum Dis"},{"key":"2026080604265243400_ooag152-B13","doi-asserted-by":"crossref","first-page":"e17088","DOI":"10.1097\/MD.0000000000017088","article-title":"High-resolution computed tomography of the lung in patients with rheumatoid arthritis: prevalence of interstitial lung disease involvement and determinants of abnormalities","volume":"98","author":"Salaffi","year":"2019","journal-title":"Medicine"},{"key":"2026080604265243400_ooag152-B14","author":"Columbia University Information Technology (CUIT)"},{"key":"2026080604265243400_ooag152-B15","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1002\/acr.20184","article-title":"Electronic medical records for discovery research in rheumatoid arthritis","volume":"62","author":"Liao","year":"2010","journal-title":"Arthritis Care Res (Hoboken)"},{"key":"2026080604265243400_ooag152-B16","first-page":"acr.70011","article-title":"Long-term outcomes in seronegative rheumatoid arthritis","author":"Kimbrough","year":"2025","journal-title":"Arthritis Care Res (Hoboken)"},{"key":"2026080604265243400_ooag152-B17","first-page":"649","author":"Xie"},{"key":"2026080604265243400_ooag152-B18","first-page":"2023","author":"Zhong","year":"2022"},{"key":"2026080604265243400_ooag152-B19","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1111\/nyas.15007","article-title":"Holistic evaluation of language models","volume":"1525","author":"Bommasani","year":"2023","journal-title":"Ann N Y Acad Sci"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag152\/70405894\/ooag152.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag152\/70405894\/ooag152.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T08:27:01Z","timestamp":1786004821000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag152\/8748000"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,7]]},"references-count":19,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,7,7]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooag152","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,8]]},"published":{"date-parts":[[2026,7,7]]},"article-number":"ooag152"}}