{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T02:48:28Z","timestamp":1793760508394,"version":"4.6.1"},"reference-count":12,"publisher":"BMJ","issue":"3","license":[{"start":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T00:00:00Z","timestamp":1637712000000},"content-version":"unspecified","delay-in-days":23,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["RMD Open"],"accepted":{"date-parts":[[2021,10,29]]},"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Disease activity measures, such as the Clinical Disease Activity Index (CDAI), are important tools for informing treatment decisions and monitoring patient outcomes in rheumatoid arthritis (RA). Yet, documentation of CDAI scores in electronic medical records and other real-world data sources is inconsistent, making it challenging to use these data for research. The purpose of this study was to validate a machine learning model to estimate CDAI scores for patients with RA using clinical notes.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>A machine learning model was developed to estimate CDAI score values using clinical notes from a specific rheumatology visit. Data from the OM1 RA Registry were used to create a training cohort of 56\u2009177 encounters and a separate validation cohort of 18\u2009726 encounters, 11\u2009985 of which passed a model-derived confidence filter; all included encounters had both a clinician-recorded CDAI score and a clinical note. Model performance was assessed using the area under the receiver operating characteristic curve (AUC), positive predictive value (PPV) and negative predictive value (NPV), calculated using a binarised version of the outcome. The Spearman\u2019s R and Pearson\u2019s R values were also calculated.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The model had a PPV of 0.80, NPV of 0.84 and AUC of 0.88 when evaluating performance using the binarised version of the outcome. The model had a Spearman\u2019s R value of 0.72 and a Pearson\u2019s R value of 0.69 when evaluating performance using the continuous CDAI numeric scores.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>A machine learning model estimates CDAI scores from clinical notes with good performance. Application of the model to real-world data sets may allow estimated CDAI scores to be used for research purposes.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/rmdopen-2021-001781","type":"journal-article","created":{"date-parts":[[2021,11,24]],"date-time":"2021-11-24T12:46:26Z","timestamp":1637757986000},"page":"e001781","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":10,"title":["Validation of a machine learning approach to estimate Clinical Disease Activity Index Scores for rheumatoid arthritis"],"prefix":"10.1136","volume":"7","author":[{"given":"Alison K.","family":"Spencer","sequence":"first","affiliation":[{"name":"Data Science, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jigar","family":"Bandaria","sequence":"additional","affiliation":[{"name":"Data Science, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1927-7248","authenticated-orcid":false,"given":"Michelle B.","family":"Leavy","sequence":"additional","affiliation":[{"name":"Research, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin","family":"Gliklich","sequence":"additional","affiliation":[{"name":"Research, Noble and Greenough School, Dedham, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhaohui","family":"Su","sequence":"additional","affiliation":[{"name":"Biostatistics, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gary","family":"Curhan","sequence":"additional","affiliation":[{"name":"Research, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Costas","family":"Boussios","sequence":"additional","affiliation":[{"name":"Data Science, OM1 Inc, Boston, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2021,11,24]]},"reference":[{"key":"2025090313070697000_7.3.e001781.1","doi-asserted-by":"publisher","DOI":"10.1136\/ard.49.11.916"},{"key":"2025090313070697000_7.3.e001781.2","doi-asserted-by":"publisher","DOI":"10.1002\/acr.21649"},{"key":"2025090313070697000_7.3.e001781.3","doi-asserted-by":"publisher","DOI":"10.1002\/acr.24042"},{"key":"2025090313070697000_7.3.e001781.4","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbspin.2011.04.008"},{"key":"2025090313070697000_7.3.e001781.5","doi-asserted-by":"publisher","DOI":"10.1080\/03009740902932835"},{"key":"2025090313070697000_7.3.e001781.6","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1002\/acr.23848","article-title":"Three quality improvement initiatives and performance of rheumatoid arthritis disease activity measures in electronic health records: results from an interrupted time series study","volume":"72","author":"Gandrup","year":"2020","journal-title":"Arthritis Care Res"},{"key":"2025090313070697000_7.3.e001781.7","doi-asserted-by":"publisher","DOI":"10.1136\/rmdopen-2021-001586"},{"key":"2025090313070697000_7.3.e001781.8","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/s00296-019-04477-4","article-title":"Comparison of an administrative algorithm for SLE disease severity to clinical SLE disease activity index scores","volume":"40","author":"Speyer","year":"2020","journal-title":"Rheumatol Int"},{"key":"2025090313070697000_7.3.e001781.9","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2019.0606","article-title":"Assessment of a deep learning model based on electronic health record data to forecast clinical outcomes in patients with rheumatoid arthritis","volume":"2","author":"Norgeot","year":"2019","journal-title":"JAMA Netw Open"},{"key":"2025090313070697000_7.3.e001781.10","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0069932"},{"key":"2025090313070697000_7.3.e001781.11","doi-asserted-by":"publisher","DOI":"10.1002\/acr.21641"},{"key":"2025090313070697000_7.3.e001781.12","doi-asserted-by":"crossref","first-page":"rkaa005","DOI":"10.1093\/rap\/rkaa005","article-title":"Applied machine learning and artificial intelligence in rheumatology","volume":"4","author":"H\u00fcgle","year":"2020","journal-title":"Rheumatol Adv Pract"}],"container-title":["RMD Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/rmdopen-2021-001781","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T20:07:12Z","timestamp":1756930032000},"score":1,"resource":{"primary":{"URL":"https:\/\/rmdopen.bmj.com\/lookup\/doi\/10.1136\/rmdopen-2021-001781"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11]]},"references-count":12,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,11,24]]},"published-print":{"date-parts":[[2021,11]]}},"alternative-id":["10.1136\/rmdopen-2021-001781"],"URL":"https:\/\/doi.org\/10.1136\/rmdopen-2021-001781","relation":{},"ISSN":["2056-5933"],"issn-type":[{"value":"2056-5933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11]]}}}