{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T08:20:50Z","timestamp":1793866850214,"version":"4.6.1"},"reference-count":27,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T00:00:00Z","timestamp":1626998400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["acrjournals.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["ACR Open Rheumatology"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Efficiently identifying eligible patients is a crucial first step for a successful clinical trial. The objective of this study was to test whether an approach using electronic health record (EHR) data and an ensemble machine learning algorithm incorporating billing codes and data from clinical notes processed by natural language processing (NLP) can improve the efficiency of eligibility screening.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      We studied patients screened for a clinical trial of rheumatoid arthritis (RA) with one or more International Classification of Diseases (ICD) code for RA and age greater than 35 years, from a tertiary care center and a community hospital. The following three groups of EHR features were considered for the algorithm:\n                      <jats:italic>1<\/jats:italic>\n                      ) structured features,\n                      <jats:italic>2<\/jats:italic>\n                      ) the counts of NLP concepts from notes,\n                      <jats:italic>3<\/jats:italic>\n                      ) health care utilization. All features were linked to dates. We applied random forest and logistic regression with least absolute shrinkage and selection operator penalty against the following two standard approaches:\n                      <jats:italic>1<\/jats:italic>\n                      ) one or more RA ICD code and no ICD codes related to exclusion criteria (Screen\n                      <jats:sub>RAICD1<\/jats:sub>\n                      <jats:sub>+EX<\/jats:sub>\n                      ) and\n                      <jats:italic>2<\/jats:italic>\n                      ) two or more RA ICD codes (Screen\n                      <jats:sub>RAICD2<\/jats:sub>\n                      ). To test the portability, we trained the algorithm at one institution and tested it at the other.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      In total, 3359 patients at Brigham and Women\u2019s Hospital (BWH) and 642 patients at Faulkner Hospital (FH) were studied, with 461 (13.7%) eligible patients at BWH and 84 (13.4%) at FH. The application of the algorithm reduced ineligible patients from chart review by 40.5% at the tertiary care center and by 57.0% at the community hospital. In contrast, Screen\n                      <jats:sub>RAICD2<\/jats:sub>\n                      reduced patients for chart review by 2.7% to 11.3%; Screen\n                      <jats:sub>RAICD1+EX<\/jats:sub>\n                      reduced patients for chart review by 63% to 65% but excluded 22% to 27% of eligible patients.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The ensemble machine learning algorithm incorporating billing codes and NLP data increased the efficiency of eligibility screening by reducing the number of patients requiring chart review while not excluding eligible patients. Moreover, this approach can be trained at one institution and applied at another for multicenter clinical trials.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/acr2.11289","type":"journal-article","created":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T05:20:31Z","timestamp":1627017631000},"page":"593-600","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Improving the Efficiency of Clinical Trial Recruitment Using an Ensemble Machine Learning to Assist With Eligibility Screening"],"prefix":"10.1002","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5772-7460","authenticated-orcid":false,"given":"Tianrun","family":"Cai","sequence":"first","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fiona","family":"Cai","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology  Cambridge Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kumar P.","family":"Dahal","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabrielle","family":"Cremone","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ethan","family":"Lam","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charlotte","family":"Golnik","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thany","family":"Seyok","sequence":"additional","affiliation":[{"name":"Brigham and Women's Hospital  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chuan","family":"Hong","sequence":"additional","affiliation":[{"name":"Harvard University  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianxi","family":"Cai","sequence":"additional","affiliation":[{"name":"Harvard University  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4797-3200","authenticated-orcid":false,"given":"Katherine P.","family":"Liao","sequence":"additional","affiliation":[{"name":"Brigham and Women\u2019s Hospital Harvard University, and Veterans Affairs Boston Healthcare System  Boston Massachusetts United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2021,7,23]]},"reference":[{"key":"e_1_2_6_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(01)00353-5"},{"key":"e_1_2_6_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-6296(02)00126-1"},{"key":"e_1_2_6_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.healthpol.2010.12.002"},{"key":"e_1_2_6_5_1","doi-asserted-by":"publisher","DOI":"10.1038\/nrd1382"},{"key":"e_1_2_6_6_1","doi-asserted-by":"publisher","DOI":"10.9778\/cmajo.20180096"},{"key":"e_1_2_6_7_1","doi-asserted-by":"publisher","DOI":"10.1200\/JOP.2012.000646"},{"key":"e_1_2_6_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cct.2010.08.007"},{"key":"e_1_2_6_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11060-011-0536-2"},{"key":"e_1_2_6_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/1740774514558307"},{"key":"e_1_2_6_11_1","doi-asserted-by":"publisher","DOI":"10.1177\/1740774511434844"},{"key":"e_1_2_6_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cct.2010.03.005"},{"key":"e_1_2_6_13_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-015-0149-3"},{"key":"e_1_2_6_14_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl-2014-002887"},{"key":"e_1_2_6_15_1","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-13-134"},{"key":"e_1_2_6_16_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocu050"},{"key":"e_1_2_6_17_1","first-page":"320","article-title":"The data gap in the ehr for clinical research eligibility screening","volume":"2017","author":"Butler A","year":"2018","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"e_1_2_6_18_1","unstructured":"Brigham and Women's Hospital sponsor.Lipids Inflammation and Cardiovascular Risk in Rheumatoid Arthritis (LiiRA). Clinicaltrials.gov identifier: NCT02714881;2016."},{"key":"e_1_2_6_19_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocw082"},{"key":"e_1_2_6_20_1","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkh061"},{"key":"e_1_2_6_21_1","unstructured":"YuS CaiT CaiT.NILE: Fast natural language processing for electronic health records.2013. URL:https:\/\/arxiv.org\/abs\/1311.6063."},{"key":"e_1_2_6_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41596-019-0227-6"},{"key":"e_1_2_6_23_1","doi-asserted-by":"crossref","first-page":"3426","DOI":"10.1007\/978-1-4419-9326-7","volume-title":"Ensemble machine learning: methods and applications","author":"Zhang C","year":"2012"},{"key":"e_1_2_6_24_1","unstructured":"R core team.R: a language and environment for statistical computing. URL:https:\/\/www.R\u2010project.org\/."},{"key":"e_1_2_6_25_1","first-page":"18","article-title":"Classification and regression by randomforest","volume":"2","author":"Liaw A","year":"2002","journal-title":"R News"},{"key":"e_1_2_6_26_1","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v033.i01"},{"key":"e_1_2_6_27_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.5937"},{"key":"e_1_2_6_28_1","first-page":"1489","article-title":"A real\u2010time screening alert improves patient recruitment efficiency","volume":"2011","author":"Weng C","year":"2011","journal-title":"AMIA Annu Symp Proc"}],"container-title":["ACR Open Rheumatology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acr2.11289","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/acr2.11289","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/acrjournals.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/acr2.11289","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,2]],"date-time":"2025-10-02T17:16:45Z","timestamp":1759425405000},"score":1,"resource":{"primary":{"URL":"https:\/\/acrjournals.onlinelibrary.wiley.com\/doi\/10.1002\/acr2.11289"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,23]]},"references-count":27,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["10.1002\/acr2.11289"],"URL":"https:\/\/doi.org\/10.1002\/acr2.11289","archive":["Portico"],"relation":{},"ISSN":["2578-5745","2578-5745"],"issn-type":[{"value":"2578-5745","type":"print"},{"value":"2578-5745","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,23]]},"assertion":[{"value":"2020-11-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}