{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T07:35:40Z","timestamp":1782804940054,"version":"3.54.5"},"reference-count":53,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2016,10,4]],"date-time":"2016-10-04T00:00:00Z","timestamp":1475539200000},"content-version":"vor","delay-in-days":811,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2015,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Objectives (1) To develop an automated eligibility screening (ES) approach for clinical trials in an urban tertiary care pediatric emergency department (ED); (2) to assess the effectiveness of natural language processing (NLP), information extraction (IE), and machine learning (ML) techniques on real-world clinical data and trials.<\/jats:p>\n               <jats:p>Data and methods We collected eligibility criteria for 13 randomly selected, disease-specific clinical trials actively enrolling patients between January 1, 2010 and August 31, 2012. In parallel, we retrospectively selected data fields including demographics, laboratory data, and clinical notes from the electronic health record (EHR) to represent profiles of all 202795 patients visiting the ED during the same period. Leveraging NLP, IE, and ML technologies, the automated ES algorithms identified patients whose profiles matched the trial criteria to reduce the pool of candidates for staff screening. The performance was validated on both a physician-generated gold standard of trial\u2013patient matches and a reference standard of historical trial\u2013patient enrollment decisions, where workload, mean average precision (MAP), and recall were assessed.<\/jats:p>\n               <jats:p>Results Compared with the case without automation, the workload with automated ES was reduced by 92% on the gold standard set, with a MAP of 62.9%. The automated ES achieved a 450% increase in trial screening efficiency. The findings on the gold standard set were confirmed by large-scale evaluation on the reference set of trial\u2013patient matches.<\/jats:p>\n               <jats:p>Discussion and conclusion By exploiting the text of trial criteria and the content of EHRs, we demonstrated that NLP-, IE-, and ML-based automated ES could successfully identify patients for clinical trials.<\/jats:p>","DOI":"10.1136\/amiajnl-2014-002887","type":"journal-article","created":{"date-parts":[[2014,7,17]],"date-time":"2014-07-17T02:27:40Z","timestamp":1405564060000},"page":"166-178","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":98,"title":["Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department"],"prefix":"10.1093","volume":"22","author":[{"given":"Yizhao","family":"Ni","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephanie","family":"Kennebeck","sequence":"additional","affiliation":[{"name":"Division of Pediatric Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Judith W","family":"Dexheimer","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"},{"name":"Division of Pediatric Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Constance M","family":"McAneney","sequence":"additional","affiliation":[{"name":"Division of Pediatric Emergency Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huaxiu","family":"Tang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Todd","family":"Lingren","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haijun","family":"Zhai","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Imre","family":"Solti","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"},{"name":"James M Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2014,7,16]]},"reference":[{"key":"2020110613021164600_R1","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/1472-6947-8-13","article-title":"Physicians\u2019 perceptions of an electronic health record-based clinical trial alert approach to subject recruitment: a survey","volume":"8","author":"Embi","year":"2008","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613021164600_R2","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1197\/jamia.M3119","article-title":"Electronic screening improves efficiency in clinical trial recruitment","volume":"16","author":"Thadani","year":"2009","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R3","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1197\/jamia.M3005","article-title":"Clinical research informatics: challenges, opportunities and definition for an emerging domain","volume":"16","author":"Embi","year":"2009","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R4","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1200\/JOP.2012.000646","article-title":"Effort required in eligibility screening for clinical trials","volume":"8","author":"Penberthy","year":"2012","journal-title":"J Oncol Pract"},{"key":"2020110613021164600_R5","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1007\/s11060-011-0536-2","article-title":"Competing for patients: an ethical framework for recruiting patients with brain tumors into clinical trials","volume":"104","author":"Ibrahim","year":"2011","journal-title":"J Neurooncol"},{"key":"2020110613021164600_R6","doi-asserted-by":"crossref","first-page":"558","DOI":"10.1016\/j.cct.2010.08.007","article-title":"Recruitment of infants with sickle cell anemia to a Phase III trial: data from the BABY HUG study","volume":"31","author":"Wynn","year":"2010","journal-title":"Contemp Clin Trials"},{"key":"2020110613021164600_R7","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1038\/nrd1382","article-title":"Key factors in the rising cost of new drug discovery and development","volume":"5","author":"Dickson","year":"2004","journal-title":"Nat Rev Drug Discov"},{"key":"2020110613021164600_R8","first-page":"111","article-title":"Enrolling patients into clinical trials faster using real time recuiting","volume":"2000","author":"Butte","year":"2000","journal-title":"Proc AMIA Symp"},{"key":"2020110613021164600_R9","first-page":"231","article-title":"Development of an electronic health record-based clinical trial alert system to enhance recruitment at the point of care","volume":"2005","author":"Embi","year":"2005","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R10","first-page":"289","article-title":"Research subject enrollment by primary care pediatricians using an electronic health record","volume":"2007","author":"Grundmeier","year":"2007","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R11","first-page":"476","article-title":"Enhancing an existing clinical information system to improve study recruitment and census gathering efficiency","volume":"2009","author":"Nkoy","year":"2009","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R12","first-page":"51","article-title":"Desktop software to identify patients eligible for recruitment into a clinical trial: using SARMA to recruit to the ROAD feasibility trial","volume":"18","author":"Treweek","year":"2010","journal-title":"Inform Prim Care"},{"key":"2020110613021164600_R13","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/1471-2288-11-16","article-title":"A clinical trial alert tool to recruit large patient samples and assess selection bias in general practice research","volume":"11","author":"Heinemann","year":"2011","journal-title":"BMC Med Res Methodol"},{"key":"2020110613021164600_R14","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1186\/1472-6947-12-47","article-title":"Computational challenges and human factors influencing the design and use of clinical research participant eligibility pre-screening tools","volume":"12","author":"Pressler","year":"2012","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613021164600_R15","first-page":"931","article-title":"The design of a pre-encounter clinical trial screening tool: ASAP","author":"Ding","year":"2008","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R16","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1016\/j.cct.2010.03.005","article-title":"Automated matching software for clinical trials eligibility: measuring efficiency and flexibility","volume":"31","author":"Penberthy","year":"2010","journal-title":"Contemp Clin Trials"},{"key":"2020110613021164600_R17","doi-asserted-by":"crossref","first-page":"2272","DOI":"10.1001\/archinte.165.19.2272","article-title":"Effect of a clinical trial alert system on physician participation in trial recruitment","volume":"165","author":"Embi","year":"2005","journal-title":"Arch Intern Med"},{"key":"2020110613021164600_R18","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1177\/1740774511434844","article-title":"Efficacy and cost-effectiveness of an automated screening algorithm in an inpatient clinical trial","volume":"9","author":"Beauharnais","year":"2012","journal-title":"Clin Trials."},{"key":"2020110613021164600_R19","doi-asserted-by":"crossref","first-page":"1370","DOI":"10.1177\/1535370213508172","article-title":"Automated determination of metastases in unstructured radiology reports for eligibility screening in oncology clinical trials","volume":"238","author":"Petkov","year":"2013","journal-title":"Exp Biol Med."},{"issue":"(e1)","key":"2020110613021164600_R20","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1136\/amiajnl-2011-000743","article-title":"Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study","volume":"19","author":"Embi","year":"2012","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R21","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1136\/jamia.2009.001024","article-title":"A systematic literature review of automated clinical coding and classification systems","volume":"17","author":"Stanfill","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R22","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/j.jbi.2009.12.004","article-title":"Formal representation of eligibility criteria: a literature review","volume":"43","author":"Weng","year":"2010","journal-title":"J Biomed Inform"},{"key":"2020110613021164600_R23","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1016\/j.jbi.2011.06.001","article-title":"Dynamic categorization of clinical research eligibility criteria by hierarchical clustering","volume":"44","author":"Luo","year":"2011","journal-title":"J Biomed Inform"},{"issue":"(Suppl 1)","key":"2020110613021164600_R24","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1136\/amiajnl-2011-000321","article-title":"EliXR: an approach to eligibility criteria extraction and representation","volume":"18","author":"Weng","year":"2011","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R25","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.jbi.2010.09.007","article-title":"A practical method for transforming free-text eligibility criteria into computable criteria","volume":"44","author":"Tu","year":"2011","journal-title":"J Biomed Inform"},{"issue":"(Suppl 1)","key":"2020110613021164600_R26","doi-asserted-by":"crossref","first-page":"S3","DOI":"10.1186\/1472-6947-12-S1-S3","article-title":"ASCOT: a text mining-based web-service for efficient search and assisted creation of clinical trials","volume":"12","author":"Korkontzelos","year":"2012","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613021164600_R27","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1016\/j.jbi.2013.06.001","article-title":"Analysis of eligibility criteria representation in industry-standard clinical trial protocols","volume":"46","author":"Bhattacharya","year":"2013","journal-title":"J Biomed Inform."},{"key":"2020110613021164600_R28"},{"key":"2020110613021164600_R29","article-title":"A semantic platform for informational retrieval from E-Health records","author":"Gurulingappa","year":"2011"},{"key":"2020110613021164600_R30","article-title":"Cengage Learning at TREC 2011 medical track","author":"King","year":"2011"},{"key":"2020110613021164600_R31","article-title":"University of Glasgow at medical records track 2011: Experiments with Terrier","author":"Limsopatham","year":"2011"},{"key":"2020110613021164600_R32","article-title":"A knowledge-based approach to medical records retrieval","author":"Demner-Fushman","year":"2011"},{"key":"2020110613021164600_R33","article-title":"Exploiting domain thesaurus for medical record retrieval","author":"Callejas","year":"2012"},{"key":"2020110613021164600_R34","article-title":"University of Glasgow at TREC 2012: Experiments with Terrier in Medical Records, Microblog, and Web Tracks","author":"Limsophatham","year":"2012"},{"key":"2020110613021164600_R35","article-title":"Retrieving medical records with \u201csennamed\u201d: NEC labs America at TREC 2012 medical records track","author":"Qi","year":"2012"},{"key":"2020110613021164600_R36","article-title":"Carterette B. Exploring evidence aggregation methods and external expansion sources for medical record search","author":"Zhu","year":"2012"},{"key":"2020110613021164600_R37","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1136\/amiajnl-2011-000465","article-title":"Overcoming barriers to NLP for clinical text: the role of shared tasks and the need for additional creative solutions","volume":"18","author":"Chapman","year":"2011","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R38","first-page":"180","article-title":"Barriers to retrieving patient information from electronic health record data: failure analysis from the TREC medical records track","volume":"2012","author":"Edinger","year":"2012","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R39","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.1016\/j.rmed.2011.04.012","article-title":"The accuracy and efficiency of electronic screening for recruitment into a clinical trial on COPD","volume":"105","author":"Schmickl","year":"2011","journal-title":"Respir Med"},{"key":"2020110613021164600_R40","article-title":"Constructing evaluation corpora for automated clinical named entity recognition","author":"Ogren","year":"2008"},{"key":"2020110613021164600_R41","article-title":"Generating typed dependency parses from phrase structure parses","author":"De Marneffe","year":"2006"},{"key":"2020110613021164600_R42","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613021164600_R43"},{"key":"2020110613021164600_R44"},{"key":"2020110613021164600_R45"},{"key":"2020110613021164600_R46","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1136\/amiajnl-2012-001487","article-title":"A sequence labeling approach to link medications and their attributes in clinical notes and clinical trial announcements for information extraction","volume":"20","author":"Li","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613021164600_R47","first-page":"144","article-title":"Building gold standard corpora for medical natural language processing tasks","volume":"2012","author":"Deleger","year":"2012","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613021164600_R48","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1006\/jbin.2001.1029","article-title":"A simple algorithm for identifying negated findings and diseases in discharge summaries","volume":"34","author":"Champman","year":"2001","journal-title":"J Biomed Inform"},{"key":"2020110613021164600_R49","first-page":"944","article-title":"Improve feature selection approach TFIDF in text mining","author":"Jing","year":"2002"},{"key":"2020110613021164600_R50"},{"key":"2020110613021164600_R51","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1108\/eb026526","article-title":"A statistical interpretation of term specificity and its application in retrieval","volume":"28","author":"Jones","year":"1972","journal-title":"Journal of Documentation"},{"key":"2020110613021164600_R52","volume-title":"Modern information retrieval","author":"Baeza-Yates","year":"1999"},{"key":"2020110613021164600_R53","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1136\/bmj.310.6973.170","article-title":"Multiple significance tests: the Bonferroni method","volume":"310","author":"Bland","year":"1995","journal-title":"BMJ"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/1\/166\/34145283\/amiajnl-2014-002887.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/1\/166\/34145283\/amiajnl-2014-002887.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:53:05Z","timestamp":1604688785000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/22\/1\/166\/834337"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,7,16]]},"references-count":53,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2014,7,16]]},"published-print":{"date-parts":[[2015,1,1]]}},"URL":"https:\/\/doi.org\/10.1136\/amiajnl-2014-002887","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2015,1]]},"published":{"date-parts":[[2014,7,16]]}}}