{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T00:54:51Z","timestamp":1775868891653,"version":"3.50.1"},"reference-count":22,"publisher":"Oxford University Press (OUP)","issue":"4","funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Objective:To develop automated classification methods for eligibility criteria in ClinicalTrials.gov to facilitate patient-trial matching for specific populations such as persons living with HIV or pregnant women.<\/jats:p><jats:p>Materials and Methods:We annotated 891 interventional cancer trials from ClinicalTrials.gov based on their eligibility for human immunodeficiency virus (HIV)-positive patients using their eligibility criteria. These annotations were used to develop classifiers based on regular expressions and machine learning (ML). After evaluating classification of cancer trials for eligibility of HIV-positive patients, we sought to evaluate the generalizability of our approach to more general diseases and conditions. We annotated the eligibility criteria for 1570 of the most recent interventional trials from ClinicalTrials.gov for HIV-positive and pregnancy eligibility, and the classifiers were retrained and reevaluated using these data.<\/jats:p><jats:p>Results:On the cancer-HIV dataset, the baseline regex model, the bag-of-words ML classifier, and the ML classifier with named entity recognition (NER) achieved macro-averaged F2 scores of 0.77, 0.87, and 0.87, respectively; the addition of NER did not result in a significant performance improvement. On the general dataset, ML + NER achieved macro-averaged F2 scores of 0.91 and 0.85 for HIV and pregnancy, respectively.<\/jats:p><jats:p>Discussion and Conclusion:The eligibility status of specific patient populations, such as persons living with HIV and pregnant women, for clinical trials is of interest to both patients and clinicians. We show that it is feasible to develop a high-performing, automated trial classification system for eligibility status that can be integrated into consumer-facing search engines as well as patient-trial matching systems.<\/jats:p>","DOI":"10.1093\/jamia\/ocw176","type":"journal-article","created":{"date-parts":[[2016,12,3]],"date-time":"2016-12-03T12:07:43Z","timestamp":1480766863000},"page":"781-787","source":"Crossref","is-referenced-by-count":44,"title":["Automated classification of eligibility criteria in clinical trials to facilitate patient-trial matching for specific patient populations"],"prefix":"10.1093","volume":"24","author":[{"given":"Kevin","family":"Zhang","sequence":"first","affiliation":[{"name":"College of Medicine and Life Sciences, University of Toledo, Toledo, OH, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dina","family":"Demner-Fushman","sequence":"additional","affiliation":[{"name":"Lister Hill National Center for Biomedical Communications, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2017,2,19]]},"reference":[{"key":"2020110612443042400_ocw176-B1","volume-title":"ClinicalTrials.gov: Policy Updates in Trial Registration and Results Reporting","author":"Williams","year":"2016"},{"key":"2020110612443042400_ocw176-B2","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/S0895-4356(00)00305-X","article-title":"Are randomized clinical trials good for us (in the short term)? Evidence for a \u2018trial effect.\u2019","volume":"54","author":"Braunholtz","year":"2001","journal-title":"J Clin Epidemiol."},{"key":"2020110612443042400_ocw176-B3","doi-asserted-by":"crossref","first-page":"352","DOI":"10.1038\/bjc.1994.306","article-title":"Centralised treatment, entry to trials and survival","volume":"70","author":"Stiller","year":"1994","journal-title":"Br J Cancer."},{"key":"2020110612443042400_ocw176-B4","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1197\/jamia.M2233","article-title":"Essie: a concept-based search engine for structured biomedical text","volume":"14","author":"Ide","year":"2007","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612443042400_ocw176-B5","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1016\/j.jbi.2013.06.001","article-title":"Analysis of eligibility criteria representation in industry-standard clinical trial protocols","volume":"46","author":"Bhattacharya","year":"2013","journal-title":"J Biomed Inform."},{"key":"2020110612443042400_ocw176-B6","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1006\/jbin.2001.1029","article-title":"A Simple Algorithm for Identifying Negated Findings and Diseases in Discharge Summaries","volume":"34","author":"Chapman","year":"2001","journal-title":"J Biomed Inform."},{"key":"2020110612443042400_ocw176-B7","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1200\/JCO.2007.14.5532","article-title":"Including persons with HIV infection in cancer clinical trials","volume":"26","author":"Persad","year":"2008","journal-title":"J Clin Oncol."},{"key":"2020110612443042400_ocw176-B8","doi-asserted-by":"crossref","first-page":"1429","DOI":"10.1089\/jwh.2011.3118","article-title":"Clinical research enrolling pregnant women: a workshop summary","volume":"20","author":"Foulkes","year":"2011","journal-title":"J Womens Health (Larchmt)."},{"key":"2020110612443042400_ocw176-B9","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1197\/jamia.M3119","article-title":"Electronic screening improves efficiency in clinical trial recruitment","volume":"16","author":"Thadani","year":"2009","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612443042400_ocw176-B10","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1200\/JOP.2012.000646","article-title":"Effort required in eligibility screening for clinical trials","volume":"8","author":"Penberthy","year":"2012","journal-title":"J Oncol Pract."},{"key":"2020110612443042400_ocw176-B11","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1186\/s12911-015-0149-3","article-title":"Increasing the efficiency of trial-patient matching: automated clinical trial eligibility Pre-screening for pediatric oncology patients","volume":"15","author":"Ni","year":"2015","journal-title":"BMC Med Inform Decis Mak."},{"key":"2020110612443042400_ocw176-B12","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1136\/amiajnl-2014-002887","article-title":"Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department","volume":"22","author":"Ni","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612443042400_ocw176-B13","doi-asserted-by":"crossref","first-page":"157","DOI":"10.4137\/CIN.S19454","article-title":"Trial prospector: Matching patients with cancer research studies using an automated and scalable approach","volume":"13","author":"Sahoo","year":"2014","journal-title":"Cancer Inform."},{"key":"2020110612443042400_ocw176-B14","doi-asserted-by":"crossref","first-page":"e141","DOI":"10.1093\/jamia\/ocu050","article-title":"Case-based reasoning using electronic health records efficiently identifies eligible patients for clinical trials","volume":"22","author":"Miotto","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612443042400_ocw176-B15","doi-asserted-by":"crossref","DOI":"10.1109\/ICMLC.2002.1174522","article-title":"Improved feature selection approach TFIDF in text mining","volume-title":"Proceedings, International Conference on Machine Learning and Cybernetics","author":"Li-Ping Jing"},{"key":"2020110612443042400_ocw176-B16","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2012","journal-title":"Mach Learn."},{"key":"2020110612443042400_ocw176-B17","first-page":"1871","article-title":"LIBLINEAR: A Library for Large Linear Classification","volume":"9","author":"Fan","year":"2008","journal-title":"J Mach Learn Res."},{"key":"2020110612443042400_ocw176-B18","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1136\/jamia.2009.002733","article-title":"An overview of MetaMap: historical perspective and recent advances","volume":"17","author":"Aronson","year":"2010","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612443042400_ocw176-B19","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1145\/1143844.1143874","article-title":"The relationship between Precision-Recall and ROC curves","volume-title":"Proceedings of the 23rd International Conference on Machine Learning 2006","author":"Davis","year":"2006"},{"key":"2020110612443042400_ocw176-B20","doi-asserted-by":"crossref","first-page":"e92209","DOI":"10.1371\/journal.pone.0092209","article-title":"Area under precision-recall curves for weighted and unweighted data","volume":"9","author":"Keilwagen","year":"2014","journal-title":"PLoS One."},{"key":"2020110612443042400_ocw176-B21","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1111\/j.1466-8238.2007.00358.x","article-title":"AUC: a misleading measure of the performance of predictive distribution models","volume":"17","author":"Lobo","year":"2008","journal-title":"Glob Ecol Biogeogr."},{"key":"2020110612443042400_ocw176-B22","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1197\/jamia.M1733","article-title":"Agreement, the f-measure, and reliability in information retrieval","volume":"12","author":"Hripcsak","year":"2005","journal-title":"J Am Med Inform Assoc."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/4\/781\/34148965\/ocw176.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/4\/781\/34148965\/ocw176.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,16]],"date-time":"2022-07-16T03:48:06Z","timestamp":1657943286000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/4\/781\/3038205"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,19]]},"references-count":22,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2017,2,19]]},"published-print":{"date-parts":[[2017,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocw176","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,7]]},"published":{"date-parts":[[2017,2,19]]}}}