{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:58:07Z","timestamp":1776113887857,"version":"3.50.1"},"reference-count":59,"publisher":"Oxford University Press (OUP)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2015,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Objective Analysis of narrative (text) data from electronic health records (EHRs) can improve population-scale phenotyping for clinical and genetic research. Currently, selection of text features for phenotyping algorithms is slow and laborious, requiring extensive and iterative involvement by domain experts. This paper introduces a method to develop phenotyping algorithms in an unbiased manner by automatically extracting and selecting informative features, which can be comparable to expert-curated ones in classification accuracy.<\/jats:p><jats:p>Materials and methods Comprehensive medical concepts were collected from publicly available knowledge sources in an automated, unbiased fashion. Natural language processing (NLP) revealed the occurrence patterns of these concepts in EHR narrative notes, which enabled selection of informative features for phenotype classification. When combined with additional codified features, a penalized logistic regression model was trained to classify the target phenotype.<\/jats:p><jats:p>Results The authors applied our method to develop algorithms to identify patients with rheumatoid arthritis and coronary artery disease cases among those with rheumatoid arthritis from a large multi-institutional EHR. The area under the receiver operating characteristic curves (AUC) for classifying RA and CAD using models trained with automated features were 0.951 and 0.929, respectively, compared to the AUCs of 0.938 and 0.929 by models trained with expert-curated features.<\/jats:p><jats:p>Discussion Models trained with NLP text features selected through an unbiased, automated procedure achieved comparable or slightly higher accuracy than those trained with expert-curated features. The majority of the selected model features were interpretable.<\/jats:p><jats:p>Conclusion The proposed automated feature extraction method, generating highly accurate phenotyping algorithms with improved efficiency, is a significant step toward high-throughput phenotyping.<\/jats:p>","DOI":"10.1093\/jamia\/ocv034","type":"journal-article","created":{"date-parts":[[2015,5,1]],"date-time":"2015-05-01T00:31:54Z","timestamp":1430440314000},"page":"993-1000","source":"Crossref","is-referenced-by-count":133,"title":["Toward high-throughput phenotyping: unbiased automated feature extraction and selection from knowledge sources"],"prefix":"10.1093","volume":"22","author":[{"given":"Sheng","family":"Yu","sequence":"first","affiliation":[{"name":"Partners HealthCare Personalized Medicine, Boston, MA, USA"},{"name":"Brigham and Women\u2019s Hospital, Boston, MA, USA"},{"name":"Harvard Medical School, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katherine P","family":"Liao","sequence":"additional","affiliation":[{"name":"Brigham and Women\u2019s Hospital, Boston, MA, USA"},{"name":"Harvard Medical School, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stanley Y","family":"Shaw","sequence":"additional","affiliation":[{"name":"Massachusetts General Hospital, Boston, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vivian S","family":"Gainer","sequence":"additional","affiliation":[{"name":"Research Computing, Partners HealthCare, Charlestown, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Susanne E","family":"Churchill","sequence":"additional","affiliation":[{"name":"Research Computing, Partners HealthCare, Charlestown, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Szolovits","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shawn N","family":"Murphy","sequence":"additional","affiliation":[{"name":"Massachusetts General Hospital, Boston, MA"},{"name":"Research Computing, Partners HealthCare, Charlestown, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Isaac S.","family":"Kohane","sequence":"additional","affiliation":[{"name":"Harvard Medical School, Boston, MA, USA"},{"name":"Boston Children\u2019s Hospital, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianxi","family":"Cai","sequence":"additional","affiliation":[{"name":"Harvard T.H. Chan School of Public Health, Boston, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2015,4,29]]},"reference":[{"key":"2020110613001408000_ocv034-B1","author":"Charles"},{"key":"2020110613001408000_ocv034-B2","doi-asserted-by":"crossref","first-page":"4401","DOI":"10.1002\/sim.5620","article-title":"Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership","volume":"31","author":"Ryan","year":"2012","journal-title":"Stat Med."},{"key":"2020110613001408000_ocv034-B3","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1038\/clpt.2011.83","article-title":"Detecting drug interactions from adverse-event reports: interaction between paroxetine and pravastatin increases blood glucose levels","volume":"90","author":"Tatonetti","year":"2011","journal-title":"Clin Pharmacol Ther."},{"key":"2020110613001408000_ocv034-B4","doi-asserted-by":"crossref","first-page":"f288","DOI":"10.1136\/bmj.f288","article-title":"QT interval and antidepressant use: a cross sectional study of electronic health records","volume":"346","author":"Castro","year":"2013","journal-title":"BMJ."},{"key":"2020110613001408000_ocv034-B5","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1002\/pds.3413","article-title":"Comparative effectiveness research using electronic health records: impacts of oral antidiabetic drugs on the development of chronic kidney disease","volume":"22","author":"Masica","year":"2013","journal-title":"Pharmacoepidemiol Drug Saf."},{"key":"2020110613001408000_ocv034-B6","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s00592-008-0090-3","article-title":"The risk of developing coronary artery disease or congestive heart failure, and overall mortality, in type 2 diabetic patients receiving rosiglitazone, pioglitazone, metformin, or sulfonylureas: a retrospective analysis","volume":"46","author":"Pantalone","year":"2009","journal-title":"Acta Diabetol."},{"key":"2020110613001408000_ocv034-B7","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1111\/j.1464-5491.2012.03577.x","article-title":"The risk of overall mortality in patients with Type 2 diabetes receiving different combinations of sulfonylureas and metformin: a retrospective analysis","volume":"29","author":"Pantalone","year":"2012","journal-title":"Diabet Med."},{"key":"2020110613001408000_ocv034-B8","doi-asserted-by":"crossref","first-page":"d1642","DOI":"10.1136\/bmj.d1642","article-title":"Effect of statin treatment on short term mortality after pneumonia episode: cohort study","volume":"342","author":"Douglas","year":"2011","journal-title":"BMJ."},{"key":"2020110613001408000_ocv034-B9","author":"Stakic"},{"key":"2020110613001408000_ocv034-B10","doi-asserted-by":"crossref","first-page":"1453","DOI":"10.1016\/j.jpsychires.2011.06.012","article-title":"Substance use disorders and comorbid Axis I and II psychiatric disorders among young psychiatric patients: findings from a large electronic health records database","volume":"45","author":"Wu","year":"2011","journal-title":"J Psychiatr Res."},{"key":"2020110613001408000_ocv034-B11","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1038\/nrg2999","article-title":"Using electronic health records to drive discovery in disease genomics","volume":"12","author":"Kohane","year":"2011","journal-title":"Nat Rev Genet."},{"key":"2020110613001408000_ocv034-B12","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1002\/art.37801","article-title":"Associations of autoantibodies, autoimmune risk alleles, and clinical diagnoses from the electronic medical records in rheumatoid arthritis cases and non\u2013rheumatoid arthritis controls","volume":"65","author":"Liao","year":"2013","journal-title":"Arthritis Rheum."},{"key":"2020110613001408000_ocv034-B13","doi-asserted-by":"crossref","first-page":"1205","DOI":"10.1093\/bioinformatics\/btq126","article-title":"PheWAS: demonstrating the feasibility of a phenome-wide scan to discover gene\u2013disease associations","volume":"26","author":"Denny","year":"2010","journal-title":"Bioinformatics."},{"key":"2020110613001408000_ocv034-B14","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1016\/j.ajhg.2011.09.008","article-title":"Variants near FOXE1 are associated with hypothyroidism and other thyroid conditions: using electronic medical records for genome- and phenome-wide studies","volume":"89","author":"Denny","year":"2011","journal-title":"Am J Hum Genet."},{"key":"2020110613001408000_ocv034-B15","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1038\/nbt.2749","article-title":"Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association Study Data","volume":"31","author":"Denny","year":"2013","journal-title":"Nat Biotechnol."},{"key":"2020110613001408000_ocv034-B16","doi-asserted-by":"crossref","DOI":"10.1161\/CIRCULATIONAHA.112.000604","article-title":"Genome- and Phenome-Wide Analysis of Cardiac Conduction Identifies Markers of Arrhythmia Risk","author":"Ritchie","year":"2013","journal-title":"Circulation"},{"key":"2020110613001408000_ocv034-B17","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1136\/amiajnl-2010-000061","article-title":"Mapping clinical phenotype data elements to standardized metadata repositories and controlled terminologies: the eMERGE Network experience","volume":"18","author":"Pathak","year":"2011","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B18","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.ajhg.2010.12.007","article-title":"Genetic basis of autoantibody positive and negative rheumatoid arthritis risk in a multi-ethnic cohort derived from electronic health records","volume":"88","author":"Kurreeman","year":"2011","journal-title":"Am J Hum Genet."},{"key":"2020110613001408000_ocv034-B19","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1212\/WNL.49.3.660","article-title":"Inaccuracy of the international classification of diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease","volume":"49","author":"Benesch","year":"1997","journal-title":"Neurology."},{"key":"2020110613001408000_ocv034-B20","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1097\/01.mlr.0000160417.39497.a9","article-title":"Accuracy of ICD-9-CM codes for identifying cardiovascular and stroke risk factors","volume":"43","author":"Birman-Deych","year":"2005","journal-title":"Med Care."},{"key":"2020110613001408000_ocv034-B21","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.thromres.2010.03.009","article-title":"Evaluation of the predictive value of ICD-9-CM coded administrative data for venous thromboembolism in the United States","volume":"126","author":"White","year":"2010","journal-title":"Thromb Res."},{"key":"2020110613001408000_ocv034-B22","first-page":"326","article-title":"The validity of ICD-9-CM codes in identifying postoperative deep vein thrombosis and pulmonary embolism","volume":"33","author":"Zhan","year":"2007","journal-title":"Jt Comm J Qual Patient Saf."},{"key":"2020110613001408000_ocv034-B23","author":"Lindberg"},{"key":"2020110613001408000_ocv034-B24","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1186\/1755-8794-4-13","article-title":"The eMERGE Network: A consortium of biorepositories linked to electronic medical records data for conducting genomic studies","volume":"4","author":"McCarty","year":"2011","journal-title":"BMC Med Genomics."},{"key":"2020110613001408000_ocv034-B25","first-page":"274","article-title":"Analyzing the heterogeneity and complexity of electronic health record oriented phenotyping algorithms","volume":"2011","author":"Conway","year":"2011","journal-title":"AMIA Annu Symp Proc."},{"key":"2020110613001408000_ocv034-B26","doi-asserted-by":"crossref","first-page":"e147","DOI":"10.1136\/amiajnl-2012-000896","article-title":"Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network","volume":"20","author":"Newton","year":"2013","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B27","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1097\/MIB.0b013e31828133fd","article-title":"Improving case definition of crohn's disease and ulcerative colitis in electronic medical records using natural language processing: a novel informatics approach","volume":"19","author":"Ananthakrishnan","year":"2013","journal-title":"Inflamm Bowel Dis."},{"key":"2020110613001408000_ocv034-B28","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1002\/acr.20184","article-title":"Electronic medical records for discovery research in rheumatoid arthritis","volume":"62","author":"Liao","year":"2010","journal-title":"Arthritis Care Res."},{"key":"2020110613001408000_ocv034-B29","doi-asserted-by":"crossref","DOI":"10.1016\/S0735-1097(14)61359-0","article-title":"Natural language processing improves phenotypic accuracy in an electronic medical record cohort of type 2 diabetes and cardiovascular disease","volume":"63","author":"Kumar","year":"2014","journal-title":"J Am Coll Cardiol."},{"key":"2020110613001408000_ocv034-B30","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1016\/j.semarthrit.2010.05.002","article-title":"Validation of psoriatic arthritis diagnoses in electronic medical records using natural language processing","volume":"40","author":"Love","year":"2011","journal-title":"Semin Arthritis Rheum."},{"key":"2020110613001408000_ocv034-B31","doi-asserted-by":"crossref","first-page":"e162","DOI":"10.1136\/amiajnl-2011-000583","article-title":"Portability of an algorithm to identify rheumatoid arthritis in electronic health records","volume":"19","author":"Carroll","year":"2012","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B32","doi-asserted-by":"crossref","first-page":"e78927","DOI":"10.1371\/journal.pone.0078927","article-title":"Modeling disease severity in multiple sclerosis using electronic health records","volume":"8","author":"Xia","year":"2013","journal-title":"PLoS ONE."},{"key":"2020110613001408000_ocv034-B33","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/0306-4573(88)90021-0","article-title":"Term-weighting approaches in automatic text retrieval","volume":"24","author":"Salton","year":"1988","journal-title":"Inf Process Manag."},{"key":"2020110613001408000_ocv034-B34","author":"Blois"},{"key":"2020110613001408000_ocv034-B35","author":"Gordon"},{"key":"2020110613001408000_ocv034-B36","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1136\/amiajnl-2011-000121","article-title":"A method and knowledge base for automated inference of patient problems from structured data in an electronic medical record","volume":"18","author":"Wright","year":"2011","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B37","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1016\/j.jbi.2010.09.009","article-title":"An automated technique for identifying associations between medications, laboratory results and problems","volume":"43","author":"Wright","year":"2010","journal-title":"J Biomed Inform."},{"key":"2020110613001408000_ocv034-B38"},{"key":"2020110613001408000_ocv034-B39","author":"Pakhomov"},{"key":"2020110613001408000_ocv034-B40","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res."},{"key":"2020110613001408000_ocv034-B41","doi-asserted-by":"crossref","DOI":"10.1136\/amiajnl-2011-000752","article-title":"Pneumonia identification using statistical feature selection","author":"Bejan","year":"2012","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B42","first-page":"189","article-title":"Na\u00efve electronic health record phenotype identification for rheumatoid arthritis","volume":"2011","author":"Carroll","year":"2011","journal-title":"AMIA Annu Symp Proc."},{"key":"2020110613001408000_ocv034-B43","article-title":"Latent Semantic Indexing of medical diagnoses using UMLS semantic structures","volume":"185","author":"Chute","year":"1991","journal-title":"Proc Annu Symp Comput Appl Med Care."},{"key":"2020110613001408000_ocv034-B44","article-title":"An evaluation of concept based latent semantic indexing for clinical information retrieval","volume":"639","author":"Chute","year":"1992","journal-title":"Proc Annu Symp Comput Appl Med Care."},{"key":"2020110613001408000_ocv034-B45","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1023\/A:1023984205118","article-title":"Knowledge-enhanced latent semantic indexing","volume":"6","author":"Guo","year":"2003","journal-title":"Inf Retr."},{"key":"2020110613001408000_ocv034-B46","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1136\/jamia.2009.002733","article-title":"An overview of MetaMap: historical perspective and recent advances","volume":"17","author":"Aronson","year":"2010","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B47","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B48","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1197\/jamia.M1176","article-title":"\u2018Understanding\u2019 medical school curriculum content using KnowledgeMap","volume":"10","author":"Denny","year":"2003","journal-title":"J Am Med Inform Assoc."},{"key":"2020110613001408000_ocv034-B49"},{"key":"2020110613001408000_ocv034-B50","author":"Yu"},{"key":"2020110613001408000_ocv034-B51"},{"key":"2020110613001408000_ocv034-B52","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the Elastic Net","volume":"67","author":"Zou","year":"2005","journal-title":"J R Stat Soc Ser B."},{"key":"2020110613001408000_ocv034-B53","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1214\/08-AOS625","article-title":"On the adaptive elastic-net with a diverging number of parameters","volume":"37","author":"Zou","year":"2009","journal-title":"Ann Stat."},{"key":"2020110613001408000_ocv034-B54","author":"Hastie"},{"key":"2020110613001408000_ocv034-B55","doi-asserted-by":"crossref","first-page":"S9","DOI":"10.1016\/j.amjmed.2008.06.011","article-title":"Cardiovascular morbidity and mortality in rheumatoid arthritis","volume":"121","author":"Gabriel","year":"2008","journal-title":"Am J Med."},{"key":"2020110613001408000_ocv034-B56","doi-asserted-by":"crossref","first-page":"1690","DOI":"10.1002\/art.24092","article-title":"Risk of cardiovascular mortality in patients with rheumatoid arthritis: a meta-analysis of observational studies","volume":"59","author":"Avi\u00f1a-Zubieta","year":"2008","journal-title":"Arthritis Care Res."},{"key":"2020110613001408000_ocv034-B57","doi-asserted-by":"crossref","first-page":"1608","DOI":"10.1136\/ard.2005.050377","article-title":"Patterns of cardiovascular risk in rheumatoid arthritis","volume":"65","author":"Solomon","year":"2006","journal-title":"Ann Rheum Dis."},{"key":"2020110613001408000_ocv034-B58","first-page":"548","article-title":"Improvements on cross-validation: The 632+ Bootstrap Method","volume":"92","author":"Efron","year":"1997","journal-title":"J Am Stat Assoc."},{"key":"2020110613001408000_ocv034-B59","doi-asserted-by":"crossref","first-page":"5320","DOI":"10.1002\/sim.2968","article-title":"A comparison of bootstrap methods and an adjusted bootstrap approach for estimating the prediction error in microarray classification","volume":"26","author":"Jiang","year":"2007","journal-title":"Stat Med."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/5\/993\/34146486\/ocv034.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/5\/993\/34146486\/ocv034.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,6]],"date-time":"2022-05-06T22:17:50Z","timestamp":1651875470000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/22\/5\/993\/929941"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,4,29]]},"references-count":59,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,4,29]]},"published-print":{"date-parts":[[2015,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocv034","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2015,9]]},"published":{"date-parts":[[2015,4,29]]}}}