{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T14:32:49Z","timestamp":1791988369605,"version":"4.4.0"},"reference-count":48,"publisher":"Oxford University Press (OUP)","issue":"e1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2016,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Objective To evaluate the phenotyping performance of three major electronic health record (EHR) components: International Classification of Disease (ICD) diagnosis codes, primary notes, and specific medications.<\/jats:p>\n               <jats:p>Materials and Methods We conducted the evaluation using de-identified Vanderbilt EHR data. We preselected ten diseases: atrial fibrillation, Alzheimer\u2019s disease, breast cancer, gout, human immunodeficiency virus infection, multiple sclerosis, Parkinson\u2019s disease, rheumatoid arthritis, and types 1 and 2 diabetes mellitus. For each disease, patients were classified into seven categories based on the presence of evidence in diagnosis codes, primary notes, and specific medications. Twenty-five patients per disease category (a total number of 175 patients for each disease, 1750 patients for all ten diseases) were randomly selected for manual chart review. Review results were used to estimate the positive predictive value (PPV), sensitivity, and F -score for each EHR component alone and in combination.<\/jats:p>\n               <jats:p>Results The PPVs of single components were inconsistent and inadequate for accurately phenotyping (0.06\u20130.71). Using two or more ICD codes improved the average PPV to 0.84. We observed a more stable and higher accuracy when using at least two components (mean\u2009\u00b1\u2009standard deviation: 0.91\u2009\u00b1\u20090.08). Primary notes offered the best sensitivity (0.77). The sensitivity of ICD codes was 0.67. Again, two or more components provided a reasonably high and stable sensitivity (0.59\u2009\u00b1\u20090.16). Overall, the best performance ( F score: 0.70\u2009\u00b1\u20090.12) was achieved by using two or more components. Although the overall performance of using ICD codes (0.67\u2009\u00b1\u20090.14) was only slightly lower than using two or more components, its PPV (0.71\u2009\u00b1\u20090.13) is substantially worse (0.91\u2009\u00b1\u20090.08).<\/jats:p>\n               <jats:p>Conclusion Multiple EHR components provide a more consistent and higher performance than a single one for the selected phenotypes. We suggest considering multiple EHR components for future phenotyping design in order to obtain an ideal result.<\/jats:p>","DOI":"10.1093\/jamia\/ocv130","type":"journal-article","created":{"date-parts":[[2015,9,4]],"date-time":"2015-09-04T00:34:06Z","timestamp":1441326846000},"page":"e20-e27","source":"Crossref","is-referenced-by-count":201,"title":["Combining billing codes, clinical notes, and medications from electronic health records provides superior phenotyping performance"],"prefix":"10.1093","volume":"23","author":[{"given":"Wei-Qi","family":"Wei","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro L","family":"Teixeira","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huan","family":"Mo","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert M","family":"Cronin","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"},{"name":"Department of Medicine, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeremy L","family":"Warner","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"},{"name":"Department of Medicine, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua C","family":"Denny","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Vanderbilt University, Nashville, TN, USA"},{"name":"Department of Medicine, Vanderbilt University, Nashville, TN, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2015,9,2]]},"reference":[{"issue":"3","key":"2020110612364285700_ocv130-B1","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1056\/NEJMp0910140","article-title":"Accelerating the use of electronic health records in physician practices","volume":"362","author":"Shea","year":"2010","journal-title":"New Engl J Med."},{"issue":"3","key":"2020110612364285700_ocv130-B2","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1038\/clpt.2010.260","article-title":"The emerging role of electronic medical records in pharmacogenomics","volume":"89","author":"Wilke","year":"2011","journal-title":"Clin Pharmacol Therap."},{"issue":"6","key":"2020110612364285700_ocv130-B3","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1038\/clpt.2012.42","article-title":"Electronic medical records as a tool in clinical pharmacology: opportunities and challenges","volume":"91","author":"Roden","year":"2012","journal-title":"Clin Pharmacol Therap."},{"issue":"1","key":"2020110612364285700_ocv130-B4","first-page":"117","article-title":"Next-generation phenotyping of electronic health records","volume":"20","author":"Hripcsak","year":"2013","journal-title":"JAMIA."},{"issue":"79","key":"2020110612364285700_ocv130-B5","doi-asserted-by":"crossref","first-page":"79re71","DOI":"10.1126\/scitranslmed.3001807","article-title":"Electronic medical records for genetic research: results of the eMERGE consortium","volume":"3","author":"Kho","year":"2011","journal-title":"Sci Trans Med."},{"issue":"e1","key":"2020110612364285700_ocv130-B6","first-page":"e147","article-title":"Validation of electronic medical record-based phenotyping algorithms: results and lessons learned from the eMERGE network","volume":"20","author":"Newton","year":"2013","journal-title":"JAMIA."},{"issue":"5","key":"2020110612364285700_ocv130-B7","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1002\/humu.22080","article-title":"Deep phenotyping for precision medicine","volume":"33","author":"Robinson","year":"2012","journal-title":"Hum Mutat."},{"issue":"2","key":"2020110612364285700_ocv130-B8","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1097\/MOL.0b013e3282f73893","article-title":"\u2018Deep phenotyping': characterizing populations in the era of genomics and systems biology","volume":"19","author":"Tracy","year":"2008","journal-title":"Curr Opin Lipidol."},{"issue":"1","key":"2020110612364285700_ocv130-B9","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1186\/s13073-015-0166-y","article-title":"Extracting research-quality phenotypes from electronic health records to support precision medicine","volume":"7","author":"Wei","year":"2015","journal-title":"Genome Med"},{"key":"2020110612364285700_ocv130-B10","doi-asserted-by":"crossref","DOI":"10.1007\/0-387-36278-9","volume-title":"Biomedical Informatics: Computer Applications in Health Care and Biomedicine","author":"Shortliffe","year":"2006","edition":"3rd ed"},{"issue":"2","key":"2020110612364285700_ocv130-B11","first-page":"219","article-title":"Impact of data fragmentation across healthcare centers on the accuracy of a high-throughput clinical phenotyping algorithm for specifying subjects with type 2 diabetes mellitus","volume":"19","author":"Wei","year":"2012","journal-title":"JAMIA."},{"issue":"4","key":"2020110612364285700_ocv130-B12","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.ijmedinf.2012.05.015","article-title":"The absence of longitudinal data limits the accuracy of high-throughput clinical phenotyping for identifying type 2 diabetes mellitus subjects","volume":"82","author":"Wei","year":"2013","journal-title":"Int J Med Inform."},{"issue":"22","key":"2020110612364285700_ocv130-B13","doi-asserted-by":"crossref","first-page":"1989","DOI":"10.1001\/archinternmed.2010.439","article-title":"Patients treated at multiple acute health care facilities: quantifying information fragmentation","volume":"170","author":"Bourgeois","year":"2010","journal-title":"Arch Intern Med."},{"issue":"2","key":"2020110612364285700_ocv130-B14","first-page":"221","article-title":"A review of approaches to identifying patient phenotype cohorts using electronic health records","volume":"21","author":"Shivade","year":"2014","journal-title":"JAMIA."},{"issue":"1","key":"2020110612364285700_ocv130-B15","first-page":"215","article-title":"Clinical research informatics and electronic health record data","volume":"9","author":"Richesson","year":"2014","journal-title":"Yearbook Med Inform."},{"issue":"1","key":"2020110612364285700_ocv130-B16","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1002\/pds.3367","article-title":"Validation of a coding algorithm to identify patients with hepatocellular carcinoma in an administrative database","volume":"22","author":"Goldberg","year":"2013","journal-title":"Pharmacoepidemiol Drug Safety."},{"issue":"7","key":"2020110612364285700_ocv130-B17","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1002\/pds.3290","article-title":"Validation of three coding algorithms to identify patients with end-stage liver disease in an administrative database","volume":"21","author":"Goldberg","year":"2012","journal-title":"Pharmacoepidemiol Drug Safety."},{"issue":"4","key":"2020110612364285700_ocv130-B18","doi-asserted-by":"crossref","first-page":"141","DOI":"10.24095\/hpcdp.30.4.06","article-title":"Myocardial infarction and the validation of physician billing and hospitalization data using electronic medical records","volume":"30","author":"Tu","year":"2010","journal-title":"Chronic Dis Can."},{"issue":"7","key":"2020110612364285700_ocv130-B19","doi-asserted-by":"crossref","first-page":"e225","DOI":"10.1016\/S0828-282X(10)70412-8","article-title":"Validation of physician billing and hospitalization data to identify patients with ischemic heart disease using data from the Electronic Medical Record Administrative data Linked Database (EMRALD)","volume":"26","author":"Tu","year":"2010","journal-title":"Can J Cardiol."},{"issue":"2","key":"2020110612364285700_ocv130-B20","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1111\/epi.12506","article-title":"Assessing the validity of using administrative data to identify patients with epilepsy","volume":"55","author":"Tu","year":"2014","journal-title":"Epilepsia."},{"issue":"5 Pt 2","key":"2020110612364285700_ocv130-B21","doi-asserted-by":"crossref","first-page":"1620","DOI":"10.1111\/j.1475-6773.2005.00444.x","article-title":"Measuring diagnoses: ICD code accuracy","volume":"40","author":"O'Malley","year":"2005","journal-title":"Health Services Res."},{"issue":"2","key":"2020110612364285700_ocv130-B22","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1111\/j.1475-6773.2005.00482.x","article-title":"Failure of ICD-9-CM codes to identify patients with comorbid chronic kidney disease in diabetes","volume":"41","author":"Kern","year":"2006","journal-title":"Health Services Res."},{"issue":"4","key":"2020110612364285700_ocv130-B23","doi-asserted-by":"crossref","first-page":"682","DOI":"10.2215\/CJN.07650713","article-title":"Performance and limitations of administrative data in the identification of AKI","volume":"9","author":"Grams","year":"2014","journal-title":"CJASN."},{"key":"2020110612364285700_ocv130-B24","volume-title":"The Electronic Medical Records and Genomics (eMERGE) Network","author":"eMERGE","year":"2014"},{"key":"2020110612364285700_ocv130-B25","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1186\/1472-6963-11-37","article-title":"The validity of using ICD-9 codes and pharmacy records to identify patients with chronic obstructive pulmonary disease","volume":"11","author":"Cooke","year":"2011","journal-title":"BMC Health Services Res."},{"issue":"e2","key":"2020110612364285700_ocv130-B26","first-page":"e275","article-title":"Using electronic health records data to identify patients with chronic pain in a primary care setting","volume":"20","author":"Tian","year":"2013","journal-title":"JAMIA."},{"issue":"7","key":"2020110612364285700_ocv130-B27","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1089\/aid.2013.0287","article-title":"Development and validation of an algorithm to identify patients newly diagnosed with HIV infection from electronic health records","volume":"30","author":"Goetz","year":"2014","journal-title":"AIDS Res Hum Retroviruses."},{"issue":"2","key":"2020110612364285700_ocv130-B28","first-page":"212","article-title":"Use of diverse electronic medical record systems to identify genetic risk for type 2 diabetes within a genome-wide association study","volume":"19","author":"Kho","year":"2012","journal-title":"JAMIA."},{"issue":"e1","key":"2020110612364285700_ocv130-B29","first-page":"e162","article-title":"Portability of an algorithm to identify rheumatoid arthritis in electronic health records","volume":"19","author":"Carroll","year":"2012","journal-title":"JAMIA."},{"key":"2020110612364285700_ocv130-B30","article-title":"Creation and validation of an EMR-based algorithm for identifying major adverse cardiac events while on statins","author":"Wei","journal-title":"Joint Summits on Translational Science, AMIA."},{"key":"2020110612364285700_ocv130-B31","first-page":"857","article-title":"A high throughput semantic concept frequency based approach for patient identification: a case study using type 2 diabetes mellitus clinical notes","volume":"2010","author":"Wei","year":"2010","journal-title":"AMIA\u2026 Annual Symposium Proceedings\/AMIA Symposium. AMIA Symposium."},{"issue":"10","key":"2020110612364285700_ocv130-B32","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1097\/GIM.0b013e3181efe2df","article-title":"Assessing the accuracy of observer-reported ancestry in a biorepository linked to electronic medical records","volume":"12","author":"Dumitrescu","year":"2010","journal-title":"Genetics Med."},{"issue":"3","key":"2020110612364285700_ocv130-B33","doi-asserted-by":"crossref","first-page":"362","DOI":"10.1038\/clpt.2008.89","article-title":"Development of a large-scale de-identified DNA biobank to enable personalized medicine","volume":"84","author":"Roden","year":"2008","journal-title":"Clin Pharmacol Therap."},{"issue":"6","key":"2020110612364285700_ocv130-B34","first-page":"806","article-title":"Evaluation of a method to identify and categorize section headers in clinical documents","volume":"16","author":"Denny","year":"2009","journal-title":"JAMIA."},{"key":"2020110612364285700_ocv130-B35","first-page":"172","article-title":"Development of an ensemble resource linking MEDications to their Indications (MEDI)","volume":"2013","author":"Wei","year":"2013","journal-title":"AMIA Summits Transl Sci."},{"key":"2020110612364285700_ocv130-B36","first-page":"1448","article-title":"Validation and Enhancement of a Computable Medication Indication Resource (MEDI) Using a Large Practice-based Dataset","author":"Wei","year":"2013","journal-title":"AMIA\u2026 Annual Symposium Proceedings\/AMIA Symposium."},{"issue":"1","key":"2020110612364285700_ocv130-B37","first-page":"19","article-title":"MedEx: a medication information extraction system for clinical narratives","volume":"17","author":"Xu","year":"2010","journal-title":"JAMIA."},{"issue":"9","key":"2020110612364285700_ocv130-B38","doi-asserted-by":"crossref","first-page":"1206","DOI":"10.1212\/WNL.56.9.1206","article-title":"Neurologists' use of ICD-9CM codes for dementia","volume":"56","author":"Pippenger","year":"2001","journal-title":"Neurology."},{"issue":"12","key":"2020110612364285700_ocv130-B39","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1038\/nbt.2749","article-title":"Systematic comparison of phenome-wide association study of electronic medical record data and genome-wide association study data","volume":"31","author":"Denny","year":"2013","journal-title":"Nat Biotechnol."},{"issue":"2","key":"2020110612364285700_ocv130-B40","first-page":"181","article-title":"Data from clinical notes: a perspective on the tension between structure and flexible documentation","volume":"18","author":"Rosenbloom","year":"2011","journal-title":"JAMIA"},{"issue":"6","key":"2020110612364285700_ocv130-B41","first-page":"571","article-title":"The granularity of medical narratives and its effect on the speed and completeness of information retrieval","volume":"5","author":"Tange","year":"1998","journal-title":"JAMIA."},{"issue":"3","key":"2020110612364285700_ocv130-B42","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1038\/clpt.2013.202","article-title":"Characterization of statin dose response in electronic medical records","volume":"95","author":"Wei","year":"2014","journal-title":"Clin Pharmacol Therap"},{"issue":"3","key":"2020110612364285700_ocv130-B43","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1504\/IJDMB.2008.020522","article-title":"Word Sense Disambiguation in biomedical ontologies with term co-occurrence analysis and document clustering","volume":"2","author":"Andreopoulos","year":"2008","journal-title":"Int J Data Min Bioinform."},{"key":"2020110612364285700_ocv130-B44","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1186\/1471-2105-7-334","article-title":"Machine learning and word sense disambiguation in the biomedical domain: design and evaluation issues","volume":"7","author":"Xu","year":"2006","journal-title":"BMC Bioinformatics."},{"issue":"5","key":"2020110612364285700_ocv130-B45","doi-asserted-by":"crossref","first-page":"728","DOI":"10.1016\/j.jbi.2011.03.011","article-title":"Document-level classification of CT pulmonary angiography reports based on an extension of the ConText algorithm","volume":"44","author":"Chapman","year":"2011","journal-title":"J BiomedI Inform"},{"key":"2020110612364285700_ocv130-B46","first-page":"1441","article-title":"Phenome based analysis as a means for discovering context dependent clinical reference ranges","volume":"2012","author":"Warner","year":"2012","journal-title":"AMIA\u2026 Annual Symposium Proceedings\/AMIA Symposium. AMIA Symposium."},{"key":"2020110612364285700_ocv130-B47","author":"Quick Medical Reference"},{"issue":"1","key":"2020110612364285700_ocv130-B48","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1001\/jama.1987.03400010071030","article-title":"DXplain. An evolving diagnostic decision-support system","volume":"258","author":"Barnett","year":"1987","journal-title":"JAMA."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/e1\/e20\/34147997\/ocv130.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/23\/e1\/e20\/34147997\/ocv130.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T17:46:06Z","timestamp":1604684766000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/23\/e1\/e20\/2379841"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,9,2]]},"references-count":48,"journal-issue":{"issue":"e1","published-online":{"date-parts":[[2015,9,2]]},"published-print":{"date-parts":[[2016,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocv130","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2016,4]]},"published":{"date-parts":[[2015,9,2]]}}}