{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T05:36:22Z","timestamp":1776922582192,"version":"3.51.2"},"reference-count":28,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["TR002062"],"award-info":[{"award-number":["TR002062"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["LM011934"],"award-info":[{"award-number":["LM011934"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["HL136659"],"award-info":[{"award-number":["HL136659"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["AG034676"],"award-info":[{"award-number":["AG034676"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["AG052425"],"award-info":[{"award-number":["AG052425"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Background<\/jats:title>\n<jats:p>The incorporation of information from clinical narratives is critical for computational phenotyping. The accurate interpretation of clinical terms highly depends on their associated context, especially the corresponding clinical section information. However, the heterogeneity across different Electronic Health Record (EHR) systems poses challenges in utilizing the section information.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Objectives<\/jats:title>\n<jats:p>Leveraging the eMERGE heart failure (HF) phenotyping algorithm, we assessed the heterogeneity quantitatively through the performance comparison of machine learning (ML) classifiers which map clinical sections containing HF-relevant terms across different EHR systems to standard sections in Health Level 7 (HL7) Clinical Document Architecture (CDA).<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Methods<\/jats:title>\n<jats:p>We experimented with both random forest models with sentence-embedding features and bidirectional encoder representations from transformers models. We trained MLs using an automated labeled corpus from an EHR system that adopted HL7 CDA standard. We assessed the performance using a blind test set (n = 300) from the same EHR system and a gold standard (n = 900) manually annotated from three other EHR systems.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec004\">\n<jats:title>Results<\/jats:title>\n<jats:p>The F-measure of those ML models varied widely (0.00\u20130.91%), indicating MLs with one tuning parameter set were insufficient to capture sections across different EHR systems. The error analysis indicates that the section does not always comply with the corresponding standardized sections, leading to low performance.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec005\">\n<jats:title>Conclusions<\/jats:title>\n<jats:p>We presented the potential use of ML techniques to map the sections containing HF-relevant terms in multiple EHR systems to standard sections. However, the findings suggested that the quality and heterogeneity of section structure across different EHRs affect applications due to the poor adoption of documentation standards.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pone.0283800","type":"journal-article","created":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T17:59:34Z","timestamp":1680285574000},"page":"e0283800","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["Assessing document section heterogeneity across multiple electronic health record systems for computational phenotyping: A case study of heart-failure phenotyping algorithm"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9191-3897","authenticated-orcid":true,"given":"Sungrim","family":"Moon","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sijia","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bhavani Singh Agnikula","family":"Kshatriya","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1691-5179","authenticated-orcid":true,"given":"Sunyang","family":"Fu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ethan D.","family":"Moser","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suzette J.","family":"Bielinski","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6349-3752","authenticated-orcid":true,"given":"Jungwei","family":"Fan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongfang","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2023,3,31]]},"reference":[{"issue":"2","key":"pone.0283800.ref001","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MCI.2014.2307227","article-title":"Jumping NLP curves: A review of natural language processing research","volume":"9","author":"E Cambria","year":"2014","journal-title":"IEEE Computational intelligence magazine"},{"issue":"3","key":"pone.0283800.ref002","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1093\/jamia\/ocx138","article-title":"Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions","volume":"25","author":"S Sohn","year":"2018","journal-title":"Journal of the American Medical Informatics Association"},{"issue":"8","key":"pone.0283800.ref003","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1007\/s12265-015-9644-2","article-title":"A robust e-epidemiology tool in phenotyping heart failure with differentiation for preserved and reduced ejection fraction: the electronic medical records and genomics (eMERGE) network","volume":"8","author":"SJ Bielinski","year":"2015","journal-title":"Journal of cardiovascular translational research"},{"key":"pone.0283800.ref004","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"J Devlin","year":"2018"},{"issue":"6","key":"pone.0283800.ref005","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1197\/jamia.M3037","article-title":"Evaluation of a method to identify and categorize section headers in clinical documents","volume":"16","author":"JC Denny","year":"2009","journal-title":"Journal of the American Medical Informatics Association"},{"issue":"1","key":"pone.0283800.ref006","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12874-019-0792-y","article-title":"Current approaches to identify sections within clinical narratives from electronic health records: a systematic review","volume":"19","author":"A Pomares-Quimbaya","year":"2019","journal-title":"BMC medical research methodology"},{"key":"pone.0283800.ref007","first-page":"17","volume-title":"The CDA TM book","author":"Springer","year":"2011"},{"issue":"4","key":"pone.0283800.ref008","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1093\/jigpal\/jzs027","article-title":"A semantic approach for fine-grain access control of e-health documents","volume":"21","author":"F Amato","year":"2013","journal-title":"Logic Journal of the IGPL"},{"key":"pone.0283800.ref009","first-page":"821","article-title":"Analyzing operative note structure in development of a section header resource","volume":"216","author":"GB Melton","year":"2015","journal-title":"Studies in health technology and informatics"},{"key":"pone.0283800.ref010","volume-title":"AMIA Annual Symposium Proceedings","author":"PJ Haug","year":"2014"},{"key":"pone.0283800.ref011","volume-title":"2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society","author":"IEEE","year":"2009"},{"key":"pone.0283800.ref012","doi-asserted-by":"crossref","first-page":"100057","DOI":"10.1016\/j.yjbinx.2019.100057","article-title":"A survey of word embeddings for clinical text.","volume":"4","author":"FK Khattak","year":"2019","journal-title":"Journal of Biomedical Informatics: X"},{"issue":"3","key":"pone.0283800.ref013","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1093\/jamia\/ocz200","article-title":"Deep learning in clinical natural language processing: a methodical review","volume":"27","author":"S Wu","year":"2020","journal-title":"Journal of the American Medical Informatics Association"},{"key":"pone.0283800.ref014","first-page":"2015","article-title":"Recognition and evaluation of clinical section headings in clinical documents using token-based formulation with conditional random fields","author":"H-J Dai","year":"2015","journal-title":"BioMed research international"},{"issue":"4","key":"pone.0283800.ref015","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1109\/JBHI.2020.3004072","article-title":"HCET: Hierarchical Clinical Embedding With Topic Modeling on Electronic Health Records for Predicting Future Depression.","volume":"25","author":"Y Meng","year":"2020","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"pone.0283800.ref016","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/W17-2336","volume-title":"Automated preamble detection in dictated medical reports","author":"W Salloum","year":"2017"},{"key":"pone.0283800.ref017","volume-title":"International Conference on Speech and Computer","author":"Springer","year":"2018"},{"key":"pone.0283800.ref018","author":"S Rosenthal","year":"2019"},{"key":"pone.0283800.ref019","first-page":"604","article-title":"Identification of patients with family history of pancreatic cancer-Investigation of an NLP System Portability","volume":"216","author":"S Mehrabi","year":"2015","journal-title":"Studies in health technology and informatics."},{"key":"pone.0283800.ref020","article-title":"Statistical Section Segmentation in Free-Text Clinical Records","author":"M Tepper","year":"2012","journal-title":"Lrec"},{"issue":"11","key":"pone.0283800.ref021","doi-asserted-by":"crossref","first-page":"2636","DOI":"10.1002\/asi.23884","article-title":"Intrainstitutional EHR collections for patient\u2010level information retrieval","volume":"68","author":"S Wu","year":"2017","journal-title":"Journal of the Association for Information Science and Technology"},{"issue":"6","key":"pone.0283800.ref022","doi-asserted-by":"crossref","first-page":"1614","DOI":"10.1093\/ije\/dys195","article-title":"Data resource profile: the Rochester Epidemiology Project (REP) medical records-linkage system","volume":"41","author":"JL St Sauver","year":"2012","journal-title":"International journal of epidemiology"},{"issue":"2","key":"pone.0283800.ref023","doi-asserted-by":"crossref","first-page":"368-j","DOI":"10.1093\/ije\/dyx268","article-title":"Data resource profile: expansion of the Rochester Epidemiology Project medical records-linkage system (E-REP).","volume":"47","author":"WA Rocca","year":"2018","journal-title":"International journal of epidemiology"},{"issue":"6","key":"pone.0283800.ref024","doi-asserted-by":"crossref","first-page":"e044353","DOI":"10.1136\/bmjopen-2020-044353","article-title":"Longitudinal cohorts for harnessing the electronic health record for disease prediction in a US population","volume":"11","author":"SM Manemann","year":"2021","journal-title":"BMJ Open"},{"key":"pone.0283800.ref025","article-title":"Predicting Section Location of Clinical Sentences using BERT Encoder-A Pilot Study","author":"S Liu","year":"2020","journal-title":"AMIA"},{"issue":"3","key":"pone.0283800.ref026","doi-asserted-by":"crossref","first-page":"316","DOI":"10.3122\/jabfm.2015.03.140244","article-title":"Physician information needs and electronic health records (EHRs): time to reengineer the clinic note","volume":"28","author":"RJ Koopman","year":"2015","journal-title":"The Journal of the American Board of Family Medicine"},{"key":"pone.0283800.ref027","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/W19-5006","article-title":"Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets","author":"Y Peng","year":"2019"},{"key":"pone.0283800.ref028","volume-title":"AMIA Annual Symposium Proceedings","author":"ES Chen","year":"2011"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0283800","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T17:59:57Z","timestamp":1680285597000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0283800"}},"subtitle":[],"editor":[{"given":"William","family":"Speier","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":28,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3,31]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0283800","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,31]]}}}