{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T12:39:57Z","timestamp":1791463197956,"version":"4.3.4"},"reference-count":24,"publisher":"Georg Thieme Verlag KG","issue":"10","funder":[{"DOI":"10.13039\/100000968","name":"American Heart Association","doi-asserted-by":"crossref","award":["938814"],"award-info":[{"award-number":["938814"]}],"id":[{"id":"10.13039\/100000968","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Thromb Haemost"],"published-print":{"date-parts":[[2026,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Rule-based natural language processing (NLP) tools can identify pulmonary embolism (PE) via radiology reports. However, their external validity remains uncertain.<\/jats:p>\n                  <jats:p>In this cross-sectional study, 1,712 hospitalized patients (with and without PE) at Mass General Brigham (MGB) hospitals (2016\u20132021) were analyzed. Two previously published NLP algorithms were applied to radiology reports to identify PE. Chart review by two physicians was the reference standard. We tested three approaches: (A) NLP applied to all patients; (B) NLP limited to radiology reports of patients with principal or secondary International Classification of Diseases 10th revision (ICD-10) PE discharge codes; and (C) NLP applied to patients with PE discharge codes or a Present-on-Admission (POA) indicator (\u201cY\u201d) for PE. All others were assumed PE-negative in Approaches B and C to minimize NLP false positives. Weighted estimates were derived from the MGB hospitalized cohort (n\u2009=\u2009381,642) to calculate F1 scores (as the harmonic mean of sensitivity and positive predictive value [PPV]).<\/jats:p>\n                  <jats:p>In Approach A, both NLP tools showed high sensitivity (82.5%, 93.0%) and specificity (98.9%, 98.7%) but low PPV (60.3%, 59.6%). Approach B improved PPV (95.2%, 94.9%) but reduced sensitivity (74.1%, 76.2%), while Approach C preserved both high sensitivity (82.5%, 93.0%) and PPV (95.6%, 95.8%). Approach C demonstrated the best performance, yielding significantly higher F1 scores for both NLP tools (88.6%, 94.4%) compared with Approach A (69.7%, 72.6%) and Approach B (83.3%, 84.5%) (P\u2009&lt;\u20090.001).<\/jats:p>\n                  <jats:p>The accuracy of PE detection improves when rule-based NLP algorithms are operationalized using administrative claims data in addition to radiology reports.<\/jats:p>","DOI":"10.1055\/a-2796-1975","type":"journal-article","created":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T00:14:04Z","timestamp":1769645644000},"page":"1049-1056","source":"Crossref","is-referenced-by-count":2,"title":["Optimizing the Accuracy of Natural Language Processing Tools for Pulmonary Embolism Detection Through Integration with Claims Data: The PE-EHR+ Study"],"prefix":"10.1055","volume":"126","author":[{"given":"Sina","family":"Rashedi","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Syed","family":"Bukhari","sequence":"additional","affiliation":[{"name":"Division of Cardiology, Department of Medicine, Johns Hopkins University, Baltimore, Maryland, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darsiya","family":"Krishnathasan","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Candrika D.","family":"Khairani","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antoine","family":"Bejjani","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Department of Internal Medicine, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariana B.","family":"Pfeferman","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4095-9414","authenticated-orcid":false,"given":"Julia","family":"Malejczyk","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mehrdad","family":"Zarghami","sequence":"additional","affiliation":[{"name":"Jamaica Hospital Medical Center, Queens, New York, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric A.","family":"Secemsky","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States"},{"name":"Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farbod N.","family":"Rahaghi","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohamad A.","family":"Hussain","sequence":"additional","affiliation":[{"name":"Division of Vascular and Endovascular Surgery and the Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hamid","family":"Mojibian","sequence":"additional","affiliation":[{"name":"Department of Radiology and Biomedical Imaging, Yale University, New Haven, Connecticut, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel Z.","family":"Goldhaber","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4571-7721","authenticated-orcid":false,"given":"David","family":"Jim\u00e9nez","sequence":"additional","affiliation":[{"name":"Respiratory Department, Hospital Ram\u00f3n y Cajal and Instituto Ram\u00f3n y Cajal de Investigaci\u00f3n Sanitaria (IRYCIS), Madrid, Spain"},{"name":"Medicine Department, Universidad de Alcal\u00e1, Campus Cient\u00edfico-Tecnol\u00f3gico, Crta. de Madrid-Barcelona, Madrid, Spain"},{"name":"CIBER Enfermedades Respiratorias (CIBERES), Instituto de Salud Carlos III, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Monreal","sequence":"additional","affiliation":[{"name":"C\u00e1tedra de Enfermedad Tromboemb\u00f3lica, Universidad Cat\u00f3lica San Antonio de Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Yang","sequence":"additional","affiliation":[{"name":"YNHH\/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Zhou","sequence":"additional","affiliation":[{"name":"YNHH\/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gregory","family":"Piazza","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Harlan M.","family":"Krumholz","sequence":"additional","affiliation":[{"name":"YNHH\/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States"},{"name":"Section of Cardiovascular Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States"},{"name":"Department of Health Policy and Management, Yale School of Public Health, New Haven, Connecticut, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liqin","family":"Wang","sequence":"additional","affiliation":[{"name":"Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Behnood","family":"Bikdeli","sequence":"additional","affiliation":[{"name":"Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States"},{"name":"Department of Health Policy and Management, Yale School of Public Health, New Haven, Connecticut, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2026,2,9]]},"reference":[{"issue":"12","key":"ref1","doi-asserted-by":"crossref","first-page":"1525","DOI":"10.1016\/j.amjcard.2005.02.030","article-title":"Trends in the incidence of pulmonary embolism and deep venous thrombosis in hospitalized patients","volume":"95","author":"P D Stein","year":"2005","journal-title":"Am J Cardiol"},{"issue":"06","key":"ref2","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1001\/jama.2019.8594","article-title":"Pulmonary embolism hospitalization, readmission, and mortality rates in US older adults, 1999-2015","volume":"322","author":"B Bikdeli","year":"2019","journal-title":"JAMA"},{"issue":"05","key":"ref3","doi-asserted-by":"crossref","first-page":"e12520","DOI":"10.1002\/rth2.12520","article-title":"Global reporting of pulmonary embolism-related deaths in the World Health Organization mortality database: vital registration data from 123 countries","volume":"5","author":"S Barco","year":"2021","journal-title":"Res Pract Thromb Haemost"},{"issue":"06","key":"ref4","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1055\/a-2039-3222","article-title":"Developing validated tools to identify pulmonary embolism in electronic databases: rationale and design of the PE-EHR+ study","volume":"123","author":"B Bikdeli","year":"2023","journal-title":"Thromb Haemost"},{"issue":"01","key":"ref5","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1186\/1471-2105-15-266","article-title":"Natural language processing of radiology reports for the detection of thromboembolic diseases and clinically relevant incidental findings","volume":"15","author":"A D Pham","year":"2014","journal-title":"BMC Bioinformatics"},{"issue":"10","key":"ref6","doi-asserted-by":"crossref","first-page":"e73","DOI":"10.1097\/MLR.0000000000000346","article-title":"Automated extraction of VTE events from narrative radiology reports in electronic health records: a validation study","volume":"55","author":"Z Tian","year":"2017","journal-title":"Med Care"},{"issue":"07","key":"ref7","doi-asserted-by":"crossref","first-page":"1052","DOI":"10.1002\/ajh.26928","article-title":"External validation of a novel electronic risk score for cancer-associated thrombosis in a comprehensive cancer center","volume":"98","author":"A Li","year":"2023","journal-title":"Am J Hematol"},{"key":"ref8","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.thromres.2021.11.020","article-title":"Developing and validating natural language processing algorithms for radiology reports compared to ICD-10 codes for identifying venous thromboembolism in hospitalized medical patients","volume":"209","author":"A A Verma","year":"2022","journal-title":"Thromb Res"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.thromres.2021.04.020","article-title":"A comparison of natural language processing to ICD-10 codes for identification and characterization of pulmonary embolism","volume":"203","author":"S A Johnson","year":"2021","journal-title":"Thromb Res"},{"issue":"02","key":"ref11","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1016\/j.jtha.2024.10.013","article-title":"Validating International Classification of Diseases Code 10th Revision algorithms for accurate identification of pulmonary embolism","volume":"23","author":"B Bikdeli","year":"2025","journal-title":"J Thromb Haemost"},{"issue":"04","key":"ref13","doi-asserted-by":"crossref","first-page":"e31","DOI":"10.1097\/MLR.0b013e318286e34f","article-title":"Predictive value of the present-on-admission indicator for hospital-acquired venous thromboembolism","volume":"53","author":"R R Khanna","year":"2015","journal-title":"Med Care"},{"issue":"07","key":"ref14","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1055\/a-2554-0043","article-title":"Use of present-on-admission indicators to improve accuracy of pulmonary embolism identification from electronic health record data","volume":"51","author":"S Rashedi","year":"2025","journal-title":"Semin Thromb Hemost"},{"issue":"04","key":"ref15","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1159\/000353863","article-title":"Diagnostic accuracy measures","volume":"36","author":"P Eusebi","year":"2013","journal-title":"Cerebrovasc Dis"},{"key":"ref16","first-page":"221","volume-title":"Diagnostic test evaluation","author":"Y Mor","year":"2023"},{"key":"ref17","doi-asserted-by":"crossref","first-page":"109271","DOI":"10.1016\/j.thromres.2025.109271","article-title":"Isolated subsegmental pulmonary embolism identification based on international classification of diseases (ICD)-10 codes and imaging reports","volume":"247","author":"S Rashedi","year":"2025","journal-title":"Thromb Res"},{"issue":"02","key":"ref18","doi-asserted-by":"crossref","first-page":"388","DOI":"10.4338\/ACI-2013-12-RA-0100","article-title":"Care everywhere, a point-to-point HIE tool: utilization and impact on patient care in the ED","volume":"5","author":"T J Winden","year":"2014","journal-title":"Appl Clin Inform"},{"key":"ref19","doi-asserted-by":"crossref","first-page":"101845","DOI":"10.1016\/j.jacadv.2025.101845","article-title":"Leveraging hybrid natural language processing techniques for large-scale pulmonary embolism identification","volume":"4","author":"S M Hassan","year":"2025","journal-title":"JACC Adv"},{"issue":"05","key":"ref20","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1093\/jamia\/ocaf048","article-title":"Utilizing large language models for detecting hospital-acquired conditions: an empirical study on pulmonary embolism","volume":"32","author":"C Cheligeer","year":"2025","journal-title":"J Am Med Inform Assoc"},{"issue":"04","key":"ref21","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1053\/j.gastro.2023.12.019","article-title":"A comparison of a large language model vs manual chart review for the extraction of data elements from the electronic health record","volume":"166","author":"J Ge","year":"2024","journal-title":"Gastroenterology"},{"issue":"05","key":"ref22","doi-asserted-by":"crossref","first-page":"782","DOI":"10.1097\/UPJ.0000000000000599","article-title":"Exploring the feasibility of GPT-4 as a data extraction tool for renal surgery operative notes","volume":"11","author":"J Y Hsueh","year":"2024","journal-title":"Urol Pract"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"24311106","DOI":"10.1101\/2024.07.27.24311106","article-title":"Large language models for accurate disease detection in electronic health records","author":"N B\u00fcrgisser","year":"2024","journal-title":"medRxiv"},{"issue":"03","key":"ref24","doi-asserted-by":"crossref","first-page":"ooae060","DOI":"10.1093\/jamiaopen\/ooae060","article-title":"Leveraging GPT-4 for identifying cancer phenotypes in electronic health records: a performance comparison between GPT-4, GPT-3.5-turbo, Flan-T5, Llama-3-8B, and spaCy's rule-based and machine learning-based methods","volume":"7","author":"K Bhattarai","year":"2024","journal-title":"JAMIA Open"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1101\/2024.04.03.24305298","article-title":"Enhancing early detection of cognitive decline in the elderly: a comparative study utilizing large language models in clinical notes","author":"X Du","journal-title":"medRxiv"},{"issue":"04","key":"ref26","doi-asserted-by":"crossref","first-page":"e240739","DOI":"10.1148\/ryai.240739","article-title":"Cybersecurity threats and mitigation strategies for large language models in health care","volume":"7","author":"T Akinci D'Antonoli","year":"2025","journal-title":"Radiol Artif Intell"}],"container-title":["Thrombosis and Haemostasis"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2796-1975.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T23:12:32Z","timestamp":1790205152000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2796-1975"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,28]]},"references-count":24,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,9,23]]},"published-print":{"date-parts":[[2026,10]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2796-1975","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["0340-6245","2567-689X"],"issn-type":[{"value":"0340-6245","type":"print"},{"value":"2567-689X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,1,28]]},"published":{"date-parts":[[2026,1,28]]}}}