{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T23:43:43Z","timestamp":1792799023910,"version":"4.5.0"},"reference-count":32,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2025,4,25]],"date-time":"2025-04-25T00:00:00Z","timestamp":1745539200000},"content-version":"unspecified","delay-in-days":114,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Dutch Cardiovascular Alliance Consortium","award":["2020B004"],"award-info":[{"award-number":["2020B004"]}],"id":[{"id":"https:\/\/ror.org\/02e8s5y35","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["Open Heart"],"accepted":{"date-parts":[[2025,3,24]]},"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Exclusion of coronary stenosis in individuals with suggestive symptoms is challenging. Cardiac CT or coronary angiography is often used but is inefficient and costly and involves risks. Sex-stratified algorithms based on electronic health records (EHRs) could be a non-invasive alternative for excluding coronary stenosis, yet their performance may vary by healthcare settings. Thus, external validation is crucial for determining their generalisability. This study aimed to externally validate sex-stratified machine learning algorithms based on EHR data to predict the absence of coronary stenosis, evaluated in diverse clinical settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Sex-stratified XGBoost algorithms were trained on EHR data from patients who underwent coronary imaging at the University Medical Center Utrecht (n=14 674) and externally tested on EHR data of 13 Cardiology centres in the Netherlands (n=9252). The outcome was defined as the absence of coronary stenosis, identified through text mining of radiology report conclusions, and predictive performance was assessed by negative predictive values (NPVs) and specificities.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>On the training cohort (9298 men (median age 55 years, 73% no coronary stenosis) and 5376 women (median age 59 years, 83% no coronary stenosis)), the algorithms showed NPVs and specificities of 0.95 and 0.14 in men and 0.93 and 0.26 in women, respectively. On the testing cohort (4762 men (median age 60 years, 60% no coronary stenosis) and 4490 women (median age 60 years, 83% no coronary stenosis)), the algorithm showed NPVs and specificities of 0.89 and 0.07 in men and 0.87 and 0.18 in women, respectively.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>This study externally validates sex\u2010stratified machine learning algorithms using EHR data to non-invasively predict the absence of coronary stenosis, with high NPVs observed across settings. However, given the modest specificity and study limitations, these findings should be considered preliminary, warranting further refinement before clinical adoption.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1136\/openhrt-2024-003072","type":"journal-article","created":{"date-parts":[[2025,4,25]],"date-time":"2025-04-25T01:35:12Z","timestamp":1745544912000},"page":"e003072","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":2,"title":["Optimising coronary imaging decisions with machine learning: an external validation study"],"prefix":"10.1136","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7086-0864","authenticated-orcid":false,"given":"L. Malin","family":"Overmars","sequence":"first","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bram","family":"van Es","sequence":"additional","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Floor","family":"Groepenhoff","sequence":"additional","affiliation":[{"name":"Experimental Cardiology, University Medical Center Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark C. H.","family":"De Groot","sequence":"additional","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"G. Aernout","family":"Somsen","sequence":"additional","affiliation":[{"name":"Cardiology, Cardiology Centers of the Netherlands, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophie Heleen","family":"Bots","sequence":"additional","affiliation":[{"name":"Division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University Utrecht Institute for Pharmaceutical Sciences, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"I. Igor","family":"Tulevski","sequence":"additional","affiliation":[{"name":"Cardiology, Cardiology Centers of the Netherlands, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonard","family":"Hofstra","sequence":"additional","affiliation":[{"name":"Cardiology, Cardiology Centers of the Netherlands, Amsterdam, The Netherlands"},{"name":"Department of Cardiology, Amsterdam UMC Locatie VUmc, Amsterdam, Noord-Holland, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hester M.","family":"den Ruijter","sequence":"additional","affiliation":[{"name":"Experimental Cardiology, University Medical Center Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wouter W.","family":"van Solinge","sequence":"additional","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Imo","family":"Hoefer","sequence":"additional","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saskia","family":"Haitjema","sequence":"additional","affiliation":[{"name":"Central Diagnostic Laboratory, University Medical Centre Utrecht, Utrecht, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"239","published-online":{"date-parts":[[2025,4,24]]},"reference":[{"key":"2025081611540129000_12.1.e003072.1","doi-asserted-by":"publisher","DOI":"10.1016\/S0735-1097(98)00606-8"},{"key":"2025081611540129000_12.1.e003072.2","doi-asserted-by":"publisher","DOI":"10.1016\/S0735-1097(98)00583-X"},{"key":"2025081611540129000_12.1.e003072.3","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2016-012652","article-title":"Cost-effectiveness of anatomical and functional test strategies for stable chest pain: public health perspective from a middle-income country","volume":"7","author":"Bertoldi","year":"2017","journal-title":"BMJ Open"},{"key":"2025081611540129000_12.1.e003072.4","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1002\/clc.22532","article-title":"Long-term Cost-Effectiveness of Diagnostic Tests for Assessing Stable Chest Pain: Modeled Analysis of Anatomical and Functional Strategies","volume":"39","author":"Bertoldi","year":"2016","journal-title":"Clin Cardiol"},{"key":"2025081611540129000_12.1.e003072.5","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1093\/ehjdh\/ztab103","article-title":"Preventing unnecessary imaging in patients suspect of coronary artery disease through machine learning of electronic health records","volume":"3","author":"Overmars","year":"2022","journal-title":"Eur Heart J Digit Health"},{"key":"2025081611540129000_12.1.e003072.6","unstructured":"Overmars LM . Big data, small vessels (doctoral dissertation). University Medical Center Utrecht, 2024"},{"key":"2025081611540129000_12.1.e003072.7","doi-asserted-by":"publisher","DOI":"10.1093\/eurheartj\/ehac238"},{"key":"2025081611540129000_12.1.e003072.8","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(02)00528-0"},{"key":"2025081611540129000_12.1.e003072.9","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.i3140"},{"key":"2025081611540129000_12.1.e003072.10","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwt298"},{"key":"2025081611540129000_12.1.e003072.11","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1186\/s12872-021-02020-7","article-title":"Routine clinical care data from thirteen cardiac outpatient clinics: design of the Cardiology Centers of the Netherlands (CCN) database","volume":"21","author":"Bots","year":"2021","journal-title":"BMC Cardiovasc Disord"},{"key":"2025081611540129000_12.1.e003072.12","doi-asserted-by":"publisher","DOI":"10.1515\/CCLM.2007.009"},{"key":"2025081611540129000_12.1.e003072.13","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1016\/j.tele.2017.08.002","article-title":"DEDUCE: A pattern matching method for automatic de-identification of Dutch medical text","volume":"35","author":"Menger","year":"2018","journal-title":"Telematics and Informatics"},{"key":"2025081611540129000_12.1.e003072.14","doi-asserted-by":"crossref","DOI":"10.2196\/40516","article-title":"A Wolf in Sheep\u2019s Clothing: Reuse of Routinely Obtained Laboratory Data in Research","volume":"24","author":"Overmars","year":"2022","journal-title":"J Med Internet Res"},{"key":"2025081611540129000_12.1.e003072.15","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-150-9-200905050-00006"},{"key":"2025081611540129000_12.1.e003072.16","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2021.11.023"},{"key":"2025081611540129000_12.1.e003072.17","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.b2393"},{"key":"2025081611540129000_12.1.e003072.18","doi-asserted-by":"publisher","DOI":"10.1186\/s41512-020-00077-0"},{"key":"2025081611540129000_12.1.e003072.19","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-4356(03)00080-5"},{"key":"2025081611540129000_12.1.e003072.20","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/s12859-022-05130-x","article-title":"Negation detection in Dutch clinical texts: an evaluation of rule-based and machine learning methods","volume":"24","author":"van Es","year":"2023","journal-title":"BMC Bioinformatics"},{"key":"2025081611540129000_12.1.e003072.21","doi-asserted-by":"crossref","unstructured":"Chen T , Guestrin C . XGBoost: a scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD \u201916; 2016:785\u201394. doi:10.1145\/2939672.2939785","DOI":"10.1145\/2939672.2939785"},{"key":"2025081611540129000_12.1.e003072.22","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-019-1466-7"},{"key":"2025081611540129000_12.1.e003072.23","unstructured":"Wickham H , Fran\u00e7ois R , Henry L , et al . Dplyr: a grammar of data manipulation. The R Foundation; 2024."},{"key":"2025081611540129000_12.1.e003072.24","unstructured":"Chen T , He T , Benesty M , et al . Xgboost: extreme gradient boosting. 2024."},{"key":"2025081611540129000_12.1.e003072.25","doi-asserted-by":"crossref","unstructured":"Wickham H . Ggplot2: elegant graphics for data analysis (use r!). 2nd edn. Cham: Springer, 2016.","DOI":"10.1007\/978-3-319-24277-4_9"},{"key":"2025081611540129000_12.1.e003072.26","unstructured":"Sadatsafavi M , Safari A , Lee TY . Predtools: prediction model tools. 2023."},{"key":"2025081611540129000_12.1.e003072.27","doi-asserted-by":"publisher","DOI":"10.21105\/joss.01686"},{"key":"2025081611540129000_12.1.e003072.28","doi-asserted-by":"crossref","first-page":"3167","DOI":"10.21105\/joss.03167","article-title":"Visualizations with statistical details: The 'ggstatsplot' approach","volume":"6","author":"Patil","year":"2021","journal-title":"JOSS"},{"key":"2025081611540129000_12.1.e003072.29","doi-asserted-by":"crossref","unstructured":"Kuhn M . Building Predictive Models in R Using the caret Package. J Stat Softw 2008;28. doi:10.18637\/jss.v028.i05","DOI":"10.18637\/jss.v028.i05"},{"key":"2025081611540129000_12.1.e003072.30","doi-asserted-by":"crossref","DOI":"10.1161\/JAHA.123.029400","article-title":"Development and Validation of a Bayesian Network-Based Model for Predicting Coronary Heart Disease Risk From Electronic Health Records","volume":"13","author":"Suo","year":"2024","journal-title":"J Am Heart Assoc"},{"key":"2025081611540129000_12.1.e003072.31","doi-asserted-by":"crossref","DOI":"10.2196\/17257","article-title":"Accurate Prediction of Coronary Heart Disease for Patients With Hypertension From Electronic Health Records With Big Data and Machine-Learning Methods: Model Development and Performance Evaluation","volume":"8","author":"Du","year":"2020","journal-title":"JMIR Med Inform"},{"key":"2025081611540129000_12.1.e003072.32","doi-asserted-by":"crossref","first-page":"956","DOI":"10.1109\/TBME.2017.2731158","article-title":"A Regularized Deep Learning Approach for Clinical Risk Prediction of Acute Coronary Syndrome Using Electronic Health Records","volume":"65","author":"Huang","year":"2018","journal-title":"IEEE Trans Biomed Eng"}],"container-title":["Open Heart"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/openhrt-2024-003072","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,16]],"date-time":"2025-08-16T18:54:11Z","timestamp":1755370451000},"score":1,"resource":{"primary":{"URL":"https:\/\/openheart.bmj.com\/lookup\/doi\/10.1136\/openhrt-2024-003072"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,4,24]]},"published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1136\/openhrt-2024-003072"],"URL":"https:\/\/doi.org\/10.1136\/openhrt-2024-003072","relation":{},"ISSN":["2053-3624"],"issn-type":[{"value":"2053-3624","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]}}}