{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T09:37:11Z","timestamp":1794649031895,"version":"4.9.1"},"reference-count":25,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T00:00:00Z","timestamp":1783555200000},"content-version":"vor","delay-in-days":16,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,6,23]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The medical literature has seen rapid growth in studies that develop and evaluate machine learning (ML) classifiers for diagnostics, accompanied by an increase in systematic reviews for synthesizing this evidence. Many such reviews use meta-analyses that pool performance metrics such as area under the receiver\u2013operating characteristic curve (AUC), sensitivity, and specificity across ML studies. In our view, quantitative pooling of such metrics is statistically and conceptually inappropriate in most settings, except when the same model is being tested across different samples. ML classifiers differ fundamentally in training data, model specification, and validation strategies. As a result, there is no underlying \u2018true AUC\u2019 that pooling aims to recover. In addition, AUC is a sampling-dependent, non-linear, and bounded measure whose value depends on relative ranking within a dataset, which makes it unsuitable for aggregation. Pooling sensitivity and specificity presents similar problems due to threshold dependence, inconsistent reporting practices, and inappropriate weighting by test-set size. Outside limited scenarios in which aggregation may be reasonable, undue emphasis on pooled performance estimates obfuscates critical descriptors of model validity and generalizability. We propose that systematic reviews of medical ML models prioritize structured, descriptive synthesis of key study characteristics, including dataset composition, validation strategy, input modalities, outcome definitions, data augmentation, independent validation, and deployability. When summary performance measures are presented, they are better presented as stratified summaries in subgroups of comparable studies. A concerted discourse on standardizing reporting standards is essential to guide systematic reviews that meaningfully inform the clinical utility of ML models.<\/jats:p>","DOI":"10.1093\/ehjdh\/ztag107","type":"journal-article","created":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T11:58:24Z","timestamp":1783511904000},"source":"Crossref","is-referenced-by-count":1,"title":["Systematic reviews of medical machine learning: limitations of pooling AUCs"],"prefix":"10.1093","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5317-6024","authenticated-orcid":false,"given":"Amulya","family":"Gupta","sequence":"first","affiliation":[{"name":"Program for AI and Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]},{"name":"Department of Cardiovascular Medicine, The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rahel","family":"Sileshi","sequence":"additional","affiliation":[{"name":"Program for AI and Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]},{"name":"Department of Cardiovascular Medicine, The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffrey","family":"Thompson","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Data Science, The University of Kansas Medical Center , Kansas City, KS ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9947-3849","authenticated-orcid":false,"given":"Amit","family":"Noheria","sequence":"additional","affiliation":[{"name":"Program for AI and Research in Cardiovascular Medicine (PARC), The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]},{"name":"Department of Cardiovascular Medicine, The University of Kansas Medical Center , 3901 Rainbow Blvd, Kansas City, KS 66160 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,9]]},"reference":[{"key":"2026071514321014400_ztag107-B1","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1093\/ehjdh\/ztae080","article-title":"Machine learning based prediction models for cardiovascular disease risk using electronic health records data: systematic review and meta-analysis","volume":"6","author":"Liu","year":"2025","journal-title":"Eur Heart J Digit Health"},{"key":"2026071514321014400_ztag107-B2","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1080\/14737167.2023.2279107","article-title":"Systematic reviews of machine learning in healthcare: a literature review","volume":"24","author":"Kolasa","year":"2024","journal-title":"Expert Rev Pharmacoecon Outcomes Res"},{"key":"2026071514321014400_ztag107-B3","doi-asserted-by":"crossref","DOI":"10.1097\/CRD.0000000000001172","article-title":"Diagnostic performance of artificial intelligence-assisted echocardiography in identifying hypertrophic cardiomyopathy: a systematic review and meta-analysis","author":"Shojaei","year":"2026","journal-title":"Cardiol Rev"},{"key":"2026071514321014400_ztag107-B4","article-title":"Machine learning prediction models for preeclampsia: systematic review and meta-analysis","volume":"28","author":"Liu","year":"2026","journal-title":"J Med Internet Res"},{"key":"2026071514321014400_ztag107-B5","doi-asserted-by":"crossref","first-page":"e872","DOI":"10.1016\/S2589-7500(23)00177-2","article-title":"Machine learning and deep learning predictive models for long-term prognosis in patients with chronic obstructive pulmonary disease: a systematic review and meta-analysis","volume":"5","author":"Smith","year":"2023","journal-title":"Lancet Digit Health"},{"key":"2026071514321014400_ztag107-B6","doi-asserted-by":"crossref","first-page":"28378","DOI":"10.1038\/s41598-025-11445-5","article-title":"Diagnostic systematic review and meta-analysis of machine learning in predicting biochemical recurrence of prostate cancer","volume":"15","author":"Ling","year":"2025","journal-title":"Sci Rep"},{"key":"2026071514321014400_ztag107-B7","article-title":"Evaluating machine learning accuracy in detecting significant coronary stenosis using CCTA-derived fractional flow reserve: meta-analysis and systematic review","volume":"55","author":"van Noort","year":"2024","journal-title":"Int J Cardiol Heart Vasc"},{"key":"2026071514321014400_ztag107-B8","doi-asserted-by":"crossref","DOI":"10.1161\/JAHA.123.033298","article-title":"Systematic review and meta-analysis of prehospital machine learning scores as screening tools for early detection of large vessel occlusion in patients with suspected stroke","volume":"13","author":"Alobaida","year":"2024","journal-title":"J Am Heart Assoc"},{"key":"2026071514321014400_ztag107-B9","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2025.33512","article-title":"Use of AI in identification of sexually transmitted infections and anogenital dermatoses: a systematic review and meta-analysis","volume":"8","author":"Soe","year":"2025","journal-title":"JAMA Netw Open"},{"key":"2026071514321014400_ztag107-B10","doi-asserted-by":"crossref","first-page":"1454","DOI":"10.1186\/s12879-024-10380-6","article-title":"Machine learning for the prediction of mortality in patients with sepsis-associated acute kidney injury: a systematic review and meta-analysis","volume":"24","author":"Lv","year":"2024","journal-title":"BMC Infect Dis"},{"key":"2026071514321014400_ztag107-B11","doi-asserted-by":"crossref","first-page":"e117","DOI":"10.1016\/S2589-7500(21)00256-9","article-title":"Point-of-care screening for heart failure with reduced ejection fraction using artificial intelligence during ECG-enabled stethoscope examination in London, UK: a prospective, observational, multicentre study","volume":"4","author":"Bachtiger","year":"2022","journal-title":"Lancet Digit Health"},{"key":"2026071514321014400_ztag107-B12","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1161\/CIRCULATIONAHA.122.062646","article-title":"Detection of left ventricular systolic dysfunction from electrocardiographic images","volume":"148","author":"Sangha","year":"2023","journal-title":"Circulation"},{"key":"2026071514321014400_ztag107-B13","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.patrec.2023.12.012","article-title":"On the bias in the AUC variance estimate","volume":"178","author":"Xu","year":"2024","journal-title":"Pattern Recognit Lett"},{"key":"2026071514321014400_ztag107-B14","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1037\/0033-2909.117.1.167","article-title":"Meta-analysis of screening and diagnostic tests","volume":"117","author":"Hasselblad","year":"1995","journal-title":"Psychol Bull"},{"key":"2026071514321014400_ztag107-B15","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1186\/s13244-021-01034-1","article-title":"A meta-analysis of the diagnostic performance of machine learning-based MRI in the prediction of axillary lymph node metastasis in breast cancer patients","volume":"12","author":"Chen","year":"2021","journal-title":"Insights Imaging"},{"key":"2026071514321014400_ztag107-B16","article-title":"Performance of machine learning algorithms for surgical site infection case detection and prediction: a systematic review and meta-analysis","volume":"84","author":"Wu","year":"2022","journal-title":"Ann Med Surg (Lond)"},{"key":"2026071514321014400_ztag107-B17","first-page":"1","volume-title":"Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 10","author":"Macaskill","year":"2010"},{"key":"2026071514321014400_ztag107-B18","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/0895-4356(94)00099-C","article-title":"Meta-analytic methods for diagnostic test accuracy","volume":"48","author":"Irwig","year":"1995","journal-title":"J Clin Epidemiol"},{"key":"2026071514321014400_ztag107-B19","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1097\/XEB.0000000000000061","article-title":"Diagnostic test accuracy: methods for systematic review and meta-analysis","volume":"13","author":"Campbell","year":"2015","journal-title":"Int J Evid Based Healthc"},{"key":"2026071514321014400_ztag107-B20","doi-asserted-by":"crossref","DOI":"10.1136\/bmj-2024-082505","article-title":"PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods","volume":"388","author":"Moons","year":"2025","journal-title":"BMJ"},{"key":"2026071514321014400_ztag107-B21","doi-asserted-by":"crossref","DOI":"10.1056\/AIoa2300032","article-title":"Evaluation of sepsis prediction models before onset of treatment","volume":"1","author":"Kamran","year":"2024","journal-title":"NEJM AI"},{"key":"2026071514321014400_ztag107-B22","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/0895-7061(94)00178-E","article-title":"Magnetic resonance imaging compared to echocardiography to assess left ventricular mass in the hypertensive patient","volume":"8","author":"Bottini","year":"1995","journal-title":"Am J Hypertens"},{"key":"2026071514321014400_ztag107-B23","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1093\/ehjdh\/ztaf003","article-title":"Comparison of machine learning and conventional criteria in detecting left ventricular hypertrophy and prognosis with electrocardiography","volume":"6","author":"Huang","year":"2025","journal-title":"Eur Heart J Digit Health"},{"key":"2026071514321014400_ztag107-B24","doi-asserted-by":"publisher","DOI":"10.1016\/j.hroo.2026.03.019","article-title":"Machine learning to classify left ventricular hypertrophy using electrocardiographic feature extraction by variational autoencoder","author":"Gupta","year":"2026","journal-title":"Heart Rhythm O2"},{"key":"2026071514321014400_ztag107-B25","first-page":"1053","article-title":"AFS2026\u201441; machine learning for predicting atrial fibrillation recurrence after catheter ablation: a systematic review","volume":"37","author":"Gupta","year":"2026","journal-title":"J Cardiovasc Electrophysiol"}],"container-title":["European Heart Journal - Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/ehjdh\/advance-article-pdf\/doi\/10.1093\/ehjdh\/ztag107\/68740943\/ztag107.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/6\/ztag107\/68740943\/ztag107.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/ehjdh\/article-pdf\/7\/6\/ztag107\/68740943\/ztag107.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T18:32:20Z","timestamp":1784140340000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/ehjdh\/article\/doi\/10.1093\/ehjdh\/ztag107\/8729334"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,23]]},"references-count":25,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,23]]}},"URL":"https:\/\/doi.org\/10.1093\/ehjdh\/ztag107","relation":{},"ISSN":["2634-3916"],"issn-type":[{"value":"2634-3916","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,7]]},"published":{"date-parts":[[2026,6,23]]},"article-number":"ztag107"}}