{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:09Z","timestamp":1785544989430,"version":"3.56.0"},"reference-count":69,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Predictive models increasingly support clinical decision-making, although imbalanced outcome distributions are common in health care datasets and can distort performance evaluation. The area under the receiver operating characteristic curve (AUROC) remains the most frequently reported metric, despite its limited ability to reflect clinically meaningful performance under class imbalance.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to examine the influences of metric selection on the clinical interpretation of predictive models in imbalanced real-world health care data.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n This was a retrospective cohort study, including 17,018 hospitalized patients with COVID-19. Two predictive models using extreme gradient boosting (XGBoost) were developed to predict kidney replacement therapy (KRT) and mortality. Model performance was assessed using AUROC, macro-\n F<\/jats:italic>\n 1<\/jats:sub>\n -score, class-specific precision and recall, calibration (curve, slope, and intercept), decision curve analysis, and learning curves. Standard rebalancing strategies were applied exclusively to the training data to evaluate their impact on performance.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n KRT occurred in 9.5%, and mortality in 18.0%. Although AUROC values were high (0.928 for KRT and 0.945 for mortality), performance in the minority class was substantially lower. For KRT, precision was 0.539 and recall 0.372; for mortality, precision was 0.725 and recall 0.718. Rebalancing strategies were associated with higher recall for the minority class, but this gain was accompanied by a reduction in precision, with minimal impact on AUROC values. As a result, AUROC remained high despite clinically relevant changes in error distribution between false positives and false negatives. The learning curves show a plateau-like shape, with stable validation performance across all training set sizes for both outcomes.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n AUROC alone is insufficient to evaluate prediction models in imbalanced health care scenarios, even with rebalancing. Routine reporting of class-aware metrics, alongside learning curve analysis, is essential to support robust and clinically meaningful evaluation of predictive models, rather than their direct translation into practice.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/86379","type":"journal-article","created":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T20:24:36Z","timestamp":1782332676000},"page":"e86379-e86379","source":"Crossref","is-referenced-by-count":0,"title":["Beyond Area Under the Receiver Operating Characteristic Curve: Evaluating Predictive Performance Metrics Under Class Imbalance in Real-World Clinical Data"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9480-1727","authenticated-orcid":false,"given":"Vanessa das Gra\u00e7as Jos\u00e9","family":"Ventura","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7366-2633","authenticated-orcid":false,"given":"Claudio Mois\u00e9s Valiense de","family":"Andrade","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9142-2919","authenticated-orcid":false,"given":"Jussara Marques de","family":"Almeida","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4212-519X","authenticated-orcid":false,"given":"Bruno Porto","family":"Pessoa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2447-2577","authenticated-orcid":false,"given":"Car\u00edsi Anne","family":"Polanczyk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7862-8701","authenticated-orcid":false,"given":"Guilherme Fonseca do","family":"Nascimento","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2559-7128","authenticated-orcid":false,"given":"Eric","family":"Boersma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1144-6262","authenticated-orcid":false,"given":"Heloisa Reniers","family":"Vianna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8978-4512","authenticated-orcid":false,"given":"Katia de Paula","family":"Farah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4913-4902","authenticated-orcid":false,"given":"Leonardo Chaves Dutra da","family":"Rocha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2075-3363","authenticated-orcid":false,"given":"Marcos Andr\u00e9","family":"Gon\u00e7alves","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4278-3771","authenticated-orcid":false,"given":"Milena Soriano","family":"Marcolino","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,6,24]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-074819","article-title":"Evaluation of clinical prediction models (part 1): from development to external validation","volume":"384","author":"Collins","journal-title":"BMJ"},{"key":"R2","doi-asserted-by":"crossref","unstructured":"Steyerberg EW . Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer Nature; 2019. URL: https:\/\/doi.org\/10.1007\/978-3-030-16399-0 [Accessed 22-08-2025]","DOI":"10.1007\/978-3-030-16399-0"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2021.01.009","article-title":"Clinical prediction models: diagnosis versus prognosis","volume":"132","author":"van Smeden","journal-title":"J Clin Epidemiol"},{"issue":"10","key":"R4","doi-asserted-by":"publisher","first-page":"2352","DOI":"10.1177\/09622802211038754","article-title":"Revisiting performance metrics for prediction with rare outcomes","volume":"30","author":"Adhikari","journal-title":"Stat Methods Med Res"},{"issue":"3","key":"R5","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1016\/j.surg.2023.05.023","article-title":"Evaluating prediction model performance","volume":"174","author":"Cabot","journal-title":"Surgery"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","journal-title":"BMJ"},{"issue":"6","key":"R7","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1111\/add.16133","article-title":"Outcome class imbalance and rare events: an underappreciated complication for overdose risk prediction modeling","volume":"118","author":"Cartus","journal-title":"Addiction"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-28579-z","article-title":"Potential and limitations of machine meta-learning (ensemble) methods for predicting COVID-19 mortality in a large inhospital Brazilian dataset","volume":"13","author":"de Paiva","journal-title":"Sci Rep"},{"issue":"9","key":"R9","doi-asserted-by":"publisher","first-page":"1242","DOI":"10.1016\/j.joca.2023.05.006","article-title":"Comparison of evaluation metrics of deep learning for imbalanced imaging data in osteoarthritis studies","volume":"31","author":"Liu","journal-title":"Osteoarthr Cartil"},{"issue":"6","key":"R10","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1161\/CIRCULATIONAHA.107.699579","article-title":"General cardiovascular risk profile for use in primary care: the Framingham Heart Study","volume":"117","author":"D\u2019Agostino","journal-title":"Circulation"},{"key":"R11","doi-asserted-by":"crossref","unstructured":"Hosmer DW Lemeshow S Sturdivant RX . Applied Logistic Regression. 1st ed. John Wiley & Sons, Inc; URL: https:\/\/doi.org\/10.1002\/9781118548387 [Accessed 22-08-2025] [doi: 10.1002\/9781118548387]","DOI":"10.1002\/9781118548387"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/s40985-018-0093-8","article-title":"Assessing the value of screening tools: reviewing the challenges and opportunities of cost-effectiveness analysis","volume":"39","author":"Iragorri","journal-title":"Public Health Rev"},{"issue":"11","key":"R13","doi-asserted-by":"publisher","first-page":"e596","DOI":"10.1161\/CIR.0000000000000678","article-title":"2019 ACC\/AHA guideline on the primary prevention of cardiovascular disease: a report of the American College of Cardiology\/American Heart Association task force on clinical practice guidelines","volume":"140","author":"Arnett","journal-title":"Circulation"},{"issue":"20","key":"R14","doi-asserted-by":"publisher","first-page":"2069","DOI":"10.1001\/jama.2020.21749","article-title":"Behavioral counseling interventions to promote a healthy diet and physical activity for cardiovascular disease prevention in adults with cardiovascular risk factors: US Preventive Services Task Force recommendation statement","volume":"324","author":"US Preventive Services Task Force","journal-title":"JAMA"},{"issue":"8","key":"R15","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1016\/j.jclinepi.2009.11.009","article-title":"Use of Brier score to assess binary predictions","volume":"63","author":"Rufibach","journal-title":"J Clin Epidemiol"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2024-082505","article-title":"PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods","volume":"388","author":"Moons","journal-title":"BMJ"},{"issue":"25","key":"R17","doi-asserted-by":"publisher","first-page":"2439","DOI":"10.1093\/eurheartj\/ehab309","article-title":"SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe","volume":"42","author":"Hageman","journal-title":"Eur Heart J"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the Achilles heel of predictive analytics","volume":"17","author":"Van Calster","journal-title":"BMC Med"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1093\/ehjdh\/ztae080","article-title":"Machine learning based prediction models for cardiovascular disease risk using electronic health records data: systematic review and meta-analysis","volume":"6","author":"Liu","journal-title":"Eur Heart J Digit Health"},{"issue":"4","key":"R20","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1093\/ehjqcco\/qcad017","article-title":"Machine-learning versus traditional approaches for atherosclerotic cardiovascular risk prognostication in primary prevention cohorts: a systematic review and meta-analysis","volume":"9","author":"Liu","journal-title":"Eur Heart J Qual Care Clin Outcomes"},{"issue":"12","key":"R21","doi-asserted-by":"publisher","first-page":"2423","DOI":"10.11124\/JBIES-24-00042","article-title":"Monitoring performance of clinical artificial intelligence in health care: a scoping review","volume":"22","author":"Andersen","journal-title":"JBI Evid Synth"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","DOI":"10.1002\/jeo2.12039","article-title":"A practical guide to the implementation of AI in orthopaedic research, part 6: how to evaluate the performance of AI research?","volume":"11","author":"Oettl","journal-title":"J Exp Orthop"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-09954-8","article-title":"On evaluation metrics for medical applications of artificial intelligence","volume":"12","author":"Hicks","journal-title":"Sci Rep"},{"issue":"11","key":"R24","doi-asserted-by":"publisher","first-page":"1270","DOI":"10.1038\/s41592-021-01302-4","article-title":"The class imbalance problem","volume":"18","author":"Megahed","journal-title":"Nat Methods"},{"issue":"8","key":"R25","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1038\/nmeth.3945","article-title":"Classification evaluation","volume":"13","author":"Lever","journal-title":"Nat Methods"},{"issue":"1","key":"R26","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-019-1426-2","article-title":"Key challenges for delivering clinical impact with artificial intelligence","volume":"17","author":"Kelly","journal-title":"BMC Med"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejrai.2025.100030","article-title":"Evaluation metrics in medical imaging AI: fundamentals, pitfalls, misapplications, and recommendations","volume":"3","author":"Kocak","journal-title":"Eur J Radiol Artif Intell"},{"issue":"3-4","key":"R28","doi-asserted-by":"publisher","DOI":"10.1002\/sim.10320","article-title":"The harms of class imbalance corrections for machine learning based prediction models: a simulation study","volume":"44","author":"Carriero","journal-title":"Stat Med"},{"key":"R29","unstructured":"Recommendations for national SARS-cov-2 testing strategies and diagnostic capacities: interim guidance, 25 June 2021. World Health Organization. 2021. URL: https:\/\/iris.who.int\/handle\/10665\/342002 [Accessed 22-08-2025]"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijid.2021.01.019","article-title":"Clinical characteristics and outcomes of patients hospitalized with COVID-19 in Brazil: results from the Brazilian COVID-19 registry","volume":"107","author":"Marcolino","journal-title":"Int J Infect Dis"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2022.963667","article-title":"Kidney health in the COVID-19 pandemic: an umbrella review of meta-analyses and systematic reviews","volume":"10","author":"Yang","journal-title":"Front Public Health"},{"issue":"2","key":"R32","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"Harris","journal-title":"J Biomed Inform"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: building an international community of software platform partners","volume":"95","author":"Harris","journal-title":"J Biomed Inform"},{"issue":"11","key":"R34","doi-asserted-by":"publisher","first-page":"899","DOI":"10.1089\/tmj.2015.0234","article-title":"The experience of a sustainable large scale Brazilian telehealth network","volume":"22","author":"Soriano Marcolino","journal-title":"Telemed J E Health"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1186\/s12877-023-04494-w","article-title":"Clinical characteristics and outcomes of COVID-19 patients with preexisting dementia: a large multicenter propensity-matched Brazilian cohort study","volume":"24","author":"Bicalho","journal-title":"BMC Geriatr"},{"key":"R36","doi-asserted-by":"crossref","unstructured":"Chen T Guestrin C . XGBoost: a scalable tree boosting system. Presented at: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Aug 13-17, 2016:785-794; San Francisco, CA. [doi: 10.1145\/2939672.2939785]","DOI":"10.1145\/2939672.2939785"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.11.011","article-title":"Tabular data: deep learning is not all you need","volume":"81","author":"Shwartz-Ziv","journal-title":"Inf Fusion"},{"issue":"3","key":"R38","doi-asserted-by":"publisher","DOI":"10.3390\/healthcare8030247","article-title":"Prediction of type 2 diabetes risk and its effect evaluation based on the XGBoost model","volume":"8","author":"Wang","journal-title":"Healthcare (Basel)"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.2196\/49023","article-title":"Practical considerations and applied examples of cross-validation for model development and evaluation in health care: tutorial","volume":"2","author":"Wilimitis","journal-title":"JMIR AI"},{"issue":"4","key":"R40","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.220232","article-title":"A guide to cross-validation for artificial intelligence in medical imaging","volume":"5","author":"Bradshaw","journal-title":"Radiol Artif Intell"},{"key":"R41","unstructured":"Kohavi R . A study of cross-validation and bootstrap for accuracy estimation and model selection. Presented at: 14th International Joint Conference on Artificial Intelligence (IJCAI \u201995); Aug 20-25, 1995:1137-1145; Montreal, Canada. [doi: 10.5555\/1643031.1643047]"},{"key":"R42","doi-asserted-by":"publisher","DOI":"10.1016\/j.spasta.2024.100843","article-title":"Automatic cross-validation in structured models: is it time to leave out leave-one-out?","volume":"62","author":"Adin","journal-title":"Spat Stat"},{"key":"R43","unstructured":"Lemaitre G Nogueira F Aridas CK . Imbalanced-learn: a Python toolbox to tackle the curse of imbalanced datasets in machine learning. arXiv. Preprint posted online on Sep 21, 2016. [doi: 10.48550\/arXiv.1609.06570]"},{"key":"R44","doi-asserted-by":"crossref","unstructured":"Garcia EA He H Bai Y Li S . ADASYN: adaptive synthetic sampling approach for imbalanced learning. Presented at: 2008 IEEE International Joint Conference on Neural Networks (IJCNN 2008); Jun 1-8, 2008:1322-1328; Hong Kong, China. [doi: 10.1109\/IJCNN.2008.4633969]","DOI":"10.1109\/IJCNN.2008.4633969"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic Minority Over-sampling Technique","volume":"16","author":"Chawla","journal-title":"J Artif Intell Res"},{"key":"R46","doi-asserted-by":"crossref","unstructured":"Han H Wang WY Mao BH . Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Huang DS Zhang XP Huang GB , editors. Advances in Intelligent Computing. Springer Nature; 2005:878-887. URL: https:\/\/doi.org\/10.1007\/11538059_91 [Accessed 28-01-2026] [doi: 10.1007\/11538059_91]","DOI":"10.1007\/11538059_91"},{"issue":"1","key":"R47","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1504\/IJKESDP.2011.039875","article-title":"Borderline over-sampling for imbalanced data classification","volume":"3","author":"Nguyen","journal-title":"Int J Knowledge Eng Soft Data Paradigms"},{"key":"R48","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2018.06.056","article-title":"Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE","volume":"465","author":"Douzas","journal-title":"Inf Sci (Ny)"},{"key":"R49","unstructured":"More A . Survey of resampling techniques for improving classification performance in unbalanced datasets. arXiv. Preprint posted online on Aug 22, 2016. [doi: 10.48550\/arXiv.1608.06048]"},{"key":"R50","unstructured":"Murphy KP . Machine Learning: A Probabilistic Perspective. MIT Press; 2012. ISBN: 0262018020"},{"key":"R51","doi-asserted-by":"publisher","DOI":"10.9735\/2229-3981","article-title":"Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation","author":"Powers","journal-title":"J Mach Learn Technol"},{"issue":"1","key":"R52","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1093\/bioinformatics\/btr597","article-title":"MissForest--non-parametric missing value imputation for mixed-type data","volume":"28","author":"Stekhoven","journal-title":"Bioinformatics"},{"issue":"4","key":"R53","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","journal-title":"Inf Process Manag"},{"key":"R54","doi-asserted-by":"crossref","unstructured":"Davis J Goadrich M . The relationship between precision-recall and ROC curves. Presented at: Proceedings of the 23rd international conference on Machine learning - ICML \u201906; Jun 25-29, 2006:233-240; Pittsburgh, PA. [doi: 10.1145\/1143844.1143874]","DOI":"10.1145\/1143844.1143874"},{"issue":"6","key":"R55","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: a novel method for evaluating prediction models","volume":"26","author":"Vickers","journal-title":"Med Decis Making"},{"key":"R56","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44851-9_15","article-title":"Optimal thresholding of classifiers to maximize F1 measure","volume":"8725","author":"Lipton","journal-title":"Mach Learn Knowl Discov Databases"},{"issue":"1","key":"R57","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","article-title":"Assessing the performance of prediction models: a framework for traditional and novel measures","volume":"21","author":"Steyerberg","journal-title":"Epidemiology (Sunnyvale)"},{"issue":"4","key":"R58","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1093\/jamia\/ocz228","article-title":"A tutorial on calibration measurements and calibration models for clinical prediction models","volume":"27","author":"Huang","journal-title":"J Am Med Inform Assoc"},{"issue":"14","key":"R59","doi-asserted-by":"publisher","first-page":"1377","DOI":"10.1001\/jama.2017.12126","article-title":"Discrimination and calibration of clinical prediction models: users\u2019 guides to the medical literature","volume":"318","author":"Alba","journal-title":"JAMA"},{"issue":"6","key":"R60","doi-asserted-by":"publisher","first-page":"7799","DOI":"10.1109\/TPAMI.2022.3220744","article-title":"The shape of learning curves: a review","volume":"45","author":"Viering","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"R61","doi-asserted-by":"crossref","unstructured":"Simon GJ Aliferis C , editors. Artificial Intelligence and Machine Learning in Health Care and Medical Sciences: Best Practices and Pitfalls. Springer International Publishing; 2024. URL: https:\/\/doi.org\/10.1007\/978-3-031-39355-6 [Accessed 10-04-2026] [doi: 10.1007\/978-3-031-39355-6]","DOI":"10.1007\/978-3-031-39355-6"},{"issue":"1","key":"R62","doi-asserted-by":"publisher","DOI":"10.1007\/s44163-024-00199-0","article-title":"The effect of resampling techniques on the performances of machine learning clinical risk prediction models in the setting of severe class imbalance: development and internal validation in a retrospective cohort","volume":"4","author":"Ke","journal-title":"Discov Artif Intell"},{"key":"R63","doi-asserted-by":"crossref","unstructured":"Dinov ID . Data Science and Predictive Analytics: Biomedical and Health Applications Using R. Springer; 2018. URL: https:\/\/link.springer.com\/book\/10.1007\/978-3-031-17483-4 [Accessed 10-04-2026]","DOI":"10.1007\/978-3-319-72347-1"},{"issue":"2","key":"R64","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooad033","article-title":"Implications of resampling data to address the class imbalance problem (IRCIP): an evaluation of impact on performance between classification algorithms in medical data","volume":"6","author":"Welvaars","journal-title":"JAMIA Open"},{"issue":"9","key":"R65","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.1093\/jamia\/ocac093","article-title":"The harm of class imbalance corrections for risk prediction models: illustration and simulation using logistic regression","volume":"29","author":"van den Goorbergh","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"R66","doi-asserted-by":"publisher","DOI":"10.3390\/medicina57050503","article-title":"Foundational statistical principles in medical research: sensitivity, specificity, positive predictive value, and negative predictive value","volume":"57","author":"Monaghan","journal-title":"Medicina (Kaunas)"},{"key":"R67","unstructured":"Clinical criteria. DynaMed. Aug 22, 2025. URL: https:\/\/www.dynamed.com\/calculators\/#cc-idx [Accessed 14-05-2026]"},{"issue":"1","key":"R68","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-023-03212-y","article-title":"Clinical prediction models and the multiverse of madness","volume":"21","author":"Riley","journal-title":"BMC Med"},{"issue":"3","key":"R69","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1007\/s42979-023-01720-5","article-title":"Mortality prediction of various cancer patients via relevant feature analysis and machine learning","volume":"4","author":"Bozkurt","journal-title":"SN Comput Sci"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e86379","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:32Z","timestamp":1785536732000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e86379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,24]]},"references-count":69,"URL":"https:\/\/doi.org\/10.2196\/86379","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,24]]},"article-number":"v10i10e86379"}}