{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T01:34:23Z","timestamp":1777080863159,"version":"3.51.4"},"reference-count":23,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T00:00:00Z","timestamp":1747785600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bucharest University of Economic Studies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computation"],"abstract":"<jats:p>Predicting corporate bankruptcy is a key task in financial risk management, and selecting a machine learning model with superior generalization performance is crucial for prediction accuracy. This study evaluates the effectiveness of k-fold cross-validation as a model selection strategy for random forest and XGBoost classifiers using a publicly available dataset of Taiwanese listed companies. We employ a nested cross-validation framework to assess the relationship between cross-validation (CV) and out-of-sample (OOS) performance on 40 different train\/test data partitions. On average, we find k-fold cross-validation to be a valid selection technique when applied within a model class; however, k-fold cross-validation may fail for specific train\/test splits. We find that 67% of model selection regret variability is explained by the particular train\/test split, highlighting an irreducible uncertainty real world practitioners must contend with. Our study extensively explores hyperparameter tuning for both classifiers and highlights key insights. Additionally, we investigate practical implementation choices in k-fold cross-validation\u2014such as the value of k or prediction strategies. We conclude that k-fold cross-validation is effective for model selection within a model class and on average, but it can be unreliable in specific cases or when comparing models from different classes\u2014this latter issue warranting further investigation.<\/jats:p>","DOI":"10.3390\/computation13050127","type":"journal-article","created":{"date-parts":[[2025,5,21]],"date-time":"2025-05-21T06:31:27Z","timestamp":1747809087000},"page":"127","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Assessing the Validity of k-Fold Cross-Validation for Model Selection: Evidence from Bankruptcy Prediction Using Random Forest and XGBoost"],"prefix":"10.3390","volume":"13","author":[{"given":"Vlad","family":"Teodorescu","sequence":"first","affiliation":[{"name":"Finance Department, The Bucharest University of Economic Studies, Pia\u021ba Roman\u0103 6, 010374 Bucure\u0219ti, Romania"}]},{"given":"Laura","family":"Obreja Bra\u0219oveanu","sequence":"additional","affiliation":[{"name":"Center of Financial and Monetary Research (CEFIMO), Finance Department, The Bucharest University of Economic Studies, Pia\u021ba Roman\u0103 6, 010374 Bucure\u0219ti, Romania"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1111\/j.1540-6261.1968.tb00843.x","article-title":"Financial Ratios, Discriminant Analysis and the Prediction of Corporate Bankruptcy","volume":"23","author":"Altman","year":"1968","journal-title":"J. Financ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"109","DOI":"10.2307\/2490395","article-title":"Financial Ratios and the Probabilistic Prediction of Bankruptcy","volume":"18","author":"Ohlson","year":"1980","journal-title":"J. Account. Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1016\/j.eswa.2017.04.006","article-title":"Machine Learning Models and Bankruptcy Prediction","volume":"83","author":"Barboza","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201916), San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_6","first-page":"77","article-title":"Benchmarking Machine Learning Models to Predict Corporate Bankruptcy","volume":"19","author":"Alanis","year":"2023","journal-title":"J. Credit. Risk"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1016\/j.ejor.2018.10.024","article-title":"Deep Learning Models for Bankruptcy Prediction Using Textual Disclosures","volume":"274","author":"Mai","year":"2019","journal-title":"Eur. J. Oper. Res."},{"key":"ref_8","unstructured":"Grinsztajn, L., Oyallon, E., and Varoquaux, G. (2022). Why Do Tree-Based Models Still Outperform Deep Learning on Tabular Data?. arXiv, Available online: https:\/\/arxiv.org\/abs\/2207.08815."},{"key":"ref_9","first-page":"2079","article-title":"On Over-Fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation","volume":"11","author":"Cawley","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"115222","DOI":"10.1016\/j.eswa.2021.115222","article-title":"Nested Cross-Validation When Selecting Classifiers Is Overzealous for Most Practical Applications","volume":"182","author":"Wainer","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1016\/j.ejor.2016.01.012","article-title":"Financial Ratios and Corporate Governance Indicators in Bankruptcy Prediction: A Comprehensive Study","volume":"252","author":"Liang","year":"2016","journal-title":"Eur. J. Oper. Res."},{"key":"ref_12","unstructured":"Teodorescu, V., and Toader, C.-I. (2024, January 13\u201314). Using Machine Learning to Model Bankruptcy Risk in Listed Companies. Proceedings of the 7th International Conference on Economics and Social Sciences, Bucharest, Romania. Issue 1."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"124761","DOI":"10.1016\/j.eswa.2024.124761","article-title":"Machine Learning Techniques in Bankruptcy Prediction: A Systematic Literature Review","volume":"255","author":"Dasilas","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v077.i01","article-title":"ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R","volume":"77","author":"Wright","year":"2017","journal-title":"J. Stat. Softw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ins.2013.09.038","article-title":"On the Importance of the Validation Technique for Classification with Imbalanced Datasets: Addressing Covariate Shift When Data Is Skewed","volume":"257","author":"Herrera","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1109\/TNNLS.2012.2199516","article-title":"Study on the Impact of Partition-Induced Dataset Shift on K-Fold Cross-Validation","volume":"23","author":"Herrera","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1109\/TPAMI.2009.187","article-title":"Sensitivity Analysis of K-Fold Cross Validation in Prediction Error Estimation","volume":"32","author":"Rodriguez","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","first-page":"626","article-title":"Cross-Validation Strategies for Balanced and Imbalanced Datasets","volume":"Volume 13653","author":"Rios","year":"2022","journal-title":"Intelligent Systems"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1145\/1882471.1882479","article-title":"Apples-to-Apples in Cross-Validation Studies: Pitfalls in Classifier Performance Measurement","volume":"12","author":"Forman","year":"2010","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/MCI.2018.2866730","article-title":"Cross-Validation for Imbalanced Datasets: Avoiding Overoptimistic and Overfitting Approaches","volume":"13","author":"Santos","year":"2018","journal-title":"IEEE Comput. Intell. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1017\/pan.2018.39","article-title":"How Cross-Validation Can Go Wrong and What to Do About It","volume":"27","author":"Neunhoeffer","year":"2019","journal-title":"Polit. Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1007\/s10462-025-11107-y","article-title":"An Experimental Survey of Imbalanced Learning Algorithms for Bankruptcy Prediction","volume":"58","author":"Gnip","year":"2025","journal-title":"Artif. Intell. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting Linear Mixed-Effects Models Using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Stat. Softw."}],"container-title":["Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-3197\/13\/5\/127\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:36:17Z","timestamp":1760031377000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-3197\/13\/5\/127"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,21]]},"references-count":23,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["computation13050127"],"URL":"https:\/\/doi.org\/10.3390\/computation13050127","relation":{},"ISSN":["2079-3197"],"issn-type":[{"value":"2079-3197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,21]]}}}