{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T00:53:35Z","timestamp":1787532815760,"version":"build-2736575974"},"reference-count":44,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T00:00:00Z","timestamp":1740355200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T00:00:00Z","timestamp":1740355200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100011937","name":"Lower Saxony Ministry of Science and Culture","doi-asserted-by":"crossref","award":["ZN3492"],"award-info":[{"award-number":["ZN3492"]}],"id":[{"id":"10.13039\/100011937","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004115","name":"Gottfried Wilhelm Leibniz Universit\u00e4t Hannover","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004115","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Gradient boosted tree ensembles (GBTEs) such as XGBoost continue to outperform other machine learning models on tabular data. However, the plethora of adjustable hyperparameters can exacerbate optimisation, especially in regression tasks with no intuitive performance measures such as accuracy and confidence. Automated machine learning frameworks alleviate the hyperparameter search for users, but if the optimisation procedure ends prematurely due to resource constraints, it is questionable whether users receive good models. To tackle this problem, we introduce a cost-efficient method to retrofit previously optimised XGBoost models by retraining them with a new weight distribution over the training instances. We base our approach on topological results, which allows us to infer model-agnostic weights for specific regions of the data distribution where the targets are more susceptible to input perturbations. By linking our theory to the training procedure of XGBoost regressors, we then establish a topologically consistent reweighting scheme, which is independent of the specific model instance. Empirically, we verify that our approach improves prediction performance, outperforms other reweighting methods and is much faster than a hyperparameter search. To enable users to find the optimal weights for their data, we provide guides based on our findings on 20 datasets.\u00a0Our code is available at: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/montymaxzuehlke\/tcr\" ext-link-type=\"uri\">https:\/\/github.com\/montymaxzuehlke\/tcr<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s10994-024-06704-x","type":"journal-article","created":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T22:21:34Z","timestamp":1740435694000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["TCR: topologically consistent reweighting for XGBoost in regression tasks"],"prefix":"10.1007","volume":"114","author":[{"given":"Monty-Maximilian","family":"Z\u00fchlke","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Kudenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,2,24]]},"reference":[{"key":"6704_CR1","doi-asserted-by":"publisher","unstructured":"Bertin-Mahieux, T. (2011). Year prediction msd. https:\/\/doi.org\/10.24432\/C50K61","DOI":"10.24432\/C50K61"},{"key":"6704_CR2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3229161","author":"V Borisov","year":"2022","unstructured":"Borisov, V., Leemann, T., Se\u00dfler, K., et al. (2022). Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems. https:\/\/doi.org\/10.1109\/TNNLS.2022.3229161","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"6704_CR3","doi-asserted-by":"publisher","first-page":"088506661880027","DOI":"10.1177\/0885066618800276","volume":"34","author":"S Bose","year":"2018","unstructured":"Bose, S., Johnson, A., Moskowitz, A., et al. (2018). Impact of intensive care unit discharge delays on patient outcomes: A retrospective cohort study. Journal of Intensive Care Medicine, 34, 088506661880027. https:\/\/doi.org\/10.1177\/0885066618800276","journal-title":"Journal of Intensive Care Medicine"},{"key":"6704_CR4","unstructured":"Branco, P., Torgo, L., & Ribeiro, R.P. (2017). SMOGN: a pre-processing approach for imbalanced regression. In: Lu\u00eds\u00a0Torgo PB, Moniz N (eds) Proceedings of the first international workshop on learning with imbalanced domains: Theory and applications, proceedings of machine learning research, vol.\u00a074. PMLR, (pp. 36\u201350), https:\/\/proceedings.mlr.press\/v74\/branco17a.html"},{"key":"6704_CR5","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. (2001). Random forests. Machine Learning, 45, 5\u201332. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Machine Learning"},{"key":"6704_CR6","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla, N. V., Bowyer, K. W., Hall, L. O., et al. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321\u2013357.","journal-title":"Journal of Artificial Intelligence Research"},{"issue":"115","key":"6704_CR7","first-page":"14","volume":"22","author":"T Chen","year":"2014","unstructured":"Chen, T. (2014). Introduction to boosted trees. University of Washington Computer Science, 22(115), 14\u201340.","journal-title":"University of Washington Computer Science"},{"key":"6704_CR8","doi-asserted-by":"crossref","unstructured":"Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, (pp. 785\u2013794).","DOI":"10.1145\/2939672.2939785"},{"key":"6704_CR9","doi-asserted-by":"crossref","unstructured":"Cobza\u015f, \u015e., Miculescu, R., & Nicolae, A. (2019). Lipschitz Functions. Lecture Notes in Mathematics, Springer International Publishing, https:\/\/books.google.de\/books?id=r9yZDwAAQBAJ","DOI":"10.1007\/978-3-030-16489-8"},{"key":"6704_CR10","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1016\/j.dss.2009.05.016","volume":"47","author":"P Cortez","year":"2009","unstructured":"Cortez, P., Cerdeira, A., Almeida, F., et al. (2009). Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47, 547\u2013553.","journal-title":"Decision Support Systems"},{"issue":"2","key":"6704_CR11","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1214\/009053604000000067","volume":"32","author":"B Efron","year":"2004","unstructured":"Efron, B., Hastie, T., Johnstone, I., et al. (2004). Least angle regression. The Annals of Statistics, 32(2), 407\u2013499. https:\/\/doi.org\/10.1214\/009053604000000067","journal-title":"The Annals of Statistics"},{"key":"6704_CR12","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1007\/s13748-013-0040-3","volume":"2","author":"H Fanaee-T","year":"2014","unstructured":"Fanaee-T, H., & Gama, J. (2014). Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, 2, 113\u2013127. https:\/\/doi.org\/10.1007\/s13748-013-0040-3","journal-title":"Progress in Artificial Intelligence"},{"key":"6704_CR13","doi-asserted-by":"publisher","unstructured":"Feng, J., Lurati, L., Ouyang, H., et\u00a0al. (2003). Predictive toxicology: Benchmarking molecular descriptors and statistical methods, (pp. 1463\u20131470), https:\/\/doi.org\/10.1021\/ci034032s","DOI":"10.1021\/ci034032s"},{"key":"6704_CR14","doi-asserted-by":"publisher","unstructured":"Fernandes, K. (2015). A proactive intelligent decision support system for predicting the popularity of online news.[SPACE]https:\/\/doi.org\/10.1007\/978-3-319-23485-4_53","DOI":"10.1007\/978-3-319-23485-4_53"},{"issue":"1","key":"6704_CR15","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1006\/jcss.1997.1504","volume":"55","author":"Y Freund","year":"1997","unstructured":"Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119\u2013139. https:\/\/doi.org\/10.1006\/jcss.1997.1504","journal-title":"Journal of Computer and System Sciences"},{"issue":"5","key":"6704_CR16","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189\u20131232. https:\/\/doi.org\/10.1214\/aos\/1013203451","journal-title":"The Annals of Statistics"},{"key":"6704_CR17","doi-asserted-by":"crossref","unstructured":"Geertsema, P., & Lu, H. (2023). Instance-based explanations for gradient boosting machine predictions with axil weights. ArXiv: abs\/2301.01864. https:\/\/api.semanticscholar.org\/CorpusID:255440273","DOI":"10.2139\/ssrn.4318194"},{"key":"6704_CR18","unstructured":"Ghorbani, A., & Zou, J. (2019). Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning, PMLR, (pp. 2242\u20132251)."},{"issue":"23","key":"6704_CR19","doi-asserted-by":"publisher","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","volume":"101","author":"AL Goldberger","year":"2000","unstructured":"Goldberger, A. L., Amaral, L. A. N., Glass, L., et al. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation., 101(23), e215\u2013e220. https:\/\/doi.org\/10.1161\/01.CIR.101.23.e215","journal-title":"Circulation."},{"key":"6704_CR20","first-page":"507","volume":"35","author":"L Grinsztajn","year":"2022","unstructured":"Grinsztajn, L., Oyallon, E., & Varoquaux, G. (2022). Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems, 35, 507\u2013520.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6704_CR21","doi-asserted-by":"publisher","unstructured":"Guyon, I., Sun-Hosoya, L., Boull\u00e9, M., et\u00a0al. (2019). Analysis of the AutoML Challenge Series 2015\u20132018, (pp. 177\u2013219) Springer International Publishing, Cham. https:\/\/doi.org\/10.1007\/978-3-030-05318-5_10","DOI":"10.1007\/978-3-030-05318-5_10"},{"key":"6704_CR22","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., & Friedman, J. (2001). The elements of statistical learning: data mining, inference, and prediction. Springer series in statistics, Springer https:\/\/books.google.de\/books?id=VRzITwgNV2UC","DOI":"10.1007\/978-0-387-21606-5"},{"key":"6704_CR23","unstructured":"Johnson, A., Pollard, T., & Mark, R. (2016a). \u201cMIMIC-III Clinical Database\u201d (version 1.4). PhysioNet."},{"key":"6704_CR24","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"A Johnson","year":"2016","unstructured":"Johnson, A., Pollard, T., Shen, L., et al. (2016). Mimic-iii, a freely accessible critical care database. Scientific Data, 3, 160035. https:\/\/doi.org\/10.1038\/sdata.2016.35","journal-title":"Scientific Data"},{"key":"6704_CR25","unstructured":"Kawala, F., Douzal-Chouakria, A., Gaussier, E., et\u00a0al. (2013). Pr\u00e9dictions d\u2019activit\u00e9 dans les r\u00e9seaux sociaux en ligne. In 4i\u00e8me conf\u00e9rence sur les mod\u00e8les et l\u2019analyse des r\u00e9seaux: Approches math\u00e9matiques et informatiques, (p.\u00a016)."},{"key":"6704_CR26","unstructured":"Ke, G., Meng, Q., Finley, T., et\u00a0al. (2017). Lightgbm: A highly efficient gradient boosting decision tree. In: I. Guyon, U. V. Luxburg, S. Bengio, et\u00a0al (eds.), Advances in neural information processing systems, vol\u00a030. Curran Associates, Inc., https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf"},{"issue":"3","key":"6704_CR27","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/S0167-7152(96)00140-X","volume":"33","author":"R Kelley Pace","year":"1997","unstructured":"Kelley Pace, R., & Barry, R. (1997). Sparse spatial autoregressions. Statistics and Probability Letters, 33(3), 291\u2013297. https:\/\/doi.org\/10.1016\/S0167-7152(96)00140-X","journal-title":"Statistics and Probability Letters"},{"key":"6704_CR28","unstructured":"Knauer, F., & Cukierski, W. (2015). Rossmann store sales. https:\/\/kaggle.com\/competitions\/rossmann-store-sales, kaggle"},{"key":"6704_CR29","unstructured":"Kunz, N. (2020). SMOGN: Synthetic minority over-sampling technique for regression with gaussian noise. https:\/\/pypi.org\/project\/smogn\/"},{"key":"6704_CR30","unstructured":"LeDell, E., & Poirier, S. (2020). H2O AutoML: Scalable automatic machine learning. In 7th ICML workshop on automated machine learning (AutoML)[SPACE]https:\/\/www.automl.org\/wp-content\/uploads\/2020\/07\/AutoML_2020_paper_61.pdf"},{"key":"6704_CR31","unstructured":"Lundberg, S.M., & Lee, S.I. (2017). A unified approach to interpreting model predictions. In Proceedings of the 31st international conference on neural information processing systems, (pp. 4768-4777), Curran Associates Inc., Red Hook, NY, USA, NIPS\u201917."},{"issue":"1","key":"6704_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0280387","volume":"18","author":"N MacNell","year":"2023","unstructured":"MacNell, N., Feinstein, L., Wilkerson, J., et al. (2023). Implementing machine learning methods with complex survey data: Lessons learned on the impacts of accounting sampling weights in gradient boosting. PLoS ONE, 18(1), 1\u201315. https:\/\/doi.org\/10.1371\/journal.pone.0280387","journal-title":"PLoS ONE"},{"key":"6704_CR33","unstructured":"Ohno-Machado, L., Fraser, H.S.F., & \u00d8hrn, A. (1998). Improving machine learning performance by removing redundant cases in medical data sets. In AMIA 1998, American medical informatics association annual symposium, Lake Buena Vista, FL, USA, November 7-11, 1998. AMIA, http:\/\/knowledge.amia.org\/amia-55142-a1998a-1.588514\/t-001-1.590475\/f-001-1.590476\/a-099-1.590738\/a-100-1.590735"},{"issue":"11","key":"6704_CR34","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1080\/14786440109462720","volume":"2","author":"K Pearson","year":"1901","unstructured":"Pearson, K. (1901). Liii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 2(11), 559\u2013572. https:\/\/doi.org\/10.1080\/14786440109462720","journal-title":"The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science"},{"key":"6704_CR35","unstructured":"Prokhorenkova, L., Gusev, G., Vorobev, A., et\u00a0al. (2018). Catboost: unbiased boosting with categorical features. In Proceedings of the 32nd international conference on neural information processing systems, (pp. 6639\u20136649). Curran Associates Inc., Red Hook, NY, USA, NIPS\u201918."},{"key":"6704_CR36","unstructured":"Rabanser, S., G\u00fcnnemann, S., & Lipton, Z. (2019). Failing loudly: An empirical study of methods for detecting dataset shift. Advances in Neural Information Processing Systems, 32"},{"key":"6704_CR37","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., & Guestrin, C. (2016). \u201cWhy should I trust you?\u201d explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, (pp. 1135\u20131144).","DOI":"10.1145\/2939672.2939778"},{"issue":"1","key":"6704_CR38","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/762\/1\/012036","volume":"762","author":"A Rogozhnikov","year":"2016","unstructured":"Rogozhnikov, A. (2016). Reweighting with boosted decision trees. Journal of Physics: Conference Series, 762(1), 012036. https:\/\/doi.org\/10.1088\/1742-6596\/762\/1\/012036","journal-title":"Journal of Physics: Conference Series"},{"key":"6704_CR39","doi-asserted-by":"publisher","unstructured":"Seiffert, C., Khoshgoftaar, T.M., Hulse, J.V., et\u00a0al. (2008). Resampling or reweighting: A comparison of boosting implementations. In 2008 20th IEEE international conference on tools with artificial intelligence, (pp. 445\u2013451), https:\/\/doi.org\/10.1109\/ICTAI.2008.59","DOI":"10.1109\/ICTAI.2008.59"},{"key":"6704_CR40","first-page":"307","volume-title":"Contributions to the Theory of Games II","author":"LS Shapley","year":"1953","unstructured":"Shapley, L. S. (1953). A value for n-person games. In H. W. Kuhn & A. W. Tucker (Eds.), Contributions to the Theory of Games II (pp. 307\u2013317). Princeton: Princeton University Press."},{"issue":"2","key":"6704_CR41","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1080\/0025570X.1989.11977419","volume":"62","author":"DJ Smith","year":"1989","unstructured":"Smith, D. J., & Vamanamurthy, M. K. (1989). How small is a unit ball? Mathematics Magazine, 62(2), 101\u2013107. https:\/\/doi.org\/10.1080\/0025570X.1989.11977419","journal-title":"Mathematics Magazine"},{"key":"6704_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10994-021-06023-5","volume":"110","author":"M Steininger","year":"2021","unstructured":"Steininger, M., Kobs, K., Davidson, P., et al. (2021). Density-based weighting for imbalanced regression. Machine Learning, 110, 1\u201325. https:\/\/doi.org\/10.1007\/s10994-021-06023-5","journal-title":"Machine Learning"},{"key":"6704_CR43","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1007\/978-3-642-40669-0_33","volume-title":"Progress in Artificial Intelligence","author":"L Torgo","year":"2013","unstructured":"Torgo, L., Ribeiro, R. P., Pfahringer, B., et al. (2013). Smote for regression. In L. Correia, L. P. Reis, & J. Cascalho (Eds.), Progress in Artificial Intelligence (pp. 378\u2013389). Springer."},{"key":"6704_CR44","doi-asserted-by":"publisher","unstructured":"Zhang, C., Li, Y., Chen, X., et\u00a0al. (2020). Doubleensemble: A new ensemble method based on sample reweighting and feature selection for financial data analysis. In 2020 IEEE international conference on data mining (ICDM). IEEE Computer Society, Los Alamitos, CA, USA, (pp. 781\u2013790), https:\/\/doi.org\/10.1109\/ICDM50108.2020.00087","DOI":"10.1109\/ICDM50108.2020.00087"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06704-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06704-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06704-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T15:08:38Z","timestamp":1743347318000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06704-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,24]]},"references-count":44,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,4]]}},"alternative-id":["6704"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06704-x","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,24]]},"assertion":[{"value":"9 February 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 November 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 November 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval and consent to participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Materials availability"}}],"article-number":"108"}}