{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T07:53:27Z","timestamp":1781855607724,"version":"3.54.5"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T00:00:00Z","timestamp":1781827200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T00:00:00Z","timestamp":1781827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003385","name":"Georg-August-Universit\u00e4t G\u00f6ttingen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003385","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Innovations Syst Softw Eng"],"published-print":{"date-parts":[[2026,12]]},"abstract":"Abstract<\/jats:title>\n \n Machine learning pipeline debugging remains costly because predictive performance depends on complex interactions between dataset characteristics and configuration choices. In practice, however, practitioners often lack tools that can both explain performance failures and estimate the likely impact of repairs without repeated reruns. To address this challenge, we propose a model-driven framework for root cause analysis and hyperparameter intervention that operates solely on structured descriptors. The framework uses three dataset-complexity meta-features, namely class overlap, class imbalance, and sparsity, together with learner hyperparameters. We evaluate the approach on two model families, Decision Trees (DT) and Multilayer Perceptrons (MLP). For each family, we construct a meta-dataset comprising 81,000 pipeline runs generated from 270 datasets and 300 hyperparameter configurations. We then train an interpretable Explainable Boosting Machine (EBM) as a meta-model and assess predictive fidelity using GroupKFold partitioning by dataset to prevent data leakage. The study yields three main findings. First, descriptor-based performance prediction generalizes well across the two model families considered, achieving\n \n \n $$Mean~Absolute~Error (MAE)=0.080$$<\/jats:tex-math>\n \n \n M<\/mml:mi>\n e<\/mml:mi>\n a<\/mml:mi>\n n<\/mml:mi>\n \n A<\/mml:mi>\n b<\/mml:mi>\n s<\/mml:mi>\n o<\/mml:mi>\n l<\/mml:mi>\n u<\/mml:mi>\n t<\/mml:mi>\n e<\/mml:mi>\n \n E<\/mml:mi>\n r<\/mml:mi>\n r<\/mml:mi>\n o<\/mml:mi>\n r<\/mml:mi>\n (<\/mml:mo>\n M<\/mml:mi>\n A<\/mml:mi>\n E<\/mml:mi>\n )<\/mml:mo>\n =<\/mml:mo>\n 0.080<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n and\n \n \n $$Root~Mean~Squared~Error (RMSE)=0.108$$<\/jats:tex-math>\n \n \n R<\/mml:mi>\n o<\/mml:mi>\n o<\/mml:mi>\n t<\/mml:mi>\n \n M<\/mml:mi>\n e<\/mml:mi>\n a<\/mml:mi>\n n<\/mml:mi>\n \n S<\/mml:mi>\n q<\/mml:mi>\n u<\/mml:mi>\n a<\/mml:mi>\n r<\/mml:mi>\n e<\/mml:mi>\n d<\/mml:mi>\n \n E<\/mml:mi>\n r<\/mml:mi>\n r<\/mml:mi>\n o<\/mml:mi>\n r<\/mml:mi>\n (<\/mml:mo>\n R<\/mml:mi>\n M<\/mml:mi>\n S<\/mml:mi>\n E<\/mml:mi>\n )<\/mml:mo>\n =<\/mml:mo>\n 0.108<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n for DT, and\n \n \n $$MAE=0.087$$<\/jats:tex-math>\n \n \n M<\/mml:mi>\n A<\/mml:mi>\n E<\/mml:mi>\n =<\/mml:mo>\n 0.087<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n and\n \n \n $$RMSE=0.115$$<\/jats:tex-math>\n \n \n R<\/mml:mi>\n M<\/mml:mi>\n S<\/mml:mi>\n E<\/mml:mi>\n =<\/mml:mo>\n 0.115<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n for MLP. Second, the trained meta-model learns directionally coherent relationships that align with prior domain knowledge, enabling root cause diagnosis to identify meaningful drivers of both pipeline success and failure. In our results, successful ML pipelines are primarily associated with low class overlap, whereas failures are typically linked to high class imbalance or overlap, often amplified by unfavorable configuration choices such as restrictive tree-growth settings or optimization-sensitive MLP hyperparameters. Importantly, these attributions remain consistent with the expectation rules even when prediction residuals are non-trivial. Third, the causal hyperparameter intervention module accurately estimates post-intervention performance for hyperparameters identified as root causes, with a mean absolute intervention error of approximately 0.036 for both DT and MLP. Taken together, these results show that the proposed framework provides actionable root cause diagnosis and reliable hyperparameter intervention guidance for debugging machine learning pipelines.\n <\/jats:p>","DOI":"10.1007\/s11334-026-00642-8","type":"journal-article","created":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T07:24:59Z","timestamp":1781853899000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["From diagnosis to repair: A model-driven framework for root cause analysis of machine learning pipelines"],"prefix":"10.1007","volume":"22","author":[{"given":"Emmanuel Charleson","family":"Dapaah","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jens","family":"Grabowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,19]]},"reference":[{"issue":"3","key":"642_CR1","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1023\/A:1021713901879","volume":"50","author":"PB Brazdil","year":"2003","unstructured":"Brazdil PB, Soares C, Costa JP (2003) Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results. Mach Learn 50(3):251\u2013277. https:\/\/doi.org\/10.1023\/A:1021713901879","journal-title":"Mach Learn"},{"key":"642_CR2","unstructured":"Vanschoren J (2018) Meta-Learning: A Survey . https:\/\/arxiv.org\/abs\/1810.03548"},{"key":"642_CR3","doi-asserted-by":"crossref","unstructured":"Thornton C, Hutter F, Hoos HH, Leyton-Brown K (2013) Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms. arXiv. arXiv:1208.3719 [cs] . http:\/\/arxiv.org\/abs\/1208.3719 Accessed 2022-12-16","DOI":"10.1145\/2487575.2487629"},{"key":"642_CR4","unstructured":"Feurer M, Klein A, Eggensperger K, Springenberg J, Blum M, Hutter F (2015) Efficient and robust automated machine learning. Advances in neural information processing systems. p 28"},{"key":"642_CR5","doi-asserted-by":"publisher","unstructured":"Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. NIPS\u201917, pp. 4768\u20134777. Curran Associates Inc., Red Hook, NY, USA . https:\/\/doi.org\/10.5555\/3295222.3295230","DOI":"10.5555\/3295222.3295230"},{"key":"642_CR6","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \"why should i trust you?\": Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD \u201916, pp. 1135\u20131144. Association for Computing Machinery, New York, NY, USA . https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"642_CR7","doi-asserted-by":"publisher","unstructured":"Dapaah EC, Grabowski J (2025) Model-Driven Root Cause Analysis for Trustworthy AI: A Data-and-Model-Centric Explanation Framework . In: 2025 ACM\/IEEE 28th International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), pp. 377\u2013385. IEEE Computer Society, Los Alamitos, CA, USA . https:\/\/doi.org\/10.1109\/MODELS-C68889.2025.00056","DOI":"10.1109\/MODELS-C68889.2025.00056"},{"key":"642_CR8","unstructured":"Nori H, Jenkins S, Koch P, Caruana R (2019) Interpretml: A unified framework for machine learning interpretability.ArXiv abs\/1909.09223"},{"key":"642_CR9","unstructured":"Bischl B, Binder M, Lang M, Pielok T, Richter J, Coors S, Thomas J, Ullmann T, Becker M, Boulesteix AL, Deng D, Lindauer M (2021) Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges . https:\/\/arxiv.org\/abs\/2107.05847"},{"key":"642_CR10","unstructured":"Bahmani M, Shawi RE, Potikyan N, Sakr S (2021) To tune or not to tune? An Approach for Recommending Important Hyperparameters. arXiv. arXiv:2108.13066 [cs] . http:\/\/arxiv.org\/abs\/2108.13066 Accessed 2023-05-09"},{"key":"642_CR11","unstructured":"\u00d6stlund F, Fahlman E (2022) Data Complexity and its effect on Classification Accuracy in Multi Class Classification Problems: A study using synthetic datasets"},{"key":"642_CR12","doi-asserted-by":"publisher","unstructured":"Gong L, Jiang S, Wang R, Jiang L (2019) Empirical Evaluation of the Impact of Class Overlap on Software Defect Prediction. In: 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 698\u2013709 . https:\/\/doi.org\/10.1109\/ASE.2019.00071 . ISSN: 2643-1572. https:\/\/ieeexplore.ieee.org\/document\/8952192\/ Accessed 2025-05-06","DOI":"10.1109\/ASE.2019.00071"},{"key":"642_CR13","doi-asserted-by":"publisher","unstructured":"Prati RC, Batista GEAPA, Monard MC (2004) Class Imbalances versus Class Overlapping: An Analysis of a Learning System Behavior. In: Goos, G., Hartmanis, J., Van\u00a0Leeuwen, J., Monroy, R., Arroyo-Figueroa, G., Sucar, L.E., Sossa, H. (eds.) MICAI 2004: Advances in Artificial Intelligence vol. 2972, pp. 312\u2013321. Springer, Berlin, Heidelberg . https:\/\/doi.org\/10.1007\/978-3-540-24694-7_32 . Series Title: Lecture Notes in Computer Science","DOI":"10.1007\/978-3-540-24694-7_32"},{"key":"642_CR14","unstructured":"Hutter F, Hoos H, Leyton-Brown K (2014) An efficient approach for assessing hyperparameter importance. In: Xing, E.P., Jebara, T. (eds.) Proceedings of the 31st International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 32, pp. 754\u2013762. PMLR, Bejing, China . https:\/\/proceedings.mlr.press\/v32\/hutter14.html"},{"key":"642_CR15","doi-asserted-by":"publisher","unstructured":"Rijn JN, Hutter F (2018) Hyperparameter importance across datasets. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. KDD \u201918, pp. 2367\u20132376. Association for Computing Machinery, New York, NY, USA .https:\/\/doi.org\/10.1145\/3219819.3220058 . https:\/\/doi.org\/10.1145\/3219819.3220058","DOI":"10.1145\/3219819.3220058"},{"key":"642_CR16","unstructured":"Jin H (2022) Hyperparameter Importance for Machine Learning Algorithms"},{"issue":"3","key":"642_CR17","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1214\/ss\/1177013604.","volume":"1","author":"T Hastie","year":"1986","unstructured":"Hastie T, Tibshirani R (1986) Generalized Additive Models. Stat Sci 1(3):297\u2013310. https:\/\/doi.org\/10.1214\/ss\/1177013604. (Publisher: Institute of Mathematical Statistics)","journal-title":"Stat Sci"},{"key":"642_CR18","unstructured":"Shao X, Wang H, Zhu X, Xiong F (2022) FIND:Explainable Framework for Meta-learning . https:\/\/arxiv.org\/abs\/2205.10362"},{"issue":"4","key":"642_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2492248.2492263","volume":"38","author":"S Dalal","year":"2013","unstructured":"Dalal S, Chhillar RS (2013) Empirical study of root cause analysis of software failure. SIGSOFT Softw Eng Notes 38(4):1\u20137. https:\/\/doi.org\/10.1145\/2492248.2492263","journal-title":"SIGSOFT Softw Eng Notes"},{"key":"642_CR20","unstructured":"Chakarov A, Nori A, Rajamani S, Sen S, Vijaykeerthy D (2016) Debugging Machine Learning Tasks. arXiv:1603.07292 [cs, stat] . arXiv: 1603.07292. Accessed 2022-03-24"},{"key":"642_CR21","unstructured":"Koh PW, Liang P (2020) Understanding Black-box Predictions via Influence Functions . https:\/\/arxiv.org\/abs\/1703.04730"},{"issue":"2","key":"642_CR22","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1007\/s10270-024-01211-y","volume":"24","author":"S R\u00e4dler","year":"2024","unstructured":"R\u00e4dler S, Berardinelli L, Winter K, Rahimi A, Rinderle-Ma S (2024) Bridging mde and ai: a systematic review of domain-specific languages and model-driven practices in ai software systems engineering. Softw Syst Model 24(2):445\u2013469. https:\/\/doi.org\/10.1007\/s10270-024-01211-y","journal-title":"Softw Syst Model"},{"key":"642_CR23","doi-asserted-by":"publisher","unstructured":"Tabassi E (2023) Artificial Intelligence Risk Management Framework (AI RMF 1.0). Technical Report NIST AI 100-1, National Institute of Standards and Technology (U.S.), Gaithersburg, MD (January) . https:\/\/doi.org\/10.6028\/NIST.AI.100-1 . http:\/\/nvlpubs.nist.gov\/nistpubs\/ai\/NIST.AI.100-1.pdf Accessed 2025-07-31","DOI":"10.6028\/NIST.AI.100-1"},{"key":"642_CR24","unstructured":"Regulation (EU) 2024\/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300\/2008, (EU) No 167\/2013, (EU) No 168\/2013, (EU) 2018\/858, (EU) 2018\/1139 and (EU) 2019\/2144 and Directives 2014\/90\/EU, (EU) 2016\/797 and (EU) 2020\/1828 (Artificial Intelligence Act)Text with EEA relevance. (2024)"},{"issue":"3","key":"642_CR25","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1109\/34.990132","volume":"24","author":"TK Ho","year":"2002","unstructured":"Ho TK, Basu M (2002) Complexity measures of supervised classification problems. IEEE Trans Pattern Anal Mach Intell 24(3):289\u2013300. https:\/\/doi.org\/10.1109\/34.990132","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"642_CR26","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.neucom.2011.03.054","volume":"75","author":"AC Lorena","year":"2012","unstructured":"Lorena AC, Costa IG, Spola\u00c3\u2019r N, Souto MCP (2012) Analysis of complexity indices for classification problems: Cancer gene expression data. Neurocomputing 75(1):33\u201342. https:\/\/doi.org\/10.1016\/j.neucom.2011.03.054","journal-title":"Neurocomputing"},{"key":"642_CR27","doi-asserted-by":"crossref","unstructured":"Mollineda RA, S\u00c3inchez JS, Sotoca JM (2005) Data Characterization for Effective Prototype Selection. In: Marques, J.S., Blanca, N., Pina, P. (eds.) Pattern Recognition and Image Analysis, pp. 27\u201334. Springer, Berlin, Heidelberg","DOI":"10.1007\/11492542_4"},{"key":"642_CR28","doi-asserted-by":"crossref","unstructured":"Lorena AC, Garcia LPF, Lehmann J, Souto MCP, Ho TK (2020) How Complex is your classification problem? A survey on measuring classification complexity","DOI":"10.1145\/3347711"},{"issue":"2","key":"642_CR29","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1145\/2641190.2641198","volume":"15","author":"J Vanschoren","year":"2014","unstructured":"Vanschoren J, Rijn JN, Bischl B, Torgo L (2014) Openml: networked science in machine learning. SIGKDD Explor Newsl 15(2):49\u201360. https:\/\/doi.org\/10.1145\/2641190.2641198","journal-title":"SIGKDD Explor Newsl"},{"key":"642_CR30","doi-asserted-by":"publisher","unstructured":"Taylor R, Ojha V, Martino I, Nicosia G (2021) Sensitivity analysis for deep learning: Ranking hyper-parameter influence. In: 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pp. 512\u2013516. IEEE Computer Society, Los Alamitos, CA, USA . https:\/\/doi.org\/10.1109\/ICTAI52525.2021.00083","DOI":"10.1109\/ICTAI52525.2021.00083"},{"key":"642_CR31","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"642_CR32","unstructured":"Tucci RR (2013) Introduction to Judea Pearl\u2019s Do-Calculus. arXiv:1305.5506 [cs] . arXiv: 1305.5506. Accessed 2022-04-22"},{"key":"642_CR33","unstructured":"Bareinboim E, Correa JD, Ibeling D, Icard T 1 On Pearl\u2019s Hierarchy and the Foundations of Causal Inference, vol 62"},{"key":"642_CR34","unstructured":"Sharma A, Kiciman E (2020) DoWhy: An End-to-End Library for Causal Inference . https:\/\/arxiv.org\/abs\/2011.04216"},{"issue":"3","key":"642_CR35","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1109\/34.990132","volume":"24","author":"TK Ho","year":"2002","unstructured":"Ho TK, Basu M (2002) Complexity measures of supervised classification problems. IEEE Trans Pattern Anal Mach Intell 24(3):289\u2013300. https:\/\/doi.org\/10.1109\/34.990132","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"642_CR36","unstructured":"Dapaah EC, Grabowski J (2025) When Data Quality Issues Collide: A Large-Scale Empirical Study of Co-Occurring Data Quality Issues in Software Defect Prediction . https:\/\/arxiv.org\/abs\/2512.17460"},{"issue":"9","key":"642_CR37","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","volume":"21","author":"H He","year":"2009","unstructured":"He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263\u20131284. https:\/\/doi.org\/10.1109\/TKDE.2008.239","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"642_CR38","doi-asserted-by":"publisher","unstructured":"Liu B, Wei Y, Zhang Y, Yang Q (2017) Deep neural networks for high dimension, low sample size data. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17. pp 2287\u20132293. https:\/\/doi.org\/10.24963\/ijcai.2017\/318","DOI":"10.24963\/ijcai.2017\/318"},{"key":"642_CR39","doi-asserted-by":"crossref","unstructured":"Smith LN (2017) Cyclical Learning Rates for Training Neural Networks . https:\/\/arxiv.org\/abs\/1506.01186","DOI":"10.1109\/WACV.2017.58"},{"key":"642_CR40","unstructured":"Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: Dasgupta S, McAllester D (eds) Proceedings of the 30th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol 28. PMLR, Atlanta, Georgia, USA, pp 1139\u20131147"},{"key":"642_CR41","unstructured":"Keskar NS, Mudigere D, Nocedal J, Smelyanskiy M, Tang PTP (2017) On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima . https:\/\/arxiv.org\/abs\/1609.04836"}],"container-title":["Innovations in Systems and Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-026-00642-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11334-026-00642-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11334-026-00642-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T07:25:13Z","timestamp":1781853913000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11334-026-00642-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,19]]},"references-count":41,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,12]]}},"alternative-id":["642"],"URL":"https:\/\/doi.org\/10.1007\/s11334-026-00642-8","relation":{},"ISSN":["1614-5046","1614-5054"],"issn-type":[{"value":"1614-5046","type":"print"},{"value":"1614-5054","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,19]]},"assertion":[{"value":"4 March 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 May 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"17"}}