{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T08:33:25Z","timestamp":1772526805618,"version":"3.50.1"},"reference-count":71,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>A key challenge in Artificial Intelligence (AI) has been the potential trade-off between the accuracy and comprehensibility of machine learning models, as that also relates to their safe and trusted adoption. While there has been a lot of talk about this trade-off, there is no systematic study that assesses to what extent it exists, how often it occurs, and for what types of datasets. Based on the analysis of 90 benchmark classification datasets, we find that this trade-off exists for most (69%) of the datasets, but that somewhat surprisingly for the majority of cases it is rather small while for only a few it is very large. Comprehensibility can be enhanced by adding yet another algorithmic step, that of surrogate modelling using so-called \u2018explainable\u2019 models. Such models can improve the accuracy-comprehensibility trade-off, especially in cases where the black box was initially better. Finally, we find that dataset characteristics related to the complexity required to model the dataset, and the level of noise, can significantly explain this trade-off and thus the cost of comprehensibility. These insights lead to specific guidelines on how and when to apply AI algorithms when comprehensibility is required.<\/jats:p>","DOI":"10.1186\/s40537-022-00579-2","type":"journal-article","created":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T09:02:58Z","timestamp":1646643778000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["The non-linear nature of the cost of comprehensibility"],"prefix":"10.1186","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3784-826X","authenticated-orcid":false,"given":"Sofie","family":"Goethals","sequence":"first","affiliation":[]},{"given":"David","family":"Martens","sequence":"additional","affiliation":[]},{"given":"Theodoros","family":"Evgeniou","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,7]]},"reference":[{"key":"579_CR1","unstructured":"Agrawal A. New York regulator orders probe into Goldman Sachs\u2019 credit card practices over Apple Card and sexism; November 12, 2019. Medianama, Online, https:\/\/www.medianama.com\/2019\/11\/223-apple-card-sexism-goldman-sachs\/. Accessed 1 Feb 2022."},{"key":"579_CR2","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780192847263.001.0001","volume-title":"Data Science ethics: concepts, Techniques and Cautionary Tales","author":"D Martens","year":"2022","unstructured":"Martens D. Data Science ethics: concepts, Techniques and Cautionary Tales. Oxford: Clarendon Press; 2022."},{"key":"579_CR3","unstructured":"Wozniak S. Tweet; November 10, 2019. Twitter, Online, accessed February 1, 2022. https:\/\/twitter.com\/stevewoz\/status\/1193330241478901760."},{"issue":"3","key":"579_CR4","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1214\/ss\/1009213726","volume":"16","author":"L Breiman","year":"2001","unstructured":"Breiman L. Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat Sci. 2001;16(3):199\u2013231.","journal-title":"Stat Sci"},{"key":"579_CR5","unstructured":"Broad Agency Announcement, Explainable Artifcial Intelligence (XAI). https:\/\/www.darpa.mil\/attachments\/DARPA-BAA-16-53.pdf. Accessed 12 Nov 2020."},{"issue":"3","key":"579_CR6","doi-asserted-by":"publisher","first-page":"1466","DOI":"10.1016\/j.ejor.2006.04.051","volume":"183","author":"D Martens","year":"2007","unstructured":"Martens D, Baesens B, Van Gestel T, Vanthienen J. Comprehensible credit scoring models using rule extraction from support vector machines. Eur J Oper Res. 2007;183(3):1466\u201376.","journal-title":"Eur J Oper Res"},{"issue":"2","key":"579_CR7","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1093\/idpl\/ipx005","volume":"7","author":"S Wachter","year":"2017","unstructured":"Wachter S, Mittelstadt B, Floridi L. Why a right to explanation of automated decision-making does not exist in the general data protection regulation. Int Data Priv Law. 2017;7(2):76\u201399.","journal-title":"Int Data Priv Law."},{"issue":"1","key":"579_CR8","doi-asserted-by":"publisher","first-page":"18","DOI":"10.3390\/e23010018","volume":"23","author":"P Linardatos","year":"2021","unstructured":"Linardatos P, Papastefanopoulos V, Kotsiantis S. Explainable ai: a review of machine learning interpretability methods. Entropy. 2021;23(1):18.","journal-title":"Entropy."},{"issue":"1","key":"579_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2594473.2594475","volume":"15","author":"AA Freitas","year":"2014","unstructured":"Freitas AA. Comprehensible classification models: a position paper. ACM SIGKDD Explorat Newslet. 2014;15(1):1\u201310.","journal-title":"ACM SIGKDD Explorat Newslet"},{"issue":"44","key":"579_CR10","doi-asserted-by":"publisher","first-page":"22071","DOI":"10.1073\/pnas.1900654116","volume":"116","author":"WJ Murdoch","year":"2019","unstructured":"Murdoch WJ, Singh C, Kumbier K, Abbasi-Asl R, Yu B. Definitions, methods, and applications in interpretable machine learning. Proc Natl Acad Sci. 2019;116(44):22071\u201380.","journal-title":"Proc Natl Acad Sci"},{"key":"579_CR11","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1162\/99608f92.5a8a3a3d","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin C, Radin J. Why are we using black box models in AI when we don\u2019t need to? A lesson from an explainable AI competition. Harvard Data Sci Rev. 2019;1:2.","journal-title":"Harvard Data Sci Rev."},{"issue":"4","key":"579_CR12","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1016\/S0169-2070(00)00057-1","volume":"16","author":"S Makridakis","year":"2000","unstructured":"Makridakis S, Hibon M. The M3-Competition: results, conclusions and implications. Int J Forecast. 2000;16(4):451\u201376.","journal-title":"Int J Forecast"},{"issue":"7553","key":"579_CR13","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436\u201344.","journal-title":"Nature."},{"key":"579_CR14","doi-asserted-by":"crossref","unstructured":"Borisov V, Leemann T, Se\u00dfler K, Haug J, Pawelczyk M, Kasneci G. Deep neural networks and tabular data: A survey. arXiv preprint arXiv:211001889. 2021.","DOI":"10.1109\/TNNLS.2022.3229161"},{"key":"579_CR15","unstructured":"Popov S, Morozov S, Babenko A. Neural oblivious decision ensembles for deep learning on tabular data. arXiv preprint arXiv:190906312. 2019."},{"key":"579_CR16","doi-asserted-by":"crossref","unstructured":"Ar\u0131k SO, Pfister T. Tabnet: Attentive interpretable tabular learning. arXiv. 2020.","DOI":"10.1609\/aaai.v35i8.16826"},{"key":"579_CR17","unstructured":"Zeng J, Ustun B, Rudin C. Interpretable classification models for recidivism prediction. arXiv preprint arXiv:150307810. 2015."},{"key":"579_CR18","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"AB Arrieta","year":"2020","unstructured":"Arrieta AB, D\u00edaz-Rodr\u00edguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, et al. Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inform Fusion. 2020;58:82\u2013115.","journal-title":"Inform Fusion."},{"issue":"8","key":"579_CR19","doi-asserted-by":"publisher","first-page":"832","DOI":"10.3390\/electronics8080832","volume":"8","author":"DV Carvalho","year":"2019","unstructured":"Carvalho DV, Pereira EM, Cardoso JS. Machine learning interpretability: a survey on methods and metrics. Electronics. 2019;8(8):832.","journal-title":"Electronics."},{"issue":"1","key":"579_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-020-00392-9","volume":"8","author":"C Shorten","year":"2021","unstructured":"Shorten C, Khoshgoftaar TM, Furht B. Deep Learning applications for COVID-19. J Big Data. 2021;8(1):1\u201354.","journal-title":"J Big Data."},{"issue":"1","key":"579_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-021-00444-8","volume":"8","author":"L Alzubaidi","year":"2021","unstructured":"Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data. 2021;8(1):1\u201374.","journal-title":"J Big Data."},{"key":"579_CR22","unstructured":"Molnar C. Interpretable machine learning. Lulu. com; 2020."},{"issue":"1","key":"579_CR23","doi-asserted-by":"publisher","first-page":"17","DOI":"10.3390\/a13010017","volume":"13","author":"E Pintelas","year":"2020","unstructured":"Pintelas E, Livieris IE, Pintelas P. A grey-box ensemble model exploiting black-box accuracy and white-box intrinsic interpretability. Algorithms. 2020;13(1):17.","journal-title":"Algorithms."},{"key":"579_CR24","doi-asserted-by":"crossref","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should i trust you?\u201d Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016. p. 1135\u20131144.","DOI":"10.1145\/2939672.2939778"},{"key":"579_CR25","unstructured":"Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Proceedings of the 31st international conference on neural information processing systems; 2017. p. 4768\u20134777."},{"issue":"1","key":"579_CR26","doi-asserted-by":"publisher","first-page":"73","DOI":"10.25300\/MISQ\/2014\/38.1.04","volume":"38","author":"D Martens","year":"2014","unstructured":"Martens D, Provost F. Explaining data-driven document classifications. MIS Quart. 2014;38(1):73\u2013100.","journal-title":"MIS Quart."},{"issue":"5","key":"579_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3236009","volume":"51","author":"R Guidotti","year":"2018","unstructured":"Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. A survey of methods for explaining black box models. ACM Comput Surv. 2018;51(5):1\u201342.","journal-title":"ACM Comput Surv"},{"issue":"1","key":"579_CR28","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1016\/j.dss.2010.12.003","volume":"51","author":"J Huysmans","year":"2011","unstructured":"Huysmans J, Dejaeger K, Mues C, Vanthienen J, Baesens B. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Syst. 2011;51(1):141\u201354.","journal-title":"Decision Support Syst"},{"key":"579_CR29","unstructured":"Allahyari H, Lavesson N. User-oriented assessment of classification model understandability. In: 11th scandinavian conference on Artificial intelligence. IOS Press; 2011."},{"key":"579_CR30","doi-asserted-by":"crossref","unstructured":"Askira-Gelman I. Knowledge discovery: comprehensibility of the results. In: Proceedings of the thirty-first Hawaii international conference on system sciences. vol.\u00a05. IEEE; 1998. p. 247\u2013255.","DOI":"10.1109\/HICSS.1998.648319"},{"key":"579_CR31","unstructured":"Bibal A, Fr\u00e9nay B. Interpretability of machine learning models and representations: an introduction. In: ESANN; 2016. ."},{"key":"579_CR32","doi-asserted-by":"crossref","unstructured":"Freitas AA. Automated machine learning for studying the trade-off between predictive accuracy and interpretability. In: International Cross-Domain Conference for Machine Learning and Knowledge Extraction. Springer; 2019. p. 48\u201366.","DOI":"10.1007\/978-3-030-29726-8_4"},{"key":"579_CR33","unstructured":"R\u00fcping S, et\u00a0al. Learning interpretable models. Universit\u00e4t Dortmund. 2006."},{"issue":"3","key":"579_CR34","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","volume":"16","author":"ZC Lipton","year":"2018","unstructured":"Lipton ZC. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue. 2018;16(3):31\u201357.","journal-title":"Queue."},{"issue":"5","key":"579_CR35","first-page":"e1379","volume":"10","author":"G Stiglic","year":"2020","unstructured":"Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. Interpretability of machine learning-based prediction models in healthcare. Wiley Interdiscipl Rev. 2020;10(5):e1379.","journal-title":"Wiley Interdiscipl Rev"},{"issue":"2","key":"579_CR36","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1037\/h0043158","volume":"63","author":"GA Miller","year":"1956","unstructured":"Miller GA. The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev. 1956;63(2):81.","journal-title":"Psychol Rev."},{"key":"579_CR37","unstructured":"Confalonieri R, Weyde T, Besold TR, Mart\u00edn FMdP. Trepan Reloaded: A Knowledge-driven Approach to Explaining Artificial Neural Networks. arXiv preprint arXiv:190608362. 2019."},{"key":"579_CR38","doi-asserted-by":"crossref","unstructured":"Ramon Y, Martens D, Evgeniou T, Praet S. Can metafeatures help improve explanations of prediction models when using behavioral and textual data? Machine Learning. 2021;p. 1\u201340.","DOI":"10.1007\/s10994-021-05981-0"},{"issue":"6","key":"579_CR39","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1057\/palgrave.jors.2601545","volume":"54","author":"B Baesens","year":"2003","unstructured":"Baesens B, Van Gestel T, Viaene S, Stepanova M, Suykens J, Vanthienen J. Benchmarking state-of-the-art classification algorithms for credit scoring. J Oper Res Soc. 2003;54(6):627\u201335.","journal-title":"J Oper Res Soc."},{"issue":"2","key":"579_CR40","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1017\/S026988890200019X","volume":"17","author":"C Lacave","year":"2002","unstructured":"Lacave C, D\u00edez FJ. A review of explanation methods for Bayesian networks. Knowl Eng Rev. 2002;17(2):107\u201327.","journal-title":"Knowl Eng Rev."},{"key":"579_CR41","unstructured":"Chubarian K, Tur\u00e1n G. Interpretability of Bayesian Network Classifiers: OBDD Approximation and Polynomial Threshold Functions. In: ISAIM; 2020."},{"key":"579_CR42","unstructured":"Garc\u00eda IdCG. Self-labeling Grey-box Model: An Interpretable Semi-supervised Classifier [Ph.D. thesis]. Queens University Belfast, United Kingdom; 2020."},{"issue":"1","key":"579_CR43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13040-017-0154-4","volume":"10","author":"RS Olson","year":"2017","unstructured":"Olson RS, La Cava W, Orzechowski P, Urbanowicz RJ, Moore JH. PMLB: a large benchmark suite for machine learning evaluation and comparison. BioData Mining. 2017;10(1):1\u201313.","journal-title":"BioData Mining."},{"issue":"1","key":"579_CR44","first-page":"3133","volume":"15","author":"M Fern\u00e1ndez-Delgado","year":"2014","unstructured":"Fern\u00e1ndez-Delgado M, Cernadas E, Barro S, Amorim D. Do we need hundreds of classifiers to solve real world classification problems? J Mach Learn Res. 2014;15(1):3133\u201381.","journal-title":"J Mach Learn Res"},{"key":"579_CR45","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.eswa.2017.04.003","volume":"82","author":"C Zhang","year":"2017","unstructured":"Zhang C, Liu C, Zhang X, Almpanidis G. An up-to-date comparison of state-of-the-art classification algorithms. Expert Syst Appl. 2017;82:128\u201350.","journal-title":"Expert Syst Appl."},{"issue":"1","key":"579_CR46","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1016\/j.ejor.2015.05.030","volume":"247","author":"S Lessmann","year":"2015","unstructured":"Lessmann S, Baesens B, Seow HV, Thomas LC. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. Eur J Oper Res. 2015;247(1):124\u201336.","journal-title":"Eur J Oper Res"},{"issue":"24","key":"579_CR47","doi-asserted-by":"publisher","first-page":"5441","DOI":"10.1039\/C8SC00148K","volume":"9","author":"A Mayr","year":"2018","unstructured":"Mayr A, Klambauer G, Unterthiner T, Steijaert M, Wegner JK, Ceulemans H, et al. Large-scale comparison of machine learning methods for drug target prediction on ChEMBL. Chem Sci. 2018;9(24):5441\u201351.","journal-title":"Chem Sci"},{"issue":"5","key":"579_CR48","doi-asserted-by":"publisher","first-page":"5268","DOI":"10.1016\/j.eswa.2010.10.031","volume":"38","author":"AC Lorena","year":"2011","unstructured":"Lorena AC, Jacintho LF, Siqueira MF, De Giovanni R, Lohmann LG, De Carvalho AC, et al. Comparing machine learning classifiers in potential distribution modelling. Exp Syst Appl. 2011;38(5):5268\u201375.","journal-title":"Exp Syst Appl."},{"key":"579_CR49","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.ins.2013.08.059","volume":"261","author":"N Maci\u00e0","year":"2014","unstructured":"Maci\u00e0 N, Bernad\u00f3-Mansilla E. Towards UCI+: a mindful repository design. Inform Sci. 2014;261:237\u201362.","journal-title":"Inform Sci"},{"key":"579_CR50","doi-asserted-by":"crossref","unstructured":"Fung G, Sandilya S, Rao RB. Rule extraction from linear support vector machines. In: Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining; 2005. p. 32\u201340.","DOI":"10.1145\/1081870.1081878"},{"key":"579_CR51","doi-asserted-by":"crossref","unstructured":"Martens D, Huysmans J, Setiono R, Vanthienen J, Baesens B. Rule extraction from support vector machines: an overview of issues and application in credit scoring. Rule extraction from support vector machines. 2008;p. 33\u201363.","DOI":"10.1007\/978-3-540-75390-2_2"},{"issue":"2","key":"579_CR52","doi-asserted-by":"publisher","first-page":"247","DOI":"10.3233\/IDA-2012-0522","volume":"16","author":"U Johansson","year":"2012","unstructured":"Johansson U, S\u00f6nstr\u00f6d C, L\u00f6fstr\u00f6m T, Bostr\u00f6m H. Obtaining accurate and comprehensible classifiers using oracle coaching. Intell Data Analy. 2012;16(2):247\u201363.","journal-title":"Intell Data Analy"},{"key":"579_CR53","doi-asserted-by":"crossref","unstructured":"Johansson U, S\u00f6nstr\u00f6d C, Accurate K\u00f6nig R. Interpretable regression trees using oracle coaching. In: IEEE Symposium on Computational Intelligence and Data Mining (CIDM). IEEE. 2014;2014:194\u2013201.","DOI":"10.1109\/CIDM.2014.7008667"},{"key":"579_CR54","first-page":"24","volume":"8","author":"M Craven","year":"1995","unstructured":"Craven M, Shavlik J. Extracting tree-structured representations of trained networks. Adv Neural Inform Process Syst. 1995;8:24\u201330.","journal-title":"Adv Neural Inform Process Syst"},{"issue":"2","key":"579_CR55","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1007\/BF02944803","volume":"19","author":"ZH Zhou","year":"2004","unstructured":"Zhou ZH. Rule extraction: using neural networks or for neural networks? J Comput Sci Technol. 2004;19(2):249\u201353.","journal-title":"J Comput Sci Technol"},{"issue":"2","key":"579_CR56","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1109\/TKDE.2008.131","volume":"21","author":"D Martens","year":"2008","unstructured":"Martens D, Baesens B, Van Gestel T. Decompositional rule extraction from support vector machines by active learning. IEEE Trans Knowl Data Eng. 2008;21(2):178\u201391.","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"579_CR57","first-page":"111","volume":"21","author":"E Alcoba\u00e7a","year":"2020","unstructured":"Alcoba\u00e7a E, Siqueira F, Rivolli A, Garcia LPF, Oliva JT, de Carvalho AC, et al. MFE: Towards reproducible meta-feature extraction. J Mach Learn Res. 2020;21:111\u20131.","journal-title":"J Mach Learn Res."},{"key":"579_CR58","unstructured":"Rivolli A, Garcia LP, Soares C, Vanschoren J, de\u00a0Carvalho AC. Characterizing classification datasets: a study of meta-features for meta-learning. arXiv preprint arXiv:180810406. 2018."},{"issue":"5","key":"579_CR59","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3347711","volume":"52","author":"AC Lorena","year":"2019","unstructured":"Lorena AC, Garcia LP, Lehmann J, Souto MC, Ho TK. How Complex is your classification problem? A survey on measuring classification complexity. ACM Comput Surv. 2019;52(5):1\u201334.","journal-title":"ACM Comput Surv"},{"key":"579_CR60","unstructured":"Singh A, Thakur N, Sharma A. A review of supervised machine learning algorithms. In: 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom). IEEE; 2016. p. 1310\u20131315."},{"key":"579_CR61","doi-asserted-by":"crossref","unstructured":"Cohen WW. Fast effective rule induction. In: Machine learning proceedings 1995. Elsevier; 1995. p. 115\u2013123.","DOI":"10.1016\/B978-1-55860-377-6.50023-2"},{"issue":"3","key":"579_CR62","first-page":"916","volume":"2","author":"JH Friedman","year":"2008","unstructured":"Friedman JH, Popescu BE. Predictive learning via rule ensembles. Annal Appl Stat. 2008;2(3):916\u201354.","journal-title":"Annal Appl Stat"},{"key":"579_CR63","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar J. Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res. 2006;7:1\u201330.","journal-title":"J Mach Learn Res"},{"key":"579_CR64","unstructured":"Nemenyi PB. Distribution-free multiple comparisons. Princeton University; 1963."},{"key":"579_CR65","doi-asserted-by":"publisher","first-page":"867","DOI":"10.2478\/v10006-012-0064-z","volume":"22","author":"B Trawi\u0144ski","year":"2012","unstructured":"Trawi\u0144ski B, Sm\u0119tek M, Telec Z, Lasota T. Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms. Int J Appl Math Comput Sci. 2012;22:867\u201381.","journal-title":"Int J Appl Math Comput Sci"},{"issue":"1","key":"579_CR66","first-page":"2349","volume":"14","author":"J Dem\u0161ar","year":"2013","unstructured":"Dem\u0161ar J, Curk T, Erjavec A, Gorup \u010c, Ho\u010devar T, Milutinovi\u010d M, et al. Orange: data mining toolbox in Python. J Mach Learn Res. 2013;14(1):2349\u201353.","journal-title":"J Mach Learn Res."},{"issue":"11","key":"579_CR67","doi-asserted-by":"publisher","first-page":"2664","DOI":"10.1109\/TNNLS.2015.2389037","volume":"26","author":"EJ de Fortuny","year":"2015","unstructured":"de Fortuny EJ, Martens D. Active learning-based pedagogical rule extraction. IEEE Trans Neural Netw Learn Syst. 2015;26(11):2664\u201377.","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"579_CR68","unstructured":"Michie D, Spiegelhalter DJ, Taylor CC. Machine learning, neural and statistical classification. Citeseer; 1994."},{"issue":"1","key":"579_CR69","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1007\/s10115-013-0700-4","volume":"42","author":"J Luengo","year":"2015","unstructured":"Luengo J, Herrera F. An automatic extraction method of the domains of competence for learning classifiers using data complexity measures. Knowl Inform Syst. 2015;42(1):147\u201380.","journal-title":"Knowl Inform Syst"},{"key":"579_CR70","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi A, Berrada M. Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access. 2018;6:52138\u201360.","journal-title":"IEEE Access."},{"key":"579_CR71","first-page":"269","volume":"2020","author":"C Schwartzenberg","year":"2020","unstructured":"Schwartzenberg C, van Engers T, Li Y. The fidelity of global surrogates in interpretable Machine Learning. BNAIC\/BeneLearn. 2020;2020:269.","journal-title":"BNAIC\/BeneLearn"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-022-00579-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-022-00579-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-022-00579-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,19]],"date-time":"2024-09-19T21:25:12Z","timestamp":1726781112000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-022-00579-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,7]]},"references-count":71,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["579"],"URL":"https:\/\/doi.org\/10.1186\/s40537-022-00579-2","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,7]]},"assertion":[{"value":"7 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"30"}}