{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:28:47Z","timestamp":1772119727396,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T00:00:00Z","timestamp":1754438400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T00:00:00Z","timestamp":1754438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004329","name":"The Slovenian Research and Innovation Agency","doi-asserted-by":"publisher","award":["P2-0103"],"award-info":[{"award-number":["P2-0103"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]},{"name":"European Union's Horizon Europe research and innovation programme under the Marie Sk\u0142odowska-Curie Postdoctoral Fellowship Programme, SMASH","award":["101081355"],"award-info":[{"award-number":["101081355"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Bias in machine learning models remains a critical challenge, particularly in datasets with numeric features where discrimination may be subtle and hard to detect. Existing fairness frameworks rely on expert knowledge of marginalized groups, such as specific racial groups, and categorical features defining them. Furthermore, most frameworks evaluate bias in models rather than datasets, despite the fact that model bias can often be traced back to dataset shortcomings. Our research aims to remedy this gap by capturing dataset flaws in a set of meta-features at the dataset level, and to warn practitioners of bias risk when using such datasets for model training. We neither restrict the feature type nor expect domain knowledge. To this end, we develop methods to synthesize biased datasets and extend current fairness metrics to continuous features in order to quantify dataset-level discrimination risks. Our approach constructs a meta-database of diverse datasets, from which we derive transferable meta-features that capture dataset properties indicative of bias risk. Our findings demonstrate that dataset-level characteristics can serve as cost-effective indicators of bias risk, providing a novel method for data auditing that does not rely on expert knowledge. This work lays the foundation for early-warning systems, moving beyond model-focused assessments toward a data-centric approach.<\/jats:p>","DOI":"10.1007\/s10994-025-06843-9","type":"journal-article","created":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T20:16:56Z","timestamp":1754511416000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Assessing the risk of discriminatory bias in classification datasets"],"prefix":"10.1007","volume":"114","author":[{"given":"Kejun","family":"Dai","sequence":"first","affiliation":[]},{"given":"Jonathan","family":"Kim","sequence":"additional","affiliation":[]},{"given":"Sa\u0161o","family":"D\u017eeroski","sequence":"additional","affiliation":[]},{"given":"J\u00f6rg","family":"Wicker","sequence":"additional","affiliation":[]},{"given":"Gillian","family":"Dobbie","sequence":"additional","affiliation":[]},{"given":"Katharina","family":"Dost","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,6]]},"reference":[{"issue":"1","key":"6843_CR1","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1038\/s41698-020-0122-1","volume":"4","author":"G Adam","year":"2020","unstructured":"Adam, G., Ramp\u00e1\u0161ek, L., Safikhani, Z., Smirnov, P., Haibe-Kains, B., & Goldenberg, A. (2020). Machine learning approaches to drug response prediction: Challenges and recent progress. NPJ Precision Oncology, 4(1), 19.","journal-title":"NPJ Precision Oncology"},{"key":"6843_CR2","doi-asserted-by":"publisher","first-page":"6271","DOI":"10.3390\/app11146271","volume":"11","author":"S Alelyani","year":"2021","unstructured":"Alelyani, S. (2021). Detection and evaluation of machine learning bias. Applied Sciences, 11, 6271.","journal-title":"Applied Sciences"},{"key":"6843_CR3","unstructured":"Angwin, J., Larson, S. M. J., & Lauren\u00a0Kirchner, P. (2016). Machine bias. www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing. Accessed July 24, 2024."},{"issue":"4","key":"6843_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3552433","volume":"14","author":"F Azzalini","year":"2022","unstructured":"Azzalini, F., Criscuolo, C., & Tanca, L. (2022). E-fair-db: Functional dependencies to discover data bias and enhance data equity. Journal of Data and Information Quality, 14(4), 1\u201326.","journal-title":"Journal of Data and Information Quality"},{"key":"6843_CR5","unstructured":"Balte, A., Pise, N., & Kulkarni, P. (2014). Meta-learning with landmarking: A survey. International Journal of Computer Applications,105(8)."},{"key":"6843_CR6","doi-asserted-by":"publisher","first-page":"4619","DOI":"10.3390\/app12094619","volume":"12","author":"E Barbierato","year":"2022","unstructured":"Barbierato, E., Vedova, M. L. D., Tessera, D., Toti, D., & Vanoli, N. (2022). A methodology for controlling bias and fairness in synthetic data generation. Applied Sciences, 12, 4619.","journal-title":"Applied Sciences"},{"key":"6843_CR7","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1147\/JRD.2019.2942287","volume":"63","author":"RK Bellamy","year":"2019","unstructured":"Bellamy, R. K., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovi\u0107, A., et al. (2019). AI fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM Journal of Research and Development, 63, 4\u20131.","journal-title":"IBM Journal of Research and Development"},{"issue":"6334","key":"6843_CR8","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1126\/science.aal4230","volume":"356","author":"A Caliskan","year":"2017","unstructured":"Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183\u2013186.","journal-title":"Science"},{"key":"6843_CR9","doi-asserted-by":"publisher","first-page":"1172","DOI":"10.1093\/jamia\/ocae060","volume":"31","author":"F Chen","year":"2024","unstructured":"Chen, F., Wang, L., Hong, J., Jiang, J., & Zhou, L. (2024). Unmasking bias in artificial intelligence: A systematic review of bias detection and mitigation strategies in electronic health record-based models. Journal of the American Medical Informatics Association, 31, 1172\u20131183.","journal-title":"Journal of the American Medical Informatics Association"},{"key":"6843_CR10","first-page":"1","volume":"7","author":"J Dem\u0161ar","year":"2006","unstructured":"Dem\u0161ar, J. (2006). Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research,\n7, 1\u201330.","journal-title":"The Journal of Machine Learning Research"},{"key":"6843_CR11","doi-asserted-by":"crossref","unstructured":"Dost, K., Taskova, K., Riddle, P., & Wicker, J. (2020). Your best guess when you know nothing: Identification and mitigation of selection bias. In Proceedings of the 2020 IEEE International Conference on Data Mining (pp. 996\u20131001).","DOI":"10.1109\/ICDM50108.2020.00115"},{"key":"6843_CR12","unstructured":"Faliagka, E., Ramantas, K., Tsakalidis, A., & Tzimas, G. (2012). Application of machine learning algorithms to an online recruitment system. In Proc. International Conference on Internet and Web Applications and Services (pp. 215\u2013220)."},{"key":"6843_CR13","doi-asserted-by":"crossref","unstructured":"Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 259\u2013268).","DOI":"10.1145\/2783258.2783311"},{"key":"6843_CR14","unstructured":"Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems, 29."},{"key":"6843_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3631326","volume":"1","author":"M Hort","year":"2024","unstructured":"Hort, M., Chen, Z., Zhang, J. M., Harman, M., & Sarro, F. (2024). Bias mitigation for machine learning classifiers: A comprehensive survey. ACM Journal on Responsible Computing, 1, 1\u201352.","journal-title":"ACM Journal on Responsible Computing"},{"key":"6843_CR16","first-page":"33563","volume":"35","author":"S Jesus","year":"2022","unstructured":"Jesus, S., Pombal, J., Alves, D., Cruz, A., Saleiro, P., Ribeiro, R., Gama, J., & Bizarro, P. (2022). Turning the tables: Biased, imbalanced, dynamic tabular datasets for ml evaluation. Advances in Neural Information Processing Systems, 35, 33563\u201333575.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6843_CR17","doi-asserted-by":"crossref","unstructured":"Jiang, L., Belitz, C., & Bosch, N. (2024). Synthetic dataset generation for fairer unfairness research. In Proceedings of the 14th Learning Analytics and Knowledge Conference (pp. 200\u2013209).","DOI":"10.1145\/3636555.3636868"},{"key":"6843_CR18","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1080\/01973533.2012.711687","volume":"34","author":"AH Jordan","year":"2012","unstructured":"Jordan, A. H., & Zitek, E. M. (2012). Marital status bias in perceptions of employees. Basic and Applied Social Psychology, 34, 474\u2013481.","journal-title":"Basic and Applied Social Psychology"},{"key":"6843_CR19","doi-asserted-by":"crossref","unstructured":"Kalousis, A., & Hilario, M. (2000). Model selection via meta-learning: A comparative study. In Proceedings 12th IEEE Internationals Conference on Tools with Artificial Intelligence (pp. 406\u2013413).","DOI":"10.1109\/TAI.2000.889901"},{"key":"6843_CR20","doi-asserted-by":"publisher","first-page":"2966","DOI":"10.1287\/mnsc.2018.3093","volume":"65","author":"A Lambrecht","year":"2019","unstructured":"Lambrecht, A., & Tucker, C. (2019). Algorithmic bias? An empirical study of apparent gender-based discrimination in the display of STEM career ads. Management Science, 65, 2966\u20132981.","journal-title":"Management Science"},{"key":"6843_CR21","first-page":"1452","volume":"12","author":"T Le Quy","year":"2022","unstructured":"Le Quy, T., Roy, A., Iosifidis, V., Zhang, W., & Ntoutsi, E. (2022). A survey on datasets for fairness-aware machine learning. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 12, 1452.","journal-title":"Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery"},{"key":"6843_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3347711","volume":"52","author":"AC Lorena","year":"2019","unstructured":"Lorena, A. C., Garcia, L. P., Lehmann, J., Souto, M. C., & Ho, T. K. (2019). How complex is your classification problem? A survey on measuring classification complexity. Acm Computing Surveys,\n52, 1\u201334.","journal-title":"Acm Computing Surveys"},{"issue":"6","key":"6843_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3457607","volume":"54","author":"N Mehrabi","year":"2021","unstructured":"Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. Acm Computing Surveys,\n54(6), 1\u201335.","journal-title":"Acm Computing Surveys"},{"issue":"6","key":"6843_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3457607","volume":"54","author":"N Mehrabi","year":"2021","unstructured":"Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys,\n54(6), 1\u201335.","journal-title":"ACM Computing Surveys"},{"key":"6843_CR25","volume-title":"Machine learning, neural and statistical classification","author":"D Michie","year":"1995","unstructured":"Michie, D., Spiegelhalter, D. J., Taylor, C. C., & Campbell, J. (1995). Machine learning, neural and statistical classification. Ellis Horwood."},{"key":"6843_CR26","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","volume":"45","author":"JG Moreno-Torres","year":"2012","unstructured":"Moreno-Torres, J. G., Raeder, T., Alaiz-Rodr\u00edguez, R., Chawla, N. V., & Herrera, F. (2012). A unifying view on dataset shift in classification. Pattern Recognition, 45, 521\u2013530.","journal-title":"Pattern Recognition"},{"key":"6843_CR27","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1111\/1475-3995.00375","volume":"9","author":"A Mukerjee","year":"2002","unstructured":"Mukerjee, A., Biswas, R., Deb, K., & Mathur, A. P. (2002). Multi-objective evolutionary algorithms for the risk-return trade-off in bank loan management. International Transactions in Operational Research, 9, 583\u2013597.","journal-title":"International Transactions in Operational Research"},{"issue":"6","key":"6843_CR28","doi-asserted-by":"publisher","first-page":"1179","DOI":"10.1007\/s10994-019-05829-8","volume":"109","author":"M Petkovi\u0107","year":"2020","unstructured":"Petkovi\u0107, M., Kocev, D., & D\u017eeroski, S. (2020). Feature ranking for multi-target regression. Machine Learning, 109(6), 1179\u20131204.","journal-title":"Machine Learning"},{"key":"6843_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.softx.2023.101526","volume":"24","author":"M Petkovi\u0107","year":"2023","unstructured":"Petkovi\u0107, M., Levati\u0107, J., Kocev, D., Breskvar, M., & D\u017eeroski, S. (2023). CLUSplus: A decision tree-based framework for predicting structured outputs. SoftwareX, 24, Article 101526.","journal-title":"SoftwareX"},{"key":"6843_CR30","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1007\/s10994-011-5256-5","volume":"85","author":"J Read","year":"2011","unstructured":"Read, J., Pfahringer, B., Holmes, G., & Frank, E. (2011). Classifier chains for multi-label classification. Machine Learning, 85, 333\u2013359.","journal-title":"Machine Learning"},{"key":"6843_CR31","unstructured":"Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K. T., & Ghani, R. (2018). Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577."},{"key":"6843_CR32","unstructured":"Simonetta, A., Trenta, A., Paoletti, M. C., & Vetr\u00f2, A., et al. (2021). Metrics for identifying bias in datasets. In International Conference of Yearly Reports on Informatics Mathematics and Engineering (pp. 10\u201317)."},{"issue":"D1","key":"6843_CR33","doi-asserted-by":"publisher","first-page":"994","DOI":"10.1093\/nar\/gkx911","volume":"46","author":"P Smirnov","year":"2018","unstructured":"Smirnov, P., Kofia, V., Maru, A., Freeman, M., Ho, C., El-Hachem, N., Adam, G.-A., Ba-Alawi, W., Safikhani, Z., & Haibe-Kains, B. (2018). Pharmacodb: An integrative database for mining in vitro anticancer drug screening studies. Nucleic Acids Research,\n46(D1), 994\u20131002.","journal-title":"Nucleic Acids Research"},{"key":"6843_CR34","doi-asserted-by":"crossref","unstructured":"Speicher, T., Heidari, H., Grgic-Hlaca, N., Gummadi, K. P., Singla, A., Weller, A., & Zafar, M. B. (2018). A unified approach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (pp. 2239\u20132248).","DOI":"10.1145\/3219819.3220046"},{"key":"6843_CR35","doi-asserted-by":"crossref","unstructured":"Struyf, J., spsampsps D\u017eeroski, S. (2006). Constraint based induction of multi-objective regression trees. In Knowledge Discovery in Inductive Databases (pp. 222\u2013233).","DOI":"10.1007\/11733492_13"},{"key":"6843_CR36","first-page":"31","volume":"1","author":"R Vilalta","year":"2004","unstructured":"Vilalta, R., Giraud-Carrier, C. G., Brazdil, P., & Soares, C. (2004). Using meta-learning to support data mining. International Journal of Computer Science and Applications,\n1, 31\u201345.","journal-title":"International Journal of Computer Science and Applications"},{"key":"6843_CR37","doi-asserted-by":"publisher","first-page":"2473","DOI":"10.1016\/j.eswa.2007.12.020","volume":"36","author":"I-C Yeh","year":"2009","unstructured":"Yeh, I.-C., & Lien, C.-H. (2009). The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients. Expert Systems with Applications,\n36, 2473\u20132480.","journal-title":"Expert Systems with Applications"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-025-06843-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-025-06843-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-025-06843-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,8]],"date-time":"2025-09-08T22:11:09Z","timestamp":1757369469000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-025-06843-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,6]]},"references-count":37,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["6843"],"URL":"https:\/\/doi.org\/10.1007\/s10994-025-06843-9","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-6370375\/v1","asserted-by":"object"}]},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,6]]},"assertion":[{"value":"3 April 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 July 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 August 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Kim is employed by Callaghan Innovation, a Crown Research Institute in New Zealand.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"204"}}