{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T06:51:45Z","timestamp":1782802305058,"version":"3.54.5"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T00:00:00Z","timestamp":1736812800000},"content-version":"vor","delay-in-days":13,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Food Agility Cooperative Research Center"},{"DOI":"10.13039\/501100001769","name":"Charles Sturt University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001769","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Over the last few decades, machine learning (ML) applications have grown exponentially, yielding several benefits to society. However, these benefits are tempered with concerns of discriminatory behaviours exhibited by ML models. In this regard, fairness in machine learning has emerged as a priority research area. Consequently, several fairness metrics and algorithms have been developed to mitigate against discriminatory behaviours that ML models may possess. Yet still, very little attention has been paid to the problem of naturally occurring changes in data patterns (<jats:italic>aka<\/jats:italic> data distributional drift), and its impact on fairness algorithms and metrics. In this work, we study this problem comprehensively by analyzing 4 fairness-unaware baseline algorithms and 7 fairness-aware algorithms, carefully curated to cover the breadth of its typology, across 5 datasets including public and proprietary data, and evaluated them using 3 predictive performance and 10 fairness metrics. In doing so, we show that (1) data distributional drift is not a trivial occurrence, and in several cases can lead to serious deterioration of fairness in so-called fair models; (2) contrary to some existing literature, the size and direction of data distributional drift is not correlated to the resulting size and direction of unfairness; and (3) choice of, and training of fairness algorithms is impacted by the effect of data distributional drift which is largely ignored in the literature. Emanating from our findings, we synthesize several policy implications of data distributional drift on fairness algorithms that can be very relevant to stakeholders and practitioners.<\/jats:p>","DOI":"10.1007\/s10994-024-06698-6","type":"journal-article","created":{"date-parts":[[2025,1,14]],"date-time":"2025-01-14T22:46:52Z","timestamp":1736894812000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Is it still fair? A comparative evaluation of fairness algorithms through the lens of covariate drift"],"prefix":"10.1007","volume":"114","author":[{"given":"Oscar Blessed","family":"Deho","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Bewong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Selasi","family":"Kwashie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiuyong","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jixue","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Srecko","family":"Joksimovic","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,14]]},"reference":[{"key":"6698_CR1","first-page":"77","volume":"23","author":"J Angwin","year":"2016","unstructured":"Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine bias: There\u2019s software used across the country to predict future criminals. And it\u2019s biased against blacks ProPublica, 23, 77.","journal-title":"And it\u2019s biased against blacks ProPublica"},{"key":"6698_CR2","doi-asserted-by":"publisher","first-page":"C5XW20","DOI":"10.24432\/C5XW20","volume":"10","author":"B Becker","year":"1996","unstructured":"Becker, B., & Kohavi, R. (1996). Adult. UCI Machine Learning Repository, 10, C5XW20. https:\/\/doi.org\/10.24432\/C5XW20","journal-title":"UCI Machine Learning Repository"},{"issue":"1","key":"6698_CR3","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0049124118782533","volume":"50","author":"R Berk","year":"2021","unstructured":"Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1), 3\u201344.","journal-title":"Sociological Methods & Research"},{"key":"6698_CR4","first-page":"11266","volume":"35","author":"Y Chen","year":"2022","unstructured":"Chen, Y., Raab, R., Wang, J., & Liu, Y. (2022). Fairness transferability subject to bounded distribution shift. Advances in Neural Information Processing Systems, 35, 11266\u201311278.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"2","key":"6698_CR5","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1089\/big.2016.0047","volume":"5","author":"A Chouldechova","year":"2017","unstructured":"Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153\u2013163.","journal-title":"Big Data"},{"issue":"4","key":"6698_CR6","doi-asserted-by":"publisher","first-page":"822","DOI":"10.1111\/bjet.13217","volume":"53","author":"OB Deho","year":"2022","unstructured":"Deho, O. B., Zhan, C., Li, J., Liu, J., Liu, L., & Le Duy, T. (2022). How do the existing fairness metrics and unfairness mitigation algorithms contribute to ethical learning analytics? British Journal of Educational Technology, 53(4), 822\u2013843.","journal-title":"British Journal of Educational Technology"},{"key":"6698_CR7","doi-asserted-by":"publisher","first-page":"1858","DOI":"10.1109\/TIT.2003.813506","volume":"49","author":"D Endres","year":"2003","unstructured":"Endres, D., & Schindelin, J. E. (2003). A new metric for probability distributions. IEEE Transactions on Information Theory, 49, 1858\u20131860.","journal-title":"IEEE Transactions on Information Theory"},{"issue":"1","key":"6698_CR8","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1086\/699936","volume":"19","author":"J Furman","year":"2019","unstructured":"Furman, J., & Seamans, R. (2019). Ai and the economy. Innovation policy and the economy, 19(1), 161\u2013191.","journal-title":"Innovation policy and the economy"},{"key":"6698_CR9","first-page":"3315","volume":"29","author":"M Hardt","year":"2016","unstructured":"Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems, 29, 3315\u20133323.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"1","key":"6698_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.ssaho.2021.100177","volume":"4","author":"C Impey","year":"2021","unstructured":"Impey, C., & Formanek, M. (2021). Moocs and 100 days of covid: Enrollment surges in massive open online astronomy classes during the coronavirus pandemic. Social Sciences & Humanities Open, 4(1), 100177.","journal-title":"Social Sciences & Humanities Open"},{"key":"6698_CR11","first-page":"33563","volume":"35","author":"S Jesus","year":"2022","unstructured":"Jesus, S., Pombal, J., Alves, D., et al. (2022). Turning the tables: Biased, imbalanced, dynamic tabular datasets for ml evaluation. Advances in Neural Information Processing Systems, 35, 33563\u201333575.","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"1","key":"6698_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-011-0463-8","volume":"33","author":"F Kamiran","year":"2012","unstructured":"Kamiran, F., & Calders, T. (2012). Data preprocessing techniques for classification without discrimination. Knowledge and Information Systems, 33(1), 1\u201333.","journal-title":"Knowledge and Information Systems"},{"key":"6698_CR13","first-page":"25773","volume":"34","author":"S Maity","year":"2021","unstructured":"Maity, S., Mukherjee, D., Yurochkin, M., et al. (2021). Does enforcing fairness mitigate biases caused by subpopulation shift? Advances in Neural Information Processing Systems, 34, 25773\u201325784.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6698_CR14","first-page":"1","volume":"36","author":"S Majumder","year":"2021","unstructured":"Majumder, S., Chakraborty, J., Bai, G. R., et al. (2021). Fair enough: Searching for sufficient measures of fairness. ACM Transactions on Software Engineering and Methodology., 36, 1.","journal-title":"ACM Transactions on Software Engineering and Methodology."},{"issue":"6","key":"6698_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3457607","volume":"54","author":"N Mehrabi","year":"2021","unstructured":"Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1\u201335.","journal-title":"ACM Computing Surveys (CSUR)"},{"issue":"1","key":"6698_CR16","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.patcog.2011.06.019","volume":"45","author":"JG Moreno-Torres","year":"2012","unstructured":"Moreno-Torres, J. G., Raeder, T., Alaiz-Rodr\u00edguez, R., et al. (2012). A unifying view on dataset shift in classification. Pattern Recognition, 45(1), 521\u2013530.","journal-title":"Pattern Recognition"},{"key":"6698_CR17","first-page":"5680","volume":"30","author":"G Pleiss","year":"2017","unstructured":"Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., & Weinberger, K. Q. (2017). On fairness and calibration. Advances in Neural Information Processing Systems, 30, 5680\u20135689.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"6698_CR18","doi-asserted-by":"publisher","first-page":"1470","DOI":"10.2307\/4093259","volume":"117","author":"RB Siegel","year":"2003","unstructured":"Siegel, R. B. (2003). Equality talk: Antisubordination and anticlassification values in constitutional struggles over brown. Harvard Law Review, 117, 1470.","journal-title":"Harvard Law Review"},{"key":"6698_CR19","doi-asserted-by":"crossref","unstructured":"Castelnovo, A., Malandri, L., Mercorio, F Mezzanzanica, M., and Cosentini, A. (2021). Towards fairness through time. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, pp 647\u2013663","DOI":"10.1007\/978-3-030-93736-2_46"},{"key":"6698_CR20","doi-asserted-by":"crossref","unstructured":"Du, W., Wu, X. (2021). Fair and robust classification under sample selection bias. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp 2999\u20133003.","DOI":"10.1145\/3459637.3482104"},{"key":"6698_CR21","doi-asserted-by":"crossref","unstructured":"Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp 214\u2013226.","DOI":"10.1145\/2090236.2090255"},{"key":"6698_CR22","doi-asserted-by":"crossref","unstructured":"Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015). Certifying and removing disparate impact. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 259\u2013268.","DOI":"10.1145\/2783258.2783311"},{"key":"6698_CR23","doi-asserted-by":"crossref","unstructured":"Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., and Roth, D. (2017). A comparative study of fairness-enhancing interventions in machine learning. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp 329\u2013338.","DOI":"10.1145\/3287560.3287589"},{"key":"6698_CR24","doi-asserted-by":"crossref","unstructured":"Gardner, J., Yu, R., Nguyen, Q., rooks, C., and Kizilcec, R. (2023). Cross-institutional transfer learning for educational models: Implications for model performance, fairness, and equity. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp 1664\u20131684.","DOI":"10.1145\/3593013.3594107"},{"key":"6698_CR25","unstructured":"Ghodsi, S., Alani, H., Ntoutsi, E. (2022). Context matters for fairness\u2013a case study on the effect of spatial distribution shifts. arXiv preprint arXiv:2206.11436."},{"key":"6698_CR26","unstructured":"Hamilton, E. (2017). Benchmarking four approaches to fairness-aware machine learning. PhD thesis."},{"key":"6698_CR27","doi-asserted-by":"crossref","unstructured":"Henzinger, T., Karimi, M., Kueffner, K., Mallik, K. (2023a). Runtime monitoring of dynamic fairness properties. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp 604\u2013614.","DOI":"10.1145\/3593013.3594028"},{"key":"6698_CR28","doi-asserted-by":"crossref","unstructured":"Henzinger, T. A., Kueffner, K., Mallik, K. (2023b). Monitoring algorithmic fairness under partial observations. In: International Conference on Runtime Verification, Springer, pp 291\u2013311.","DOI":"10.1007\/978-3-031-44267-4_15"},{"key":"6698_CR29","doi-asserted-by":"crossref","unstructured":"Islam, M. T., Fariha, A., Meliou, A., Salimi, B. (2022). Through the data management lens: Experimental analysis and evaluation of fair classification. In: Proceedings of the 2022 International Conference on Management of Data, pp 232\u2013246.","DOI":"10.1145\/3514221.3517841"},{"key":"6698_CR30","doi-asserted-by":"crossref","unstructured":"Kamishima, T., Akaho, S., Asoh, H., Sakuma, J. (2012). Fairness-aware classifier with prejudice remover regularizer. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, pp 35\u201350.","DOI":"10.1007\/978-3-642-33486-3_3"},{"key":"6698_CR31","doi-asserted-by":"crossref","unstructured":"Kamp, S., Zhao, ALL., Kutty, S. (2021). Robustness of fairness: An experimental analysis. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, pp 591\u2013606.","DOI":"10.1007\/978-3-030-93736-2_43"},{"key":"6698_CR32","doi-asserted-by":"publisher","unstructured":"Li, Y., Wang, X., Xing, Y., Fan, S., Wang, R., Liu, Y., and Shi, C. (2024). Graph fairness learning under distribution shifts. In: Proceedings of the ACM on Web Conference 2024. Association for Computing Machinery, New York, NY, USA, WWW \u201924, p 676-684, https:\/\/doi.org\/10.1145\/3589334.3645508.","DOI":"10.1145\/3589334.3645508"},{"key":"6698_CR33","doi-asserted-by":"crossref","unstructured":"Rezaei, A., Liu, A., Memarrast, O., Ziebart, B. D. (2021). Robust fairness under covariate shift. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp 9419\u20139427.","DOI":"10.1609\/aaai.v35i11.17135"},{"key":"6698_CR34","unstructured":"Roth, D. (2018). A comparison of fairness-aware machine learning algorithms. PhD thesis."},{"key":"6698_CR35","doi-asserted-by":"crossref","unstructured":"Speicher, T., Heidari, H., Grgic-Hlaca, N., Gummadi, K. P., Singla, A., Weller, A., and Zafar, M. B. (2018). A unified approach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 2239\u20132248.","DOI":"10.1145\/3219819.3220046"},{"key":"6698_CR36","unstructured":"Taskesen, B., Nguyen, V.A., Kuhn, D., Blanchet, J. (2020). A distributionally robust approach to fair classification. arXiv preprint arXiv:2007.09530."},{"key":"6698_CR37","doi-asserted-by":"crossref","unstructured":"Xu, Y., Wilson, K. (2021). Early alert systems during a pandemic: A simulation study on the impact of concept drift. In: LAK21: 11th International Learning Analytics and Knowledge Conference, pp 504\u2013510.","DOI":"10.1145\/3448139.3448190"},{"key":"6698_CR38","unstructured":"Yao, W., Zhou, Z., Li, Z., Han B, Liu Y. (2024). Understanding fairness surrogate functions in algorithmic fairness. Transactions on Machine Learning Research."},{"key":"6698_CR39","unstructured":"Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. (2013). Learning fair representations. In: International conference on machine learning, PMLR, pp 325\u2013333."},{"key":"6698_CR40","doi-asserted-by":"crossref","unstructured":"Zhang, B. H., Lemoine, B., Mitchell, M. (2018). Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, pp 335\u2013340.","DOI":"10.1145\/3278721.3278779"},{"key":"6698_CR41","doi-asserted-by":"crossref","unstructured":"Zien, A., Kr\u00e4mer, N., Sonnenburg, S., R\u00e4tsch, G. (2009). The feature importance ranking measure. In: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2009, Bled, Slovenia, September 7-11, 2009, Proceedings, Part II 20, Springer, pp 694\u2013709.","DOI":"10.1007\/978-3-642-04174-7_45"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06698-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-024-06698-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-024-06698-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T16:18:39Z","timestamp":1738945119000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-024-06698-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["6698"],"URL":"https:\/\/doi.org\/10.1007\/s10994-024-06698-6","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"26 May 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 December 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no Conflict of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The study was approved by the governing ethical board (Ethics Protocol Application ID: 204198).","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}}],"article-number":"8"}}