{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T16:08:12Z","timestamp":1780589292638,"version":"3.54.1"},"reference-count":54,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T00:00:00Z","timestamp":1622419200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,31]],"date-time":"2021-05-31T00:00:00Z","timestamp":1622419200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1934925"],"award-info":[{"award-number":["1934925"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1934494"],"award-info":[{"award-number":["1934494"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Collaborative on Gun Violence Research"},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1453392"],"award-info":[{"award-number":["1453392"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2021,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Algorithmic decision making is becoming more prevalent, increasingly impacting people\u2019s daily lives. Recently, discussions have been emerging about the fairness of decisions made by machines. Researchers have proposed different approaches for improving the fairness of these algorithms. While these approaches can help machines make fairer decisions, they have been developed and validated on fairly<jats:italic>clean<\/jats:italic>data sets. Unfortunately, most real-world data have complexities that make them more<jats:italic>dirty<\/jats:italic>. This work considers two of these complexities by analyzing the impact of two real-world data issues on fairness\u2014missing values and selection bias\u2014for categorical data. After formulating this problem and showing its existence, we propose fixing algorithms for data sets containing missing values and\/or selection bias that use different forms of reweighting and resampling based upon the missing value generation process. We conduct an extensive empirical evaluation on both real-world and synthetic data using various fairness metrics, and demonstrate how different missing values generated from different mechanisms and selection bias impact prediction fairness, even when prediction accuracy remains fairly constant.<\/jats:p>","DOI":"10.1007\/s41060-021-00259-z","type":"journal-article","created":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T04:07:33Z","timestamp":1622520453000},"page":"101-119","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["Analyzing the impact of missing values and selection bias on fairness"],"prefix":"10.1007","volume":"12","author":[{"given":"Yanchen","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa","family":"Singh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,5,31]]},"reference":[{"issue":"4","key":"259_CR1","doi-asserted-by":"publisher","first-page":"1012","DOI":"10.1111\/j.1741-3737.2005.00191.x","volume":"67","author":"AC Acock","year":"2005","unstructured":"Acock, A.C.: Working with missing values. J. Marriage Fam. 67(4), 1012\u20131028 (2005)","journal-title":"J. Marriage Fam."},{"key":"259_CR2","doi-asserted-by":"crossref","unstructured":"Allison, P.D.: Missing Data, vol. 136. Sage Publications (2001)","DOI":"10.4135\/9781412985079"},{"key":"259_CR3","unstructured":"American Bar Association: Disparate-impact claims under the adea (2019). https:\/\/www.americanbar.org\/groups\/gpsolo\/publications\/gp_solo\/2011\/september\/disparate_impact_claims_adea\/"},{"issue":"6","key":"259_CR4","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/3209581","volume":"61","author":"R Baeza-Yates","year":"2018","unstructured":"Baeza-Yates, R.: Bias on the web. Commun. ACM 61(6), 54\u201361 (2018)","journal-title":"Commun. ACM"},{"issue":"5","key":"259_CR5","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1111\/j.1467-842X.2001.tb00294.x","volume":"25","author":"DA Bennett","year":"2001","unstructured":"Bennett, D.A.: How can i deal with missing data in my study? Aust. N. Z. J. Public Health 25(5), 464\u2013469 (2001)","journal-title":"Aust. N. Z. J. Public Health"},{"key":"259_CR6","unstructured":"Biddle, D.: Adverse Impact and Test Validation: A Practitioner\u2019s Guide to Valid and Defensible Employment Testing. Gower (2005)"},{"key":"259_CR7","unstructured":"Binns, R.: Fairness in machine learning: Lessons from political philosophy. In: Conference on Fairness, Accountability and Transparency, New York, NY, USA. pp. 149\u2013159 (2018)"},{"key":"259_CR8","unstructured":"Bolukbasi, T., Chang, K.W., Zou, J., Saligrama, V., Kalai, A.: Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In: Proceedings of International Conference on Neural Information Processing Systems. pp. 4356\u20134364. NIPS\u201916, Curran Associates Inc., USA (2016)"},{"key":"259_CR9","unstructured":"Bronner, L.:Why statistics don\u2019t capture the full extent of the systemic bias in policing (2020). FiveThirtyEight, June 25, 2020. https:\/\/fivethirtyeight.com\/features\/whystatistics-dontcapture-the-full-extent-of-the-systemic-bias-in-policing"},{"key":"259_CR10","unstructured":"Brown,T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dharwal, P., Neelakantan, A.,Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G.,Henighan, T., Child, R., Ramesh, A., Ziegler, D.M., Wu, J., Winter, C., Hesse, C.,Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C.,McCandlish, S., Radford, A., Sutskever, I., Amodei, D.: Language models are fewshotlearners (2020). arXiv:2005.14165"},{"key":"259_CR11","unstructured":"Buolamwini, J., Gebru, T.: Gender shades: Intersectional accuracy disparities in commercial gender classification. In: Friedler, S.A., Wilson, C. (eds.) Proceedings of the 1st Conference on Fairness, Accountability and Transparency. Proceedings of Machine Learning Research, vol.\u00a081, pp. 77\u201391. PMLR, New York, NY, USA (23\u201324 Feb 2018)"},{"key":"259_CR12","doi-asserted-by":"crossref","unstructured":"Calders, T., Kamiran, F., Pechenizkiy, M.: Building classifiers with independency constraints. In: IEEE International Conference on Data Mining Workshops, Miami, FL, USA. pp. 13\u201318 (2009)","DOI":"10.1109\/ICDMW.2009.83"},{"key":"259_CR13","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s10618-010-0190-x","volume":"21","author":"T Calders","year":"2010","unstructured":"Calders, T., Verwer, S.: Three naive bayes approaches for discrimination-free classification. Data Min. Knowl. Discov. 21, 277\u2013292 (2010)","journal-title":"Data Min. Knowl. Discov."},{"key":"259_CR14","unstructured":"Calmon, F., Wei, D., Vinzamuri, B., Natesan\u00a0Ramamurthy, K., Varshney, K.R.: Optimized pre-processing for discrimination prevention. In: Advances in Neural Information Processing Systems 30, pp. 3992\u20134001. Curran Associates, Inc. (2017)"},{"issue":"2","key":"259_CR15","first-page":"9","volume":"16","author":"J Carpenter","year":"2004","unstructured":"Carpenter, J., Goldstein, H.: Multiple imputation in mlwin. Multilevel Modell. Newslet. 16(2), 9\u201318 (2004)","journal-title":"Multilevel Modell. Newslet."},{"key":"259_CR16","unstructured":"Center for Democracy & Technology: Ai & machine learning. https:\/\/cdt.org\/area-of-focus\/privacy-data\/ai-machine-learning"},{"issue":"2","key":"259_CR17","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1089\/big.2016.0047","volume":"5","author":"A Chouldechova","year":"2017","unstructured":"Chouldechova, A.: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data 5(2), 153\u2013163 (2017)","journal-title":"Big Data"},{"key":"259_CR18","unstructured":"Chouldechova, A., Roth, A.: The frontiers of fairness in machine learning (2018). arXiv:1810.08810"},{"key":"259_CR19","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1186\/2193-1801-2-222","volume":"2","author":"Y Dong","year":"2013","unstructured":"Dong, Y., Peng, J.: Principled missing data methods for researchers. SpringerPlus 2, 222 (2013)","journal-title":"SpringerPlus"},{"key":"259_CR20","unstructured":"Dua, D., Graff, C.: UCI machine learning repository (2017). http:\/\/archive.ics.uci.edu\/ml"},{"key":"259_CR21","doi-asserted-by":"crossref","unstructured":"Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of Innovations in Theoretical Computer Science Conference, pp. 214\u2013226. Association for Computing Machinery, New York, NY, USA (2012)","DOI":"10.1145\/2090236.2090255"},{"issue":"12","key":"259_CR22","doi-asserted-by":"publisher","first-page":"3692","DOI":"10.1016\/j.patcog.2008.05.019","volume":"41","author":"A Farhangfar","year":"2008","unstructured":"Farhangfar, A., Kurgan, L., Dy, J.: Impact of imputation of missing values on classification error for discrete data. Pattern Recognit. 41(12), 3692\u20133705 (2008)","journal-title":"Pattern Recognit."},{"key":"259_CR23","doi-asserted-by":"crossref","unstructured":"Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, NSW, Australia, pp. 259\u2013268 (2015)","DOI":"10.1145\/2783258.2783311"},{"issue":"3","key":"259_CR24","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1145\/230538.230561","volume":"14","author":"B Friedman","year":"1996","unstructured":"Friedman, B., Nissenbaum, H.: Bias in computer systems. ACM Trans. Inf. Syst. 14(3), 330\u2013347 (1996)","journal-title":"ACM Trans. Inf. Syst."},{"key":"259_CR25","doi-asserted-by":"crossref","unstructured":"Goel, N., Yaghini, M., Faltings, B.: Non-discriminatory machine learning through convex fairness criteria. In: Conference on Artificial Intelligence, New Orleans, LA, USA (2018)","DOI":"10.1145\/3278721.3278722"},{"key":"259_CR26","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1146\/annurev.psych.58.110405.085530","volume":"60","author":"JW Graham","year":"2009","unstructured":"Graham, J.W.: Missing data analysis: making it work in the real world. Annu. Rev. Psychol. 60, 549\u2013576 (2009)","journal-title":"Annu. Rev. Psychol."},{"key":"259_CR27","doi-asserted-by":"crossref","unstructured":"Green, B., Chen, Y.: Disparate interactions: An algorithm-in-the-loop analysis of fairness in risk assessments. In: Conference on Fairness, Accountability, and Transparency. Atlanta, GA, USA (2019)","DOI":"10.1145\/3287560.3287563"},{"key":"259_CR28","unstructured":"Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. In: Proceedings of International Conference on Neural Information Processing Systems, Red Hook, NY, USA, pp. 3323\u20133331 (2016)"},{"issue":"1","key":"259_CR29","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1198\/000313007X172556","volume":"61","author":"NJ Horton","year":"2007","unstructured":"Horton, N.J., Kleinman, K.P.: Much ado about nothing. Am. Stat. 61(1), 79\u201390 (2007)","journal-title":"Am. Stat."},{"issue":"1","key":"259_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12874-017-0442-1","volume":"17","author":"JC Jakobsen","year":"2017","unstructured":"Jakobsen, J.C., Gluud, C., Wetterslev, J., Winkel, P.: When and how should multiple imputation be used for handling missing data in randomised clinical trials-a practical guide with flowcharts. BMC Med. Res. Methodol. 17(1), 1\u201310 (2017)","journal-title":"BMC Med. Res. Methodol."},{"key":"259_CR31","unstructured":"Jeff, L., Surya Mattu, L.K., Angwin, J.: How we analyzed the compas recidivism algorithm (2016). https:\/\/www.propublica.org\/article\/how-we-analyzed-the-compas-recidivismalgorithm"},{"issue":"1","key":"259_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10115-011-0463-8","volume":"33","author":"F Kamiran","year":"2012","unstructured":"Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33(1), 1\u201333 (2012)","journal-title":"Knowl. Inf. Syst."},{"key":"259_CR33","doi-asserted-by":"crossref","unstructured":"Kamiran, F., Calders, T., Pechenizkiy, M.: Discrimination aware decision tree learning. In: Proceedings of IEEE International Conference on Data Mining, Washington, DC, USA, pp. 869\u2013874 (2010)","DOI":"10.1109\/ICDM.2010.50"},{"key":"259_CR34","doi-asserted-by":"crossref","unstructured":"Kamishima, T., Akaho, S., Sakuma, J.: Fairness-aware learning through regularization approach. In: IEEE International Conference on Data Mining Workshops, Vancouver, BC, Canada, pp. 643\u2013650 (2011)","DOI":"10.1109\/ICDMW.2011.83"},{"key":"259_CR35","doi-asserted-by":"crossref","unstructured":"Kamishima, T., Akaho, S., Asoh, H., Sakuma, J.: Fairness-aware classifier with prejudice remover regularizer. In: Machine Learning and Knowledge Discovery in Databases, Berlin, Heidelberg, pp. 35\u201350 (2012)","DOI":"10.1007\/978-3-642-33486-3_3"},{"issue":"5","key":"259_CR36","doi-asserted-by":"publisher","first-page":"402","DOI":"10.4097\/kjae.2013.64.5.402","volume":"64","author":"H Kang","year":"2013","unstructured":"Kang, H.: The prevention and handling of the missing data. Korean J. Anesthesiol. 64(5), 402\u2013406 (2013)","journal-title":"Korean J. Anesthesiol."},{"key":"259_CR37","unstructured":"Kehl, D.L., Kessler, S.A.: Algorithms in the criminal justice system: Assessing the Use of Risk Assessments in Sentencing. Berkman Klein Center for Internet & Society (2017)"},{"key":"259_CR38","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1017\/S0003055401000235","volume":"95","author":"G King","year":"2001","unstructured":"King, G., Honaker, J., Joseph, A., Scheve, K.: Analyzing incomplete political science data: an alternative algorithm for multiple imputation. Am. Polit. Sci. Rev. 95, 49\u201369 (2001)","journal-title":"Am. Polit. Sci. Rev."},{"key":"259_CR39","doi-asserted-by":"publisher","DOI":"10.1201\/9781315108230","volume-title":"Feature Engineering and Selection: A Practical Approach for Predictive Models","author":"M Kuhn","year":"2019","unstructured":"Kuhn, M., Johnson, K.: Feature Engineering and Selection: A Practical Approach for Predictive Models. CRC Press, Chapman & Hall\/CRC Data Science Series (2019)"},{"key":"259_CR40","doi-asserted-by":"crossref","unstructured":"Little, R., Rubin, D.: Statistical Analysis with Missing Data. Wiley Series in Probability and Statistics, Wiley (2002)","DOI":"10.1002\/9781119013563"},{"key":"259_CR41","doi-asserted-by":"crossref","unstructured":"Little, R.J.A., Schenker, N.: Missing Data, pp. 39\u201375. Springer, Boston (1995)","DOI":"10.1007\/978-1-4899-1292-3_2"},{"key":"259_CR42","unstructured":"Mart\u0131inez-Plumed, F., Ferri, C., Nieves, D., Hern\u2019andez-Orallo, J.: Fairness and missing values (2019). arXiv:1905.12728"},{"key":"259_CR43","doi-asserted-by":"publisher","first-page":"13","DOI":"10.3389\/fdata.2019.00013","volume":"2","author":"A Olteanu","year":"2019","unstructured":"Olteanu, A., Castillo, C., Diaz, F., K\u0131c\u0131man, E.: Social data: biases, methodological pitfalls, and ethical boundaries. Front. Big Data 2, 13 (2019)","journal-title":"Front. Big Data"},{"key":"259_CR44","doi-asserted-by":"crossref","unstructured":"Pickles, A.: Missing data, problems and solutions. In: Kempf-Leonard, K. (ed.) Encyclopedia of Social Measurement, pp. 689\u2013694. Elsevier, New York (2005)","DOI":"10.1016\/B0-12-369398-5\/00125-0"},{"key":"259_CR45","unstructured":"Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., Weinberger, K.Q.: On fairness and calibration. In: Proceedings of International Conference on Neural Information Processing Systems Long Beach, CA, USA, pp. 5684\u20135693 (2017)"},{"key":"259_CR46","volume-title":"Multiple Imputation for Nonresponse in Surveys","author":"D Rubin","year":"2004","unstructured":"Rubin, D.: Multiple Imputation for Nonresponse in Surveys. Wiley, Wiley Classics Library (2004)"},{"issue":"3","key":"259_CR47","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1093\/biomet\/63.3.581","volume":"63","author":"DB Rubin","year":"1976","unstructured":"Rubin, D.B.: Inference and missing data. Biometrika 63(3), 581\u2013592 (1976)","journal-title":"Biometrika"},{"key":"259_CR48","doi-asserted-by":"crossref","unstructured":"Schafer, J.L., Graham, J.W.: Missing data: our view of the state of the art. Psychol. Methods 7(2), 147\u201377 (2002)","DOI":"10.1037\/1082-989X.7.2.147"},{"key":"259_CR49","unstructured":"The White House: Artificial intelligence for the american people. https:\/\/trumpwhitehouse.archives.gov\/ai"},{"key":"259_CR50","doi-asserted-by":"crossref","unstructured":"Van Buuren, S.: Flexible Imputation of Missing Data. CRC Press (2018)","DOI":"10.1201\/9780429492259"},{"key":"259_CR51","doi-asserted-by":"crossref","unstructured":"Verma, S., Rubin, J.: Fairness definitions explained. In: Proceedings of the International Workshop on Software Fairness, pp. 1\u20137. FairWare \u201918, Association for Computing Machinery, New York, NY, USA (2018)","DOI":"10.1145\/3194770.3194776"},{"key":"259_CR52","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1089\/big.2014.0063","volume":"2","author":"T Wang","year":"2014","unstructured":"Wang, T., Wang, D.: Why amazon\u2019s ratings might mislead you: the story of herding effects. Big Data 2, 196\u2013204 (2014)","journal-title":"Big Data"},{"key":"259_CR53","unstructured":"Zafar, M.B., Valera, I., Gomez-Rodriguez, M., Gummadi, K.P.: Fairness constraints: mechanisms for fair classification. In: Proceedings of International Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, FL, USA. pp. 962\u2013970 (2017)"},{"key":"259_CR54","first-page":"325","volume":"28","author":"R Zemel","year":"2013","unstructured":"Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. Proceedings of International Conference on Machine Learning, Atlanta, GA, USA 28, 325\u2013333 (2013)","journal-title":"Proceedings of International Conference on Machine Learning, Atlanta, GA, USA"}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-021-00259-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-021-00259-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-021-00259-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,4]],"date-time":"2023-11-04T10:37:54Z","timestamp":1699094274000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-021-00259-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,31]]},"references-count":54,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,8]]}},"alternative-id":["259"],"URL":"https:\/\/doi.org\/10.1007\/s41060-021-00259-z","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"value":"2364-415X","type":"print"},{"value":"2364-4168","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,31]]},"assertion":[{"value":"4 July 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 May 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest\/Competing interests"}},{"value":"The data and the code are available at","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Availability of data and material"}},{"order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Code availability"}}]}}