{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T19:24:14Z","timestamp":1770146654220,"version":"3.49.0"},"reference-count":39,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T00:00:00Z","timestamp":1637798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The deployment of machine learning (ML) systems in applications with societal impact has motivated the study of fairness for marginalized groups. Often, the protected attribute is absent from the training dataset for legal reasons. However, datasets still contain proxy attributes that capture protected information and can inject unfairness in the ML model. Some deployed systems allow auditors, decision makers, or affected users to report issues or seek recourse by flagging individual samples. In this work, we examine such systems and consider a feedback-based framework where the protected attribute is unavailable and the flagged samples are indirect knowledge. The reported samples are used as guidance to identify the proxy attributes that are causally dependent on the (unknown) protected attribute. We work under the causal interventional fairness paradigm. Without requiring the underlying structural causal model a priori, we propose an approach that performs conditional independence tests on observed data to identify such proxy attributes. We theoretically prove the optimality of our algorithm, bound its complexity, and complement it with an empirical evaluation demonstrating its efficacy on various real-world and synthetic datasets.<\/jats:p>","DOI":"10.3390\/e23121571","type":"journal-article","created":{"date-parts":[[2021,11,29]],"date-time":"2021-11-29T05:23:02Z","timestamp":1638163382000},"page":"1571","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Interventional Fairness with Indirect Knowledge of Unobserved Protected Attributes"],"prefix":"10.3390","volume":"23","author":[{"given":"Sainyam","family":"Galhotra","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Chicago, Chicago, IL 60637, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karthikeyan","family":"Shanmugam","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4435-0486","authenticated-orcid":false,"given":"Prasanna","family":"Sattigeri","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7376-5536","authenticated-orcid":false,"given":"Kush R.","family":"Varshney","sequence":"additional","affiliation":[{"name":"IBM Research, Yorktown Heights, NY 10598, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,25]]},"reference":[{"key":"ref_1","unstructured":"Chiappa, S. (February, January 27). Path-specific counterfactual fairness. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, AI, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nabi, R., and Shpitser, I. (2018, January 2\u20137). Fair inference on outcomes. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11553"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Salimi, B., Rodriguez, L., Howe, B., and Suciu, D. (July, January 30). Interventional fairness: Causal database repair for algorithmic fairness. Proceedings of the 2019 International Conference on Management of Data, Amsterdam, The Netherlands.","DOI":"10.1145\/3299869.3319901"},{"key":"ref_4","unstructured":"Galhotra, S., Shanmugam, K., Sattigeri, P., and Varshney, K.R. (2020). Fair Data Integration. arXiv."},{"key":"ref_5","unstructured":"Ploeg, M.V., and Perrin, E. (2004). Eliminating Health Disparities: Measurement and Data Needs, National Academies Press."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, J., Kallus, N., Mao, X., Svacha, G., and Udell, M. (2019, January 29\u201331). Fairness under unawareness: Assessing disparity when protected class is unobserved. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.","DOI":"10.1145\/3287560.3287594"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Coston, A., Ramamurthy, K.N., Wei, D., Varshney, K.R., Speakman, S., Mustahsan, Z., and Chakraborty, S. (2019, January 27\u201328). Fair Transfer Learning with Missing Protected Attributes. Proceedings of the AAAI\/ACM Conference on Artificial Intelligence, Ethics, and Society, Honolulu, HI, USA.","DOI":"10.1145\/3306618.3314236"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ustun, B., Spangher, A., and Liu, Y. (2019, January 29\u201331). Actionable recourse in linear classification. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.","DOI":"10.1145\/3287560.3287566"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1109\/MSPEC.2020.9173891","article-title":"Can AI hiring systems be made antiracist?","volume":"57","author":"Hsu","year":"2020","journal-title":"IEEE Spectr."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Xu, D., Wu, Y., Yuan, S., Zhang, L., and Wu, X. (2019, January 10\u201316). Achieving causal fairness through generative adversarial networks. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Macao, China.","DOI":"10.24963\/ijcai.2019\/201"},{"key":"ref_11","unstructured":"Kusner, M., Loftus, J., Russell, C., and Silva, R. (2017, January 4\u20139). Counterfactual Fairness. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_12","unstructured":"Jiang, R., Pacchiano, A., Stepleton, T., Jiang, H., and Chiappa, S. (2019). Wasserstein fair classification. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chiappa, S., and Isaac, W.S. (2018). A causal Bayesian networks viewpoint on fairness. IFIP International Summer School on Privacy and Identity Management, Springer.","DOI":"10.1007\/978-3-030-16744-8_1"},{"key":"ref_14","unstructured":"Kilbertus, N., Rojas Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., and Sch\u00f6lkopf, B. (2017, January 4\u20139). Avoiding Discrimination through Causal Reasoning. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, J., and Bareinboim, E. (2018, January 2\u20137). Fairness in Decision-Making\u2014The Causal Explanation Formula. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11564"},{"key":"ref_16","unstructured":"Zhang, J., and Bareinboim, E. (2018, January 2\u20138). Equality of Opportunity in Classification: A Causal Approach. Proceedings of the 32nd Conference on Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Khademi, A., and Honavar, V. (2019). Algorithmic Bias in Recidivism Prediction: A Causal Perspective. arXiv.","DOI":"10.1609\/aaai.v34i10.7192"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Khademi, A., Lee, S., Foley, D., and Honavar, V. (2019, January 13\u201317). Fairness in algorithmic decision making: An excursion through the lens of causality. Proceedings of the World Wide Web Conference, San Francisco, CA, USA.","DOI":"10.1145\/3308558.3313559"},{"key":"ref_19","unstructured":"Russell, C., Kusner, M.J., Loftus, J., and Silva, R. (2017, January 4\u20139). When worlds collide: Integrating different counterfactual assumptions in fairness. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1111\/j.1467-8640.1993.tb00310.x","article-title":"Hybrid algorithms for the constraint satisfaction problem","volume":"9","author":"Prosser","year":"1993","journal-title":"Comput. Intell."},{"key":"ref_21","unstructured":"Tandon, R., Shanmugam, K., Ravikumar, P.K., and Dimakis, A.G. (2014, January 8\u201313). On the information theoretic limits of learning Ising models. Proceedings of the 32nd Conference on Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1093\/biostatistics\/kxm045","article-title":"Sparse inverse covariance estimation with the graphical lasso","volume":"9","author":"Friedman","year":"2008","journal-title":"Biostatistics"},{"key":"ref_23","unstructured":"(2021, May 01). Medical Expenditure Panel Survey, Available online: https:\/\/meps.ahrq.gov\/mepsweb\/."},{"key":"ref_24","unstructured":"(2021, May 01). German Data: Uci Machine Learning Repository. Available online: https:\/\/archive.ics.uci.edu\/ml\/datasets\/Statlog+%28German+Credit+Data%29."},{"key":"ref_25","unstructured":"(2021, May 01). Adult Data: Uci Machine Learning Repository. Available online: https:\/\/archive.ics.uci.edu\/ml\/datasets\/adult."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"20180017","DOI":"10.1515\/jci-2018-0017","article-title":"Approximate kernel-based conditional independence tests for fast non-parametric causal discovery","volume":"7","author":"Strobl","year":"2019","journal-title":"J. Causal Inference"},{"key":"ref_27","unstructured":"Friedman, J., Hastie, T., Tibshirani, R., and Tibshirani, M.R. (2021, May 01). Package \u2018glasso\u2019. Available online: https:\/\/cran.r-project.org\/web\/packages\/glasso\/index.html."},{"key":"ref_28","unstructured":"Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., Wang, X., and Chi, E.H. (2020). Fairness without demographics through adversarially reweighted learning. arXiv."},{"key":"ref_29","unstructured":"Hashimoto, T., Srivastava, M., Namkoong, H., and Liang, P. (2018, January 10\u201315). Fairness without demographics in repeated loss minimization. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yan, S., Kao, H.t., and Ferrara, E. (2020, January 19\u201323). Fair Class Balancing: Enhancing Model Fairness without Observing Sensitive Attributes. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual.","DOI":"10.1145\/3340531.3411980"},{"key":"ref_31","unstructured":"Celis, L.E., Huang, L., and Vishnoi, N.K. (2020). Fair Classification with Noisy Protected Attributes. arXiv."},{"key":"ref_32","unstructured":"Blum, A., and Stangl, K. (2019). Recovering from biased data: Can fairness constraints improve accuracy?. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kamishima, T., Akaho, S., Asoh, H., and Sakuma, J. (2012, January 23\u201327). Fairness-aware classifier with prejudice remover regularizer. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Bristol, UK.","DOI":"10.1007\/978-3-642-33486-3_3"},{"key":"ref_34","unstructured":"Zafar, M.B., Valera, I., Rogriguez, M.G., and Gummadi, K.P. (2017, January 20\u201322). Fairness constraints: Mechanisms for fair classification. Proceedings of the Artificial Intelligence and Statistics, Lauderdale, FL, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s10618-010-0190-x","article-title":"Three naive bayes approaches for discrimination-free classification","volume":"21","author":"Calders","year":"2010","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Celis, L.E., Huang, L., Keswani, V., and Vishnoi, N.K. (2019, January 29\u201331). Classification with fairness constraints: A meta-algorithm with provable guarantees. Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, GA, USA.","DOI":"10.1145\/3287560.3287586"},{"key":"ref_37","first-page":"3315","article-title":"Equality of opportunity in supervised learning","volume":"29","author":"Hardt","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","unstructured":"Calmon, F.P., Wei, D., Vinzamuri, B., Ramamurthy, K.N., and Varshney, K.R. (2017, January 4\u20139). Optimized pre-processing for discrimination prevention. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_39","unstructured":"Xu, Z., Liu, J., Cheng, D., Li, J., Liu, L., and Wang, K. (2020). Assessing the Fairness of Classifiers with Collider Bias. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1571\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:35:48Z","timestamp":1760168148000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/12\/1571"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,25]]},"references-count":39,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["e23121571"],"URL":"https:\/\/doi.org\/10.3390\/e23121571","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,11,25]]}}}