{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T08:18:58Z","timestamp":1778228338360,"version":"3.51.4"},"reference-count":35,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,3,12]],"date-time":"2022-03-12T00:00:00Z","timestamp":1647043200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Machine learning (ML) models are increasingly being used for high-stake applications that can greatly impact people\u2019s lives. Sometimes, these models can be biased toward certain social groups on the basis of race, gender, or ethnicity. Many prior works have attempted to mitigate this \u201cmodel discrimination\u201d by updating the training data (pre-processing), altering the model learning process (in-processing), or manipulating the model output (post-processing). However, more work can be done in extending this situation to intersectional fairness, where we consider multiple sensitive parameters (e.g., race) and sensitive options (e.g., black or white), thus allowing for greater real-world usability. Prior work in fairness has also suffered from an accuracy\u2013fairness trade-off that prevents both accuracy and fairness from being high. Moreover, the previous literature has not clearly presented holistic fairness metrics that work with intersectional fairness. In this paper, we address all three of these problems by (a) creating a bias mitigation technique called DualFair and (b) developing a new fairness metric (i.e., AWI, a measure of bias of an algorithm based upon inconsistent counterfactual predictions) that can handle intersectional fairness. Lastly, we test our novel mitigation method using a comprehensive U.S. mortgage lending dataset and show that our classifier, or fair loan predictor, obtains relatively high fairness and accuracy metrics.<\/jats:p>","DOI":"10.3390\/make4010011","type":"journal-article","created":{"date-parts":[[2022,3,13]],"date-time":"2022-03-13T22:29:43Z","timestamp":1647210583000},"page":"240-253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Developing a Novel Fair-Loan Classifier through a Multi-Sensitive Debiasing Pipeline: DualFair"],"prefix":"10.3390","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6109-451X","authenticated-orcid":false,"given":"Arashdeep","family":"Singh","sequence":"first","affiliation":[{"name":"Floyd B. Buchanan High School, 1560 N Minnewawa Ave, Clovis, CA 93619, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8186-7383","authenticated-orcid":false,"given":"Jashandeep","family":"Singh","sequence":"additional","affiliation":[{"name":"Floyd B. Buchanan High School, 1560 N Minnewawa Ave, Clovis, CA 93619, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7613-5636","authenticated-orcid":false,"given":"Ariba","family":"Khan","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA 02139, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9306-1256","authenticated-orcid":false,"given":"Amar","family":"Gupta","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St, Cambridge, MA 02139, USA"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2022, February 06). Machine Bias\u2014ProPublica. Available online: https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing.","DOI":"10.1201\/9781003278290-37"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Dastin, J. (2022, February 06). Amazon Scraps Secret AI Recruiting Tool That Showed Bias against Women. Available online: https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G.","DOI":"10.1201\/9781003278290-44"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"608","DOI":"10.1038\/d41586-019-03228-6","article-title":"Millions of Black People Affected by Racial Bias in Health-Care Algorithms","volume":"574","author":"Ledford","year":"2019","journal-title":"Nature"},{"key":"ref_4","unstructured":"Weber, M., Yurochkin, M., Sherif, B., and Vanio, M. (2022, February 06). Black Loans Matter: Fighting Bias for AI Fairness in Lending. Available online: https:\/\/mitibmwatsonailab.mit.edu\/research\/blog\/black-loans-matter-fighting-bias-for-ai-fairness-in-lending\/."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.jfineco.2021.05.047","article-title":"Consumer-Lending Discrimination in the FinTech Era","volume":"143","author":"Bartlett","year":"2022","journal-title":"J. Financ. Econ."},{"key":"ref_6","unstructured":"Kusner, M.J., Loftus, J.R., Russell, C., and Silva, R. (2018). Counterfactual Fairness. arXiv."},{"key":"ref_7","unstructured":"Chiappa, S., and Gillam, T.P.S. (2018). Path-Specific Counterfactual Fairness. arXiv."},{"key":"ref_8","unstructured":"Brunet, M.E., Alkalay-Houlihan, C., Anderson, A., and Zemel, R. (2019, January 9\u201315). Understanding the Origins of Bias in Word Embeddings. Proceedings of the 36th International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_9","unstructured":"Calmon, F., Wei, D., Vinzamuri, B., Natesan Ramamurthy, K., and Varshney, K.R. (2017). Optimized Pre-Processing for Discrimination Prevention. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, B.H., Lemoine, B., and Mitchell, M. (2018, January 2\u20133). Mitigating Unwanted Biases with Adversarial Learning. Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA.","DOI":"10.1145\/3278721.3278779"},{"key":"ref_11","unstructured":"Wu, Y., Zhang, L., and Wu, X. (2018). Fairness-Aware Classification: Criterion, Convexity, and Bounds. arXiv."},{"key":"ref_12","unstructured":"Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., and Roth, A. (2017). A Convex Framework for Fair Regression. arXiv."},{"key":"ref_13","unstructured":"Hardt, M., Price, E., and Srebro, N. (2016). Equality of Opportunity in Supervised Learning. arXiv."},{"key":"ref_14","unstructured":"Bellamy, R.K.E., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., and Mojsilovic, A. (2018). AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias. arXiv."},{"key":"ref_15","unstructured":"(2022, February 06). Microsoft Trust in FATE to Keep AI Safe. Available online: https:\/\/gulfnews.com\/technology\/microsoft-trust-in-fate-to-keep-ai-safe-1.2289745."},{"key":"ref_16","unstructured":"(2022, February 06). Responsible AI: Putting Our Principles into Action. Available online: https:\/\/blog.google\/technology\/ai\/responsible-ai-principles\/."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Madaio, M.A., Stark, L., Wortman Vaughan, J., and Wallach, H. (May, January 29). Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI \u201920, New York, NY, USA.","DOI":"10.1145\/3313831.3376445"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/978-3-030-12524-0_2","article-title":"The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems","volume":"Volume 95","author":"Tokhi","year":"2019","journal-title":"Robotics and Well-Being"},{"key":"ref_19","unstructured":"Weiser, S. (2022, February 06). Building Trust in Human-Centric AI. Available online: https:\/\/ec.europa.eu\/futurium\/en\/ai-alliance-consultation\/guidelines."},{"key":"ref_20","unstructured":"Gershgorn, D. (2022, February 06). Facebook Says It Has a Tool to Detect Bias in Its Artificial Intelligence. Available online: https:\/\/qz.com\/1268520\/facebook-says-it-has-a-tool-to-detect-bias-in-its-artificial-intelligence\/."},{"key":"ref_21","unstructured":"(2022, February 06). EXPLAIN 2019\u2014ASE 2019. Available online: https:\/\/2019.ase-conferences.org\/home\/explain-2019."},{"key":"ref_22","first-page":"115:1","article-title":"A Survey on Bias and Fairness in Machine Learning","volume":"54","author":"Mehrabi","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_23","unstructured":"(2022, February 06). ACM FAccT. Available online: https:\/\/facctconference.org\/."},{"key":"ref_24","unstructured":"Kenna, D. (2021). Using Adversarial Debiasing to Remove Bias from Word Embeddings. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.ins.2017.09.064","article-title":"Exploiting Reject Option in Classification for Social Discrimination Control","volume":"425","author":"Kamiran","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chakraborty, J., Majumder, S., and Menzies, T. (2021, January 23\u201328). Bias in Machine Learning Software: Why? How? What to Do?. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece.","DOI":"10.1145\/3468264.3468537"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chakraborty, J., Majumder, S., Yu, Z., and Menzies, T. (2020, January 27\u201329). Fairway: A Way to Build Fair ML Software. Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC\/FSE 2020, Jabalpur, India.","DOI":"10.1145\/3368089.3409697"},{"key":"ref_28","unstructured":"Saleiro, P., Kuester, B., Hinkson, L., London, J., Stevens, A., Anisfeld, A., Rodolfa, K.T., and Ghani, R. (2019). Aequitas: A Bias and Fairness Audit Toolkit. arXiv."},{"key":"ref_29","unstructured":"Ghosh, A., Genuit, L., and Reagan, M. (2021). Characterizing Intersectional Group Fairness with Worst-Case Comparisons. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Fuster, A., Goldsmith-Pinkham, P., Ramadorai, T., and Walther, A. (2021). Predictably Unequal? The Effects of Machine Learning on Credit Markets, Social Science Research Network. SSRN Scholarly Paper ID 3072038.","DOI":"10.1111\/jofi.13090"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gill, N., Hall, P., Montgomery, K., and Schmidt, N. (2020). A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing. Information, 11.","DOI":"10.3390\/info11030137"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/s11023-020-09529-4","article-title":"Algorithmic Fairness in Mortgage Lending: From Absolute Conditions to Relational Trade-offs","volume":"31","author":"Lee","year":"2021","journal-title":"Minds Mach."},{"key":"ref_33","unstructured":"(2022, February 06). HMDA Data Browser, Available online: https:\/\/ffiec.cfpb.gov\/data-browser\/data\/2020?category=states."},{"key":"ref_34","unstructured":"Wick, M., Panda, S., and Tristan, J.B. (2019). Unlocking Fairness: A Trade-off Revisited. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_35","unstructured":"Zhang, Y., Zhou, F., Li, Z., Wang, Y., and Chen, F. (2021). Bias-Tolerant Fair Classification. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/4\/1\/11\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:35:34Z","timestamp":1760135734000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/4\/1\/11"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,12]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["make4010011"],"URL":"https:\/\/doi.org\/10.3390\/make4010011","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,12]]}}}