{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T18:57:58Z","timestamp":1784141878572,"version":"3.55.0"},"reference-count":62,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T00:00:00Z","timestamp":1623456000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T00:00:00Z","timestamp":1623456000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI Ethics"],"published-print":{"date-parts":[[2021,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>There is growing concern that decision-making informed by machine learning (ML) algorithms may unfairly discriminate based on personal demographic attributes, such as race and gender. Scholars have responded by introducing numerous mathematical definitions of fairness to test the algorithm, many of which are in conflict with one another. However, these reductionist representations of fairness often bear little resemblance to real-life fairness considerations, which in practice are highly contextual. Moreover, fairness metrics tend to be implemented within narrow and targeted fairness toolkits for algorithm assessments that are difficult to integrate into an algorithm\u2019s broader ethical assessment. In this paper, we derive lessons from ethical philosophy and welfare economics as they relate to the contextual factors relevant for fairness. In particular we highlight the debate around the acceptability of particular inequalities and the inextricable links between fairness, welfare and autonomy. We propose Key Ethics Indicators (KEIs) as a way towards providing a more holistic understanding of whether or not an algorithm is aligned to the decision-maker\u2019s ethical values.<\/jats:p>","DOI":"10.1007\/s43681-021-00067-y","type":"journal-article","created":{"date-parts":[[2021,6,12]],"date-time":"2021-06-12T12:03:27Z","timestamp":1623499407000},"page":"529-544","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics"],"prefix":"10.1007","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7725-2503","authenticated-orcid":false,"given":"Michelle Seng Ah","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5444-2280","authenticated-orcid":false,"given":"Luciano","family":"Floridi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5102-6564","authenticated-orcid":false,"given":"Jatinder","family":"Singh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,6,12]]},"reference":[{"key":"67_CR1","volume-title":"Law and autonomous systems series: algorithmic credit scoring and the regulation of consumer credit markets","author":"N Aggarwal","year":"2018","unstructured":"Aggarwal, N.: Law and autonomous systems series: algorithmic credit scoring and the regulation of consumer credit markets. University of Oxford Business Law Blog, Oxford (2018)"},{"key":"67_CR2","doi-asserted-by":"crossref","unstructured":"Aggarwal, N.: The norms of algorithmic credit scoring. Available at SSRN 3569083 (2020)","DOI":"10.2139\/ssrn.3569083"},{"key":"67_CR3","doi-asserted-by":"crossref","unstructured":"Aristotle, Sinclair, T. A.: Aristotle: the politics. Penguin Books Limited, Translated with an Introduction by TA Sinclair (1962)","DOI":"10.1093\/oseo\/instance.00259307"},{"key":"67_CR4","unstructured":"Arya, V., Bellamy, R.K.E., Chen, P.-Y., Dhurandhar, A., Hind, M., Hoffman, S.C., Houde, S., Liao, Q.\u00a0V., Luss, R., Mojsilovi\u0107, A, et\u00a0al.: One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv preprint arXiv:1909.03012 (2019)"},{"key":"67_CR5","unstructured":"Bellamy, R.K.E., Dey, K., Hind, M., Hoffman, S.C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A. et\u00a0al.: AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv preprint arXiv:1810.01943 (2018)"},{"key":"67_CR6","doi-asserted-by":"crossref","unstructured":"Binns, R.: On the apparent conflict between individual and group fairness. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp 514\u2013524 (2020)","DOI":"10.1145\/3351095.3372864"},{"issue":"2","key":"67_CR7","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1089\/big.2016.0047","volume":"5","author":"A Chouldechova","year":"2017","unstructured":"Chouldechova, A.: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data 5(2), 153\u2013163 (2017)","journal-title":"Big Data"},{"key":"67_CR8","doi-asserted-by":"crossref","unstructured":"Cobbe, J., Seng, M., Lee, A.H., Singh, J.: Reviewable automated decision making: a framework for accountable algorithmic systems. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp 598\u2013609 (2021)","DOI":"10.1145\/3442188.3445921"},{"issue":"4","key":"67_CR9","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1111\/j.1745-3984.1973.tb00802.x","volume":"10","author":"NS Cole","year":"1973","unstructured":"Cole, N.S.: Bias in selection. J. Educ. Meas. 10(4), 237\u2013255 (1973)","journal-title":"J. Educ. Meas."},{"key":"67_CR10","doi-asserted-by":"publisher","DOI":"10.4324\/9780203900055","volume-title":"Black feminist thought: Knowledge, consciousness, and the politics of empowerment","author":"PH Collins","year":"2002","unstructured":"Collins, P.H.: Black feminist thought: Knowledge, consciousness, and the politics of empowerment. Routledge, London (2002)"},{"key":"67_CR11","unstructured":"Corbett-Davies, S., Goel, S.: The measure and mismeasure of fairness: A critical review of fair machine learning. arXiv preprint arXiv:1808.00023 (2018)"},{"key":"67_CR12","unstructured":"Crenshaw, K.: Demarginalizing the intersection of race and sex: a black feminist critique of antidiscrimination doctrine, feminist theory and antiracist politics. U. Chi. Legal f. 139 (1989)"},{"key":"67_CR13","volume-title":"Perpetual bebt in dilicon Savannah","author":"KP Donovan","year":"2019","unstructured":"Donovan, K.P., Park, E.: Perpetual bebt in dilicon Savannah. Boston Review, Boston (2019)"},{"key":"67_CR14","doi-asserted-by":"crossref","unstructured":"Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd innovations in theoretical computer science conference. ACM, pp 214\u2013226 (2012)","DOI":"10.1145\/2090236.2090255"},{"issue":"3","key":"67_CR15","first-page":"185","volume":"10","author":"R Dworkin","year":"1981","unstructured":"Dworkin, R.: What is equality? Part 1: equality of welfare. Philos. Public Aff. 10(3), 185\u2013246 (1981)","journal-title":"Philos. Public Aff."},{"key":"67_CR16","doi-asserted-by":"crossref","DOI":"10.1515\/9781503619609","volume-title":"Ethics and the business of bioscience","author":"ML Eaton","year":"2004","unstructured":"Eaton, M.L.: Ethics and the business of bioscience. Stanford University Press, Stanford (2004)"},{"issue":"4","key":"67_CR17","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1257\/mic.20130139","volume":"7","author":"R Elul","year":"2015","unstructured":"Elul, R., Gottardi, P.: Bankruptcy: is it enough to forgive or must we also forget? Am. Econ. J. Microecon. 7(4), 294\u2013338 (2015)","journal-title":"Am. Econ. J. Microecon."},{"key":"67_CR18","doi-asserted-by":"crossref","unstructured":"Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, pp 259\u2013268 (2015)","DOI":"10.1145\/2783258.2783311"},{"key":"67_CR19","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:osobl\/9780199215911.001.0001","volume-title":"Fairness, responsibility, and welfare","author":"M Fleurbaey","year":"2008","unstructured":"Fleurbaey, M., et al.: Fairness, responsibility, and welfare. Oxford University Press, Oxford (2008)"},{"key":"67_CR20","doi-asserted-by":"crossref","unstructured":"Floridi, L., Cowls, J.: A unified framework of five principles for AI in society. Issue 1.1, Summer 2019 1, 1 (2019)","DOI":"10.1162\/99608f92.8cd550d1"},{"key":"67_CR21","unstructured":"Friedler, S.A., Scheidegger, C., Venkatasubramanian, S.: On the (im) possibility of fairness. arXiv preprint arXiv:1609.07236 (2016)"},{"key":"67_CR22","unstructured":"Gajane, P., Pechenizkiy, M.: On formalizing fairness in prediction with machine learning. arXiv preprint arXiv:1710.03184 (2017)"},{"key":"67_CR23","volume-title":"Morals by agreement","author":"D Gauthier","year":"1986","unstructured":"Gauthier, D.: Morals by agreement. Oxford University Press, Oxford (1986).. (on Demand)"},{"key":"67_CR24","doi-asserted-by":"crossref","unstructured":"Gillis, T.B.: False dreams of algorithmic fairness: the case of credit pricing. Available at SSRN 3571266 (2020)","DOI":"10.2139\/ssrn.3571266"},{"issue":"2","key":"67_CR25","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1002\/job.1956","volume":"36","author":"B Goldman","year":"2015","unstructured":"Goldman, B., Cropanzano, R.: Justice and fairness are not the same thing. J. Organ. Behav. 36(2), 313\u2013318 (2015)","journal-title":"J. Organ. Behav."},{"issue":"1","key":"67_CR26","doi-asserted-by":"publisher","first-page":"9","DOI":"10.2307\/257990","volume":"12","author":"J Greenberg","year":"1987","unstructured":"Greenberg, J.: A taxonomy of organizational justice theories. Acad. Manag. Rev. 12(1), 9\u201322 (1987)","journal-title":"Acad. Manag. Rev."},{"key":"67_CR27","unstructured":"Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. CoRR abs\/1610.02413 (2016)"},{"key":"67_CR28","doi-asserted-by":"crossref","unstructured":"Heidari, H., Loi, M., Gummadi, K. P., Krause, A.: A moral framework for understanding of fair ml through economic models of equality of opportunity. arXiv preprint arXiv:1809.03400 (2018)","DOI":"10.1145\/3287560.3287584"},{"key":"67_CR29","doi-asserted-by":"crossref","unstructured":"Holstein, K., Vaughan, J.K., Daum\u00e9\u00a0III , H., Dudik, M., Wallach, H.: Improving fairness in machine learning systems: What do industry practitioners need?. In: Proceedings of the 2019 CHI conference on human factors in computing systems, pp 1\u201316 (2019)","DOI":"10.1145\/3290605.3300830"},{"key":"67_CR30","doi-asserted-by":"crossref","unstructured":"Hutchinson, B., Mitchell, M.: 50 years of test (un) fairness: lessons for machine learning. In: Proceedings of the Conference on Fairness, Accountability, and Transparency (2019)","DOI":"10.1145\/3287560.3287600"},{"key":"67_CR31","unstructured":"Kagan, S.: 30. Equality and Desert. What Do We Deserve? A reader on justice and desert 298 (1999)"},{"key":"67_CR32","volume-title":"The geometry of desert","author":"S Kagan","year":"2014","unstructured":"Kagan, S.: The geometry of desert. Oxford University Press, Oxford (2014)"},{"key":"67_CR33","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1007\/978-3-642-30487-3_8","volume-title":"Discrimination and Privacy in the Information Society","author":"F Kamiran","year":"2013","unstructured":"Kamiran, F., \u017dliobait\u0117, I.: Explainable and non-explainable discrimination in classification. In: Discrimination and Privacy in the Information Society, pp. 155\u2013170. Springer, Berlin (2013)"},{"key":"67_CR34","doi-asserted-by":"crossref","unstructured":"Kant, I., Gregor, M.: The metaphysics of morals (1996)","DOI":"10.1017\/CBO9780511809644"},{"key":"67_CR35","unstructured":"Kilbertus, N., Ball, P.J., Kusner, M.J., Weller, A., Silva, R.: The sensitivity of counterfactual fairness to unmeasured confounding. arXiv preprint arXiv:1907.01040 (2019)"},{"key":"67_CR36","unstructured":"Kim, M., Reingold, O., Rothblum, G.: Fairness through computationally-bounded awareness. Adv. Neural Inform. Process. Syst. 4842\u20134852 (2018)"},{"key":"67_CR37","unstructured":"Kleinberg, J., Mullainathan, S., Raghavan, M.: Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807 (2016)"},{"key":"67_CR38","unstructured":"Koren, J.R.: What does that Web search say about your credit? (2016). https:\/\/www.latimes.com\/business\/la-fi-zestfinance-baidu-20160715-snap-story.html"},{"key":"67_CR39","unstructured":"Kusner, M.J., Loftus, J.R., Russell, C., Silva, R.: Counterfactual Fairness. arXiv e-prints, Article arXiv:1703.06856 (2017), arXiv:1703.06856"},{"key":"67_CR40","first-page":"2","volume":"34","author":"M Lee","year":"2020","unstructured":"Lee, M., Floridi, L., Denev, A.: Innovating with confidence: embedding AI governance and fairness in a financial services risk management framework. Berkeley Technol. Law J. 34, 2 (2020)","journal-title":"Berkeley Technol. Law J."},{"key":"67_CR41","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09529-4","author":"MSA Lee","year":"2020","unstructured":"Lee, M.S.A., Floridi, L.: Algorithmic fairness in mortgage lending: from absolute conditions to relational trade-offs. Minds Mach. (2020). https:\/\/doi.org\/10.1007\/s11023-020-09529-4","journal-title":"Minds Mach."},{"key":"67_CR42","doi-asserted-by":"crossref","unstructured":"Lee, M.S.A., Singh, J.: The landscape and gaps in open source fairness toolkits. Available at SSRN (2020)","DOI":"10.2139\/ssrn.3695002"},{"key":"67_CR43","doi-asserted-by":"crossref","unstructured":"Lee, M.S.A., Singh, J.: Risk identification questionnaire for unintended bias in machine learning development lifecycle. Available at SSRN (2021)","DOI":"10.2139\/ssrn.3777093"},{"issue":"17","key":"67_CR44","doi-asserted-by":"publisher","first-page":"556","DOI":"10.2307\/2025310","volume":"70","author":"D Lewis","year":"1973","unstructured":"Lewis, D.: Causation. J. Philos. 70(17), 556\u2013567 (1973)","journal-title":"J. Philos."},{"key":"67_CR45","doi-asserted-by":"crossref","unstructured":"Liu, L.T., Dean, S., Rolf, E., Simchowitz, M., Hardt, M.: Delayed impact of fair machine learning. arXiv preprint arXiv:1803.04383 (2018)","DOI":"10.24963\/ijcai.2019\/862"},{"key":"67_CR46","volume-title":"Anarchy, state, and utopia","author":"R Nozick","year":"1974","unstructured":"Nozick, R.: Anarchy, state, and utopia, vol. 5038. Basic Books, New York (1974)"},{"key":"67_CR47","volume-title":"Equality or priority","author":"D Parfit","year":"1991","unstructured":"Parfit, D.: Equality or priority. Department of Philosophy, University of Kansas, Kansas (1991)"},{"key":"67_CR48","doi-asserted-by":"publisher","DOI":"10.17016\/FEDS.2009.33","volume-title":"Determinants of the locations of payday lenders, pawnshops and check-cashing outlets","author":"RA Prager","year":"2009","unstructured":"Prager, R.A., et al.: Determinants of the locations of payday lenders, pawnshops and check-cashing outlets. Federal Reserve Board Washington, Washington (2009)"},{"key":"67_CR49","doi-asserted-by":"crossref","unstructured":"Rawls, D.: A theory of justice, rev. ed (1999)","DOI":"10.4159\/9780674042582"},{"key":"67_CR50","doi-asserted-by":"crossref","DOI":"10.2307\/j.ctv31xf5v0","volume-title":"Justice as fairness: a restatement","author":"J Rawls","year":"2001","unstructured":"Rawls, J.: Justice as fairness: a restatement. Harvard University Press, Harvard (2001)"},{"key":"67_CR51","unstructured":"Saleiro, P., Kuester, B., Stevens, A., Anisfeld, A., Hinkson, L., London, L., Ghani, R.: Aequitas: a bias and fairness audit toolkit. arXiv preprint arXiv:1811.05577 (2018)"},{"key":"67_CR52","doi-asserted-by":"publisher","DOI":"10.1093\/0198235119.001.0001","volume-title":"The rejection of consequentialism: a philosophical investigation of the considerations underlying rival moral conceptions","author":"S Scheffler","year":"1994","unstructured":"Scheffler, S.: The rejection of consequentialism: a philosophical investigation of the considerations underlying rival moral conceptions. Oxford University Press, Oxford (1994)"},{"key":"67_CR53","volume-title":"Inequality reexamined","author":"AK Sen","year":"1992","unstructured":"Sen, A.K.: Inequality reexamined. Oxford University Press, Oxford (1992)"},{"key":"67_CR54","unstructured":"Suresh, H., Guttag, J.V.: A framework for understanding unintended consequences of machine learning (2019). arXiv:1901.10002"},{"key":"67_CR55","unstructured":"Van\u00a0Dam, A.: Searching for images of CEOs or managers? The results almost always show men. The Washington Post (3 01 2019)"},{"key":"67_CR56","doi-asserted-by":"crossref","unstructured":"Veale, M., Van\u00a0Kleek, M., Binns, R.: Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making. In Proceedings of the 2018 chi conference on human factors in computing systems, pp 1\u201314 (2018)","DOI":"10.1145\/3173574.3174014"},{"key":"67_CR57","doi-asserted-by":"crossref","unstructured":"Verma, S., Rubin, J.: Fairness definitions explained. In: 2018 IEEE\/ACM International Workshop on Software Fairness (FairWare). IEEE, pp 1\u20137 (2018)","DOI":"10.1145\/3194770.3194776"},{"key":"67_CR58","doi-asserted-by":"crossref","unstructured":"Wachter, S., Mittelstadt, S., Russell, R.: Why fairness cannot be automated: bridging the gap between EU non-discrimination law and AI. Available at SSRN (2020)","DOI":"10.31228\/osf.io\/jq2b6"},{"key":"67_CR59","doi-asserted-by":"crossref","unstructured":"Wachter, S., Mittelstadt, S., Russell, C.: Bias preservation in machine learning: the legality of fairness metrics under EU non-discrimination law. Available at SSRN (2021)","DOI":"10.2139\/ssrn.3792772"},{"key":"67_CR60","unstructured":"Walzer, M.: Spheres of justice, a defense of pluralism and equality, New York, Basic (1983)"},{"key":"67_CR61","doi-asserted-by":"crossref","unstructured":"Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., Zhu J.: Explainable AI: a brief survey on history, research areas, approaches and challenges. In: CCF international conference on natural language processing and Chinese computing. Springer, pp 563\u2013574 (2019)","DOI":"10.1007\/978-3-030-32236-6_51"},{"key":"67_CR62","unstructured":"Zillman, C.: Fortune 500 Female CEOs Reaches All-Time Record of 33. Fortune (16 05 2019)"}],"container-title":["AI and Ethics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-021-00067-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s43681-021-00067-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-021-00067-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,30]],"date-time":"2022-12-30T22:40:48Z","timestamp":1672440048000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s43681-021-00067-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,12]]},"references-count":62,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,11]]}},"alternative-id":["67"],"URL":"https:\/\/doi.org\/10.1007\/s43681-021-00067-y","relation":{},"ISSN":["2730-5953","2730-5961"],"issn-type":[{"value":"2730-5953","type":"print"},{"value":"2730-5961","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,12]]},"assertion":[{"value":"23 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Michelle Seng Ah Lee is employed part-time at Deloitte LLP.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}