{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T22:03:59Z","timestamp":1776117839666,"version":"3.50.1"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,2]]},"abstract":"<jats:p>\n            Numerous fairness metrics have been proposed and employed by artificial intelligence (AI) experts to quantitatively measure bias and define fairness in AI models. Recognizing the need to accommodate stakeholders' diverse fairness understandings, efforts are underway to solicit their input. However, conveying AI fairness metrics to stakeholders without AI expertise, capturing their personal preferences, and seeking a collective consensus remain challenging and underexplored. To bridge this gap, we propose a new framework, EARN (\n            <jats:italic toggle=\"yes\">Explain, Ask, Review, and Negotiate<\/jats:italic>\n            ) Fairness, which facilitates collective metric decisions among stakeholders without requiring AI expertise. The framework features an adaptable interactive system and a stakeholder-centered EARN Fairness process to\n            <jats:italic toggle=\"yes\">Explain<\/jats:italic>\n            fairness metrics,\n            <jats:italic toggle=\"yes\">Ask<\/jats:italic>\n            stakeholders' personal metric preferences,\n            <jats:italic toggle=\"yes\">Review<\/jats:italic>\n            metrics collectively, and\n            <jats:italic toggle=\"yes\">Negotiate<\/jats:italic>\n            a consensus on metric selection. To gather empirical results, we applied the framework to a credit rating scenario and conducted a user study involving 18 decision subjects without AI knowledge. We elicited their personal metric preferences and subsequently we studied how they reached metric consensus in team sessions. Our work shows that the EARN Fairness framework supports stakeholders to express and negotiate fairness preferences, and we provide practical guidance for implementing human-centered AI fairness in high-risk contexts. Through this approach, we aim to reach consensus of fairness perspectives, fostering more equitable and inclusive AI fairness.\n          <\/jats:p>","DOI":"10.1145\/3710908","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T11:36:19Z","timestamp":1747740979000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholders"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0310-3158","authenticated-orcid":false,"given":"Lin","family":"Luo","sequence":"first","affiliation":[{"name":"School of Computing Science, University of Glasgow, Glasgow, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6813-9952","authenticated-orcid":false,"given":"Yuri","family":"Nakao","sequence":"additional","affiliation":[{"name":"FUJITSU LIMITED, Tokyo, Japan and Graduate School of Arts and Sciences, The University of Tokyo, Tokyo, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9858-6844","authenticated-orcid":false,"given":"Mathieu","family":"Chollet","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, Glasgow, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4405-8952","authenticated-orcid":false,"given":"Hiroya","family":"Inakoshi","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Laboratory, FUJITSU LIMITED, Tokyo, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6482-1973","authenticated-orcid":false,"given":"Simone","family":"Stumpf","sequence":"additional","affiliation":[{"name":"School of Computing Science, University of Glasgow, Glasgow, United Kingdom"}]}],"member":"320","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934262"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","unstructured":"Sergio Alonso Enrique Herrera-Viedma Francisco Cabrerizo Carlos Porcel and A.G. L\u00f3pez-Herrera. 2007. Using Visualization Tools to Guide Consensus in Group Decision Making Vol. 4578. 77--85. doi:10.1007\/978--3--540--73400-0_10","DOI":"10.1007\/978--3--540--73400-0_10"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581227"},{"key":"e_1_2_2_4_1","volume-title":"Recent Advances in Algorithmic Biases and Fairness in Financial Services: A Survey. In Proceedings of the Future Technologies Conference (FTC)","volume":"1","author":"Bajracharya Aakriti","year":"2022","unstructured":"Aakriti Bajracharya, Utsab Khakurel, Barron Harvey, and Danda B. Rawat. 2023. Recent Advances in Algorithmic Biases and Fairness in Financial Services: A Survey. In Proceedings of the Future Technologies Conference (FTC) 2022, Volume 1, Kohei Arai (Ed.). Springer International Publishing, Cham, 809--822."},{"key":"e_1_2_2_5_1","article-title":"Dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python","volume":"22","author":"Baniecki Hubert","year":"2021","unstructured":"Hubert Baniecki, Wojciech Kretowicz, Piotr Piatyszek, Jakub Wisniewski, and Przemyslaw Biecek. 2021. Dalex: Responsible Machine Learning with Interactive Explainability and Fairness in Python. J. Mach. Learn. Res. 22, 1, Article 214 (jan 2021), 7 pages.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942287"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372864"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/VAST47406.2019.8986948"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-07939--1"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3616865"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445308"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533154"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","unstructured":"Damien Dablain Bartosz Krawczyk and Nitesh Chawla. 2022. Towards A Holistic View of Bias in Machine Learning: Bridging Algorithmic Fairness and Imbalanced Learning. doi:10.48550\/arXiv.2207.06084","DOI":"10.48550\/arXiv.2207.06084"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581527"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00203"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287589"},{"key":"e_1_2_2_22_1","volume-title":"Transitioning from Real to Synthetic data: Quantifying the bias in model. ArXiv abs\/2105.04144","author":"Gupta Aman","year":"2021","unstructured":"Aman Gupta, Deepak L. Bhatt, and Anubha Pandey. 2021. Transitioning from Real to Synthetic data: Quantifying the bias in model. ArXiv abs\/2105.04144 (2021). https:\/\/api.semanticscholar.org\/CorpusID:234336067"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3640543.3645209"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157469"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372831"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.24432\/C5NC77"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3631326"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.4230\/LIPIcs.FORC.2020.2"},{"key":"e_1_2_2_29_1","volume-title":"NeurIPS","author":"Jackson Alex","year":"2023","unstructured":"Alex Jackson, Michael Luck, and Elizabeth Black. 2023. Modelling Moral Debate using Reason-based Abstract Argumentation Theory. https:\/\/neurips.cc\/ 37th International Conference on Neural Information Processing Systems, NeurIPS 2023, NeurIPS 2023 ; Conference date: 10--12--2023 Through 16--12--2023."},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"3182","author":"Kahng Anson","year":"2019","unstructured":"Anson Kahng, Min Kyung Lee, Ritesh Noothigattu, Ariel Procaccia, and Christos-Alexandros Psomas. 2019. Statistical Foundations of Virtual Democracy. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 3173--3182. https:\/\/proceedings.mlr.press\/v97\/kahng19a.html"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/IC4.2009.4909197"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642849"},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"2572","author":"Kearns Michael","year":"2018","unstructured":"Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, 2564--2572. https:\/\/proceedings.mlr.press\/v80\/kearns18a.html"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287592"},{"key":"e_1_2_2_35_1","volume-title":"New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence, Juan F. de Paz Santana, Daniel H. de la Iglesia, and Alfonso Jos\u00e9 L\u00f3pez Rivero (Eds.)","author":"Kobayashi Kenji","unstructured":"Kenji Kobayashi and Yuri Nakao. 2022. One-vs.-One Mitigation of Intersectional Bias: A General Method for Extending Fairness-Aware Binary Classification. In New Trends in Disruptive Technologies, Tech Ethics and Artificial Intelligence, Juan F. de Paz Santana, Daniel H. de la Iglesia, and Alfonso Jos\u00e9 L\u00f3pez Rivero (Eds.). Springer International Publishing, Cham, 43--54."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145249"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490354.3494408"},{"key":"e_1_2_2_38_1","volume-title":"Garnett (Eds.)","volume":"30","author":"Kusner Matt J","year":"2017","unstructured":"Matt J Kusner, Joshua Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual Fairness. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1037\/amp0000972"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3375158"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359284"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025884"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359283"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3512899"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372867"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2022.2067936"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514258"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359159"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/3524938.3525714"},{"key":"e_1_2_2_51_1","volume-title":"Aequitas: A Bias and Fairness Audit Toolkit. arXiv:1811.05577 [cs.LG]","author":"Saleiro Pedro","year":"2019","unstructured":"Pedro Saleiro, Benedict Kuester, Loren Hinkson, Jesse London, Abby Stevens, Ari Anisfeld, Kit T. Rodolfa, and Rayid Ghani. 2019. Aequitas: A Bias and Fairness Audit Toolkit. arXiv:1811.05577 [cs.LG]"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314248"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1214\/17-AOAS1058"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330664"},{"key":"e_1_2_2_55_1","volume-title":"Design Methods for Artificial Intelligence Fairness and Transparency. In IUI Workshops. https:\/\/api.semanticscholar.org\/CorpusID:235789821","author":"Stumpf Simone","year":"2021","unstructured":"Simone Stumpf, Lorenzo Strappelli, Subeida Ahmed, Yuri Nakao, Aisha Naseer, Giulia Del Gamba, and Daniele Regoli. 2021. Design Methods for Artificial Intelligence Fairness and Transparency. In IUI Workshops. https:\/\/api.semanticscholar.org\/CorpusID:235789821"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3631700.3664912"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3194770.3194776"},{"key":"e_1_2_2_58_1","volume-title":"Fairlearn: Assessing and Improving Fairness of AI Systems. arXiv:2303.16626 [cs.LG]","author":"Weerts Hilde","year":"2023","unstructured":"Hilde Weerts, Miroslav Dud\u00edk, Richard Edgar, Adrin Jalali, Roman Lutz, and Michael Madaio. 2023. Fairlearn: Assessing and Improving Fairness of AI Systems. arXiv:2303.16626 [cs.LG]"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2934619"},{"key":"e_1_2_2_60_1","volume-title":"Rashida Richardson, Jason Schultz, Oscar Schwartz, et al .","author":"Whittaker Meredith","year":"2018","unstructured":"Meredith Whittaker, Kate Crawford, Roel Dobbe, Genevieve Fried, Elizabeth Kaziunas, Varoon Mathur, Sarah Mysers West, Rashida Richardson, Jason Schultz, Oscar Schwartz, et al . 2018. AI now report 2018. AI Now Institute at New York University New York."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-019-00355-w"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114850"},{"key":"e_1_2_2_63_1","volume-title":"AI Fairness in the Financial Industry: A Machine Learning Pipeline Approach. Future And Fintech","author":"Jun Xu.","year":"2022","unstructured":"Jun Xu. 2022. AI Fairness in the Financial Industry: A Machine Learning Pipeline Approach. Future And Fintech, The: Abcdi And Beyond (2022), 117."},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376447"},{"key":"e_1_2_2_65_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"333","author":"Zemel Rich","year":"2013","unstructured":"Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning Fair Representations. In Proceedings of the 30th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 28), Sanjoy Dasgupta and David McAllester (Eds.). PMLR, Atlanta, Georgia, USA, 325--333. https:\/\/proceedings.mlr.press\/v28\/zemel13.html"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579601"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375862"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675272"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710908","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3710908","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T09:24:58Z","timestamp":1755768298000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710908"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":68,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,5,2]]}},"alternative-id":["10.1145\/3710908"],"URL":"https:\/\/doi.org\/10.1145\/3710908","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"2025-05-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}