{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T03:43:32Z","timestamp":1784691812371,"version":"3.55.0"},"reference-count":82,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T00:00:00Z","timestamp":1700784000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["PZ00P2_186090"],"award-info":[{"award-number":["PZ00P2_186090"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021856","name":"Ministero dell\u2019Universit\u00e1 e della Ricerca","doi-asserted-by":"publisher","award":["2020W3A5FY"],"award-info":[{"award-number":["2020W3A5FY"]}],"id":[{"id":"10.13039\/501100021856","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003407","name":"Ministero dell\u2019Istruzione, dell\u2019Universit\u00e1 e della Ricerca","doi-asserted-by":"publisher","award":["2022B3BP5S"],"award-info":[{"award-number":["2022B3BP5S"]}],"id":[{"id":"10.13039\/501100003407","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning is part of the daily life of people and companies worldwide. Unfortunately, bias in machine learning algorithms risks unfairly influencing the decision-making process and reiterating possible discrimination. While the interest of the software engineering community in software fairness is rapidly increasing, there is still a lack of understanding of various aspects connected to fair machine learning engineering, i.e., the software engineering process involved in developing fairness-critical machine learning systems. Questions connected to the practitioners\u2019 awareness and maturity about fairness, the skills required to deal with the matter, and the best development phase(s) where fairness should be faced more are just some examples of the knowledge gaps currently open. In this paper, we provide insights into how fairness is perceived and managed in practice, to shed light on the instruments and approaches that practitioners might employ to properly handle fairness. We conducted a survey with 117 professionals who shared their knowledge and experience highlighting the relevance of fairness in practice, and the skills and tools required to handle it. The key results of our study show that fairness is still considered a second-class quality aspect in the development of artificial intelligence systems. The building of specific methods and development environments, other than automated validation tools, might help developers to treat fairness throughout the software lifecycle and revert this trend.<\/jats:p>","DOI":"10.1007\/s10664-023-10402-y","type":"journal-article","created":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T12:02:26Z","timestamp":1700827346000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":56,"title":["Fairness-aware machine learning engineering: how far are we?"],"prefix":"10.1007","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2590-436X","authenticated-orcid":false,"given":"Carmine","family":"Ferrara","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5491-0873","authenticated-orcid":false,"given":"Giulia","family":"Sellitto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0975-8972","authenticated-orcid":false,"given":"Filomena","family":"Ferrucci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9337-5116","authenticated-orcid":false,"given":"Fabio","family":"Palomba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4238-1425","authenticated-orcid":false,"given":"Andrea","family":"De Lucia","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,11,24]]},"reference":[{"key":"10402_CR1","doi-asserted-by":"publisher","unstructured":"Aggarwal A, Lohia P, Nagar S, Dey K, Saha D (2019) Black box fairness testing of machine learning models. In: Proceedings of the 2019 27th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2019, Association for Computing Machinery, New York, NY, USA, pp\u00a0625-635. https:\/\/doi.org\/10.1145\/3338906.3338937","DOI":"10.1145\/3338906.3338937"},{"issue":"2","key":"10402_CR2","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1207\/S15327590IJHC1602_04","volume":"16","author":"D Andrews","year":"2003","unstructured":"Andrews D, Nonnecke B, Preece J (2003) Electronic survey methodology: a case study in reaching hard-to-involve internet users. Int J Human-Comput Interact 16(2):185\u2013210. https:\/\/doi.org\/10.1207\/S15327590IJHC1602_04","journal-title":"Int J Human-Comput Interact"},{"key":"10402_CR3","unstructured":"Angwin J, Larson J (2016) Machine bias - there\u2019s software used across the country to predict future criminals. and it\u2019s biased against blacks. https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing"},{"issue":"2","key":"10402_CR4","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1093\/ijpor\/eds020","volume":"25","author":"OA Avdeyeva","year":"2013","unstructured":"Avdeyeva OA, Matland RE (2013) An experimental test of mail surveys as a tool for social inquiry in russia. Int J Public Opi Res 25(2):173\u2013194","journal-title":"Int J Public Opi Res"},{"issue":"1","key":"10402_CR5","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1080\/15228835.2017.1416512","volume":"36","author":"N Bantilan","year":"2018","unstructured":"Bantilan N (2018) Themis-ml: A fairness-aware machine learning interface for end-to-end discrimination discovery and mitigation. J Technol Human Serv 36(1):15\u201330. https:\/\/doi.org\/10.1080\/15228835.2017.1416512","journal-title":"J Technol Human Serv"},{"key":"10402_CR6","unstructured":"Barocas S, Hardt M, Narayanan A (2017) Fairness in machine learning. Nips tutorial 1:2"},{"key":"10402_CR7","doi-asserted-by":"publisher","unstructured":"Bellamy RKE, Dey K, Hind M, Hoffman SC, Houde S, Kannan K, Lohia P, Martino J, Mehta S, Mojsilovi\u0107 A, Nagar S, Ramamurthy KN, Richards J, Saha D, Sattigeri P, Singh M, Varshney KR, Zhang, Y (2019) Ai fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias. IBM J Res Dev 63(4\/5):4:1\u20134:15. https:\/\/doi.org\/10.1147\/JRD.2019.2942287","DOI":"10.1147\/JRD.2019.2942287"},{"key":"10402_CR8","doi-asserted-by":"publisher","unstructured":"Bird S, Hutchinson B, Kenthapadi K, K\u0131c\u0131man E, Mitchell M (2019) Fairness-aware machine learning: Practical challenges and lessons learned. In: companion proceedings of The 2019 World Wide Web Conference, WWW \u201919, Association for Computing Machinery, New York, NY, USA, pp\u00a01297-1298. https:\/\/doi.org\/10.1145\/3308560.3320086","DOI":"10.1145\/3308560.3320086"},{"key":"10402_CR9","doi-asserted-by":"crossref","unstructured":"Biswas S, Rajan H (2020) Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp\u00a0642\u2013653","DOI":"10.1145\/3368089.3409704"},{"key":"10402_CR10","doi-asserted-by":"publisher","unstructured":"Biswas S, Rajan H (2021) Fair preprocessing: towards understanding compositional fairness of data transformers in machine learning pipeline. In: Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2021, Association for Computing Machinery, New York, NY, USA, pp\u00a0981-993. https:\/\/doi.org\/10.1145\/3468264.3468536","DOI":"10.1145\/3468264.3468536"},{"key":"10402_CR11","doi-asserted-by":"crossref","unstructured":"Brun Y, Meliou A (2018) Software fairness. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, pp\u00a0754\u2013759","DOI":"10.1145\/3236024.3264838"},{"key":"10402_CR12","doi-asserted-by":"publisher","unstructured":"Caliskan A, Bryson JJ, Narayanan A (2017) Semantics derived automatically from language corpora contain human-like biases. Science 356(6334):183\u2013186. https:\/\/doi.org\/10.1126\/science.aal4230","DOI":"10.1126\/science.aal4230"},{"key":"10402_CR13","unstructured":"Calmon FP, Wei D, Vinzamuri B, Ramamurthy KN, Varshney KR (2017) Optimized pre-processing for discrimination prevention. In: Proceedings of the 31st international conference on neural information processing systems, NIPS\u201917, Curran Associates Inc., Red Hook, NY, USA, pp\u00a03995-4004"},{"key":"10402_CR14","doi-asserted-by":"crossref","unstructured":"Catania B, Guerrini G, Accinelli C (2022) Fairness & friends in the data science era. AI & SOCIETY pp\u00a01\u201311","DOI":"10.1007\/s00146-022-01472-5"},{"key":"10402_CR15","doi-asserted-by":"publisher","unstructured":"Celis LE, Huang L, Keswani V, Vishnoi NK (2019) Classification with fairness constraints: a meta-algorithm with provable guarantees. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* \u201919, Association for Computing Machinery, New York, NY, USA, pp\u00a0319-328. https:\/\/doi.org\/10.1145\/3287560.3287586","DOI":"10.1145\/3287560.3287586"},{"key":"10402_CR16","doi-asserted-by":"crossref","unstructured":"Chakraborty J, Majumder S, Menzies T (2021) Bias in machine learning software: why? how? what to do? In: Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, pp\u00a0429\u2013440","DOI":"10.1145\/3468264.3468537"},{"key":"10402_CR17","doi-asserted-by":"publisher","unstructured":"Chakraborty J, Majumder S, Yu Z, Menzies T (2020) Fairway: a way to build fair ml software. In: Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2020, Association for Computing Machinery, New York, NY, USA, pp\u00a0654-665. https:\/\/doi.org\/10.1145\/3368089.3409697","DOI":"10.1145\/3368089.3409697"},{"key":"10402_CR18","doi-asserted-by":"publisher","unstructured":"Chakraborty J, Peng K, Menzies T (2021) Making fair ml software using trustworthy explanation. In: Proceedings of the 35th IEEE\/ACM International Conference on Automated Software Engineering, ASE \u201920, Association for Computing Machinery, New York, NY, USA, pp\u00a01229-1233. https:\/\/doi.org\/10.1145\/3324884.3418932","DOI":"10.1145\/3324884.3418932"},{"key":"10402_CR19","unstructured":"Chakraborty J, Xia T, Fahid FM, Menzies T (2019) Software engineering for fairness: a case study with hyperparameter optimization. arXiv:1905.05786"},{"key":"10402_CR20","doi-asserted-by":"publisher","unstructured":"Chen Z, Zhang JM, Sarro F, Harman M (2022) Maat: a novel ensemble approach to addressing fairness and performance bugs for machine learning software. In: Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2022, Association for Computing Machinery, New York, NY, USA, pp\u00a01122-1134. https:\/\/doi.org\/10.1145\/3540250.3549093","DOI":"10.1145\/3540250.3549093"},{"issue":"5","key":"10402_CR21","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1145\/3376898","volume":"63","author":"A Chouldechova","year":"2020","unstructured":"Chouldechova A, Roth A (2020) A snapshot of the frontiers of fairness in machine learning. Commun ACM 63(5):82\u201389","journal-title":"Commun ACM"},{"issue":"1","key":"10402_CR22","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1086\/269355","volume":"57","author":"AH Church","year":"1993","unstructured":"Church AH (1993) Estimating the effect of incentives on mail survey response rates: A meta-analysis. Public Opin Q 57(1):62\u201379","journal-title":"Public Opin Q"},{"key":"10402_CR23","doi-asserted-by":"crossref","unstructured":"Dastin J (2018) Amazon scraps secret ai recruiting tool that showed bias against women. In: Ethics of data and analytics, Auerbach Publications, pp\u00a0296\u2013299","DOI":"10.1201\/9781003278290-44"},{"key":"10402_CR24","unstructured":"Ebert F, Serebrenik A, Treude C, Novielli N, Castor, F (2022) On recruiting experienced github contributors for interviews and surveys on prolific. In: International workshop on recruiting participants for empirical software engineering"},{"issue":"6","key":"10402_CR25","doi-asserted-by":"publisher","first-page":"2074","DOI":"10.1007\/s10618-022-00854-z","volume":"36","author":"A Fabris","year":"2022","unstructured":"Fabris A, Messina S, Silvello G, Susto GA (2022) Algorithmic fairness datasets: the story so far. Data Min Knowl Discov 36(6):2074\u20132152","journal-title":"Data Min Knowl Discov"},{"key":"10402_CR26","doi-asserted-by":"publisher","unstructured":"Ferrara C, Sellitto G, Ferrucci F, Palomba F, De\u00a0Lucia A (2022) Fairness in machine learning systems: How far are we? \u2013 online appendix. https:\/\/doi.org\/10.6084\/m9.figshare.21680519","DOI":"10.6084\/m9.figshare.21680519"},{"key":"10402_CR27","doi-asserted-by":"crossref","unstructured":"Finkelstein A, Harman M, Mansouri SA, Ren J, Zhang Y (2008) \u201cfairness analysis\u201d in requirements assignments. In: 2008 16th IEEE International Requirements Engineering Conference, IEEE, pp\u00a0115\u2013124","DOI":"10.1109\/RE.2008.61"},{"key":"10402_CR28","doi-asserted-by":"crossref","unstructured":"Friedler SA, Scheidegger C, Venkatasubramanian S, Choudhary S, Hamilton EP, Roth D (2019) A comparative study of fairness-enhancing interventions in machine learning. In: Proceedings of the conference on fairness, accountability, and transparency, pp\u00a0329\u2013338","DOI":"10.1145\/3287560.3287589"},{"key":"10402_CR29","doi-asserted-by":"crossref","unstructured":"Galhotra S, Brun Y, Meliou A (2017) Fairness testing: testing software for discrimination. In: Proceedings of the 2017 11th joint meeting on foundations of software engineering, pp\u00a0498\u2013510","DOI":"10.1145\/3106237.3106277"},{"key":"10402_CR30","doi-asserted-by":"publisher","unstructured":"Garousi V, Felderer M, Kuhrmann M, Herkilo\u011flu K (2017) What industry wants from academia in software testing? hearing practitioners\u2019 opinions. In: Proceedings of the 21st international conference on evaluation and assessment in software engineering, EASE \u201917, Association for Computing Machinery, New York, NY, USA, pp\u00a065-69. https:\/\/doi.org\/10.1145\/3084226.3084264","DOI":"10.1145\/3084226.3084264"},{"issue":"2","key":"10402_CR31","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1007\/s00146-022-01455-6","volume":"38","author":"B Giovanola","year":"2023","unstructured":"Giovanola B, Tiribelli S (2023) Beyond bias and discrimination: redefining the ai ethics principle of fairness in healthcare machine-learning algorithms. AI & society 38(2):549\u2013563","journal-title":"AI & society"},{"issue":"2","key":"10402_CR32","doi-asserted-by":"publisher","first-page":"99","DOI":"10.3109\/01421599609034141","volume":"18","author":"L Haggarty","year":"1996","unstructured":"Haggarty L (1996) What is content analysis? Medical Teacher 18(2):99\u2013101. https:\/\/doi.org\/10.3109\/01421599609034141","journal-title":"Medical Teacher"},{"issue":"4","key":"10402_CR33","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1023\/A:1011922615502","volume":"6","author":"T Hall","year":"2001","unstructured":"Hall T, Flynn V (2001) Ethical issues in software engineering research: a survey of current practice. Empir Softw Eng 6(4):305\u2013317","journal-title":"Empir Softw Eng"},{"key":"10402_CR34","unstructured":"Hapke H, Nelson C (2020) Building machine learning pipelines. O\u2019Reilly Media"},{"key":"10402_CR35","doi-asserted-by":"publisher","unstructured":"Horkoff J (2019) Non-functional requirements for machine learning: Challenges and new directions. In: 2019 IEEE 27th International Requirements Engineering Conference (RE), pp\u00a0386\u2013391. https:\/\/doi.org\/10.1109\/RE.2019.00050","DOI":"10.1109\/RE.2019.00050"},{"key":"10402_CR36","doi-asserted-by":"publisher","unstructured":"Hort M, Sarro F (2021) Did you do your homework? raising awareness on software fairness and discrimination. In: 2021 36th IEEE\/ACM international conference on automated software engineering (ASE), pp\u00a01322\u20131326. https:\/\/doi.org\/10.1109\/ASE51524.2021.9678568","DOI":"10.1109\/ASE51524.2021.9678568"},{"key":"10402_CR37","doi-asserted-by":"publisher","unstructured":"Hort M, Zhang JM, Sarro F, Harman M (2021) Fairea: a model behaviour mutation approach to benchmarking bias mitigation methods. In: Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2021, Association for Computing Machinery, New York, NY, USA, pp\u00a0994-1006. https:\/\/doi.org\/10.1145\/3468264.3468565","DOI":"10.1145\/3468264.3468565"},{"key":"10402_CR38","doi-asserted-by":"crossref","unstructured":"Islam MT, Fariha A, Meliou A (2021) Through the data management lens: experimental analysis and evaluation of fair classification. arXiv:2101.07361","DOI":"10.1145\/3514221.3517841"},{"key":"10402_CR39","doi-asserted-by":"publisher","unstructured":"Johnson B, Brun Y (2022) Fairkit-learn: a fairness evaluation and comparison toolkit. In: 2022 IEEE\/ACM 44th international conference on software engineering: companion proceedings (ICSE-Companion), pp\u00a070\u201374. https:\/\/doi.org\/10.1145\/3510454.3516830","DOI":"10.1145\/3510454.3516830"},{"issue":"1","key":"10402_CR40","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1145\/566493.566495","volume":"27","author":"BA Kitchenham","year":"2002","unstructured":"Kitchenham BA, Pfleeger SL (2002) Principles of survey research part 2: designing a survey. ACM SIGSOFT Softw Eng Notes 27(1):18\u201320","journal-title":"ACM SIGSOFT Softw Eng Notes"},{"key":"10402_CR41","doi-asserted-by":"publisher","unstructured":"Kolltveit AB, Li J (2023) Operationalizing machine learning models: A systematic literature review. In: Proceedings of the 1st workshop on software engineering for responsible AI, SE4RAI \u201922, Association for Computing Machinery, New York, NY, USA, pp\u00a01-8. https:\/\/doi.org\/10.1145\/3526073.3527584","DOI":"10.1145\/3526073.3527584"},{"key":"10402_CR42","unstructured":"Kusner MJ, Loftus J, Russell C, Silva R (2017) Counterfactual fairness. Adv Neural Inf Process Syst 30"},{"issue":"3","key":"10402_CR43","volume":"12","author":"T Le Quy","year":"2022","unstructured":"Le Quy T, Roy A, Iosifidis V, Zhang W, Ntoutsi E (2022) A survey on datasets for fairness-aware machine learning. Wiley Interdiscip Rev: Data Min Knowl Discov 12(3):e1452","journal-title":"Wiley Interdiscip Rev: Data Min Knowl Discov"},{"key":"10402_CR44","doi-asserted-by":"crossref","unstructured":"Li Y, Meng L, Chen L, Yu L, Wu D, Zhou Y, Xu B (2022) Training data debugging for the fairness of machine learning software. In: 2022 IEEE\/ACM 44th international conference on software engineering (ICSE), IEEE, pp\u00a02215\u20132227","DOI":"10.1145\/3510003.3510091"},{"key":"10402_CR45","doi-asserted-by":"publisher","unstructured":"Madaio M, Egede L, Subramonyam H, Wortman\u00a0Vaughan J, Wallach H (2022) Assessing the fairness of ai systems: Ai practitioners\u2019 processes, challenges, and needs for support. Proc ACM Hum-Comput Interact 6(CSCW1). https:\/\/doi.org\/10.1145\/3512899","DOI":"10.1145\/3512899"},{"issue":"2","key":"10402_CR46","first-page":"159","volume":"1","author":"P Mayring","year":"2004","unstructured":"Mayring P et al (2004) Qualitative content analysis. A companion to qualitative research 1(2):159\u2013176","journal-title":"A companion to qualitative research"},{"issue":"6","key":"10402_CR47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3457607","volume":"54","author":"N Mehrabi","year":"2021","unstructured":"Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A (2021) A survey on bias and fairness in machine learning. ACM computing surveys (CSUR) 54(6):1\u201335","journal-title":"ACM computing surveys (CSUR)"},{"key":"10402_CR48","doi-asserted-by":"publisher","unstructured":"Mehrabi N, Morstatter F, Saxena N, Lerman K, Galstyan A (2021) A survey on bias and fairness in machine learning. ACM Comput. Surv 54(6). https:\/\/doi.org\/10.1145\/3457607","DOI":"10.1145\/3457607"},{"key":"10402_CR49","doi-asserted-by":"publisher","unstructured":"Menzies T, Peng K, Lustosa A (2021) Fairer software made easier (using \u201ckeys\u201d). In: 2021 36th IEEE\/ACM international conference on automated software engineering workshops (ASEW), pp\u00a0108\u2013113. https:\/\/doi.org\/10.1109\/ASEW52652.2021.00031","DOI":"10.1109\/ASEW52652.2021.00031"},{"key":"10402_CR50","unstructured":"Miller CC (2015) Can an algorithm hire better than a human. The New York Times 25"},{"key":"10402_CR51","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1146\/annurev-statistics-042720-125902","volume":"8","author":"S Mitchell","year":"2021","unstructured":"Mitchell S, Potash E, Barocas S, D\u2019Amour A, Lum K (2021) Algorithmic fairness: Choices, assumptions, and definitions. Ann Rev Stat Appl 8:141\u2013163","journal-title":"Ann Rev Stat Appl"},{"key":"10402_CR52","doi-asserted-by":"crossref","unstructured":"Morin KH (2013) Value of a pilot study","DOI":"10.3928\/01484834-20130920-10"},{"key":"10402_CR53","unstructured":"Moumoulidou Z, McGregor A, Meliou A (2020) Diverse data selection under fairness constraints. arXiv:2010.09141"},{"key":"10402_CR54","unstructured":"Nemoto T, Beglar D (2014) Likert-scale questionnaires. In: JALT 2013 conference proceedings, pp\u00a01\u20138. https:\/\/jalt-publications.org\/sites\/default\/files\/pdf-article\/jalt2013_001.pdf"},{"issue":"6464","key":"10402_CR55","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","volume":"366","author":"Z Obermeyer","year":"2019","unstructured":"Obermeyer Z, Powers B, Vogeli C, Mullainathan S (2019) Dissecting racial bias in an algorithm used to manage the health of populations. Science 366(6464):447\u2013453","journal-title":"Science"},{"key":"10402_CR56","unstructured":"Olson P (2011) The algorithm that beats your bank manager. CNN Money March 15"},{"key":"10402_CR57","doi-asserted-by":"crossref","unstructured":"Pagano TP, Loureiro RB, Lisboa FVN, Peixoto RM, Guimar\u00e3es, GAS, Cruz GOR, Araujo MM, Santos LL, Cruz MAS, Oliveira ELS, Winkler I, Nascimento EGS (2023) Bias and unfairness in machine learning models: a systematic review on datasets, tools, fairness metrics, and identification and mitigation methods. Big Data and Cognitive Computing 7(1). https:\/\/www.mdpi.com\/2504-2289\/7\/1\/15","DOI":"10.3390\/bdcc7010015"},{"key":"10402_CR58","doi-asserted-by":"publisher","first-page":"2719","DOI":"10.1007\/s10664-016-9485-x","volume":"22","author":"C Palomares","year":"2017","unstructured":"Palomares C, Quer C, Franch X (2017) Requirements reuse and requirement patterns: a state of the practice survey. Empir Softw Eng 22:2719\u20132762","journal-title":"Empir Softw Eng"},{"key":"10402_CR59","doi-asserted-by":"publisher","unstructured":"Patel AR, Chandrasekaran J, Lei Y, Kacker RN, Kuhn DR (2022) A combinatorial approach to fairness testing of machine learning models. In: 2022 IEEE international conference on software testing, verification and validation workshops (ICSTW), pp\u00a094\u2013101. https:\/\/doi.org\/10.1109\/ICSTW55395.2022.00030","DOI":"10.1109\/ICSTW55395.2022.00030"},{"issue":"4","key":"10402_CR60","doi-asserted-by":"publisher","first-page":"2426","DOI":"10.1109\/TSE.2022.3220713","volume":"49","author":"K Peng","year":"2023","unstructured":"Peng K, Chakraborty J, Menzies T (2023) Fairmask: better fairness via model-based rebalancing of protected attributes. IEEE Trans Softw Eng 49(4):2426\u20132439. https:\/\/doi.org\/10.1109\/TSE.2022.3220713","journal-title":"IEEE Trans Softw Eng"},{"issue":"3","key":"10402_CR61","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/s10664-022-10116-7","volume":"27","author":"A Perera","year":"2022","unstructured":"Perera A, Aleti A, Tantithamthavorn C, Jiarpakdee J, Turhan B, Kuhn L, Walker K (2022) Search-based fairness testing for regression-based machine learning systems. Empir Softw Eng 27(3):79","journal-title":"Empir Softw Eng"},{"key":"10402_CR62","doi-asserted-by":"publisher","unstructured":"Pessach D, Shmueli E (2022) a review on fairness in machine learning. ACM Comput Surv 55(3). https:\/\/doi.org\/10.1145\/3494672","DOI":"10.1145\/3494672"},{"key":"10402_CR63","doi-asserted-by":"publisher","unstructured":"Rafi DM, Moses KRK, Petersen K, M\u00e4ntyl\u00e4 MV (2012) Benefits and limitations of automated software testing: Systematic literature review and practitioner survey. In: 2012 7th international workshop on automation of software test (AST), pp\u00a036\u201342. https:\/\/doi.org\/10.1109\/IWAST.2012.6228988","DOI":"10.1109\/IWAST.2012.6228988"},{"key":"10402_CR64","unstructured":"Rech J, Althoff KD (2004) Artificial intelligence and software engineering: status and future trends. KI 18(3):5\u201311"},{"key":"10402_CR65","unstructured":"Reid B, Wagner M, d\u2019Amorim M, Treude C (2022) Software engineering user study recruitment on prolific: an experience report. arXiv:2201.05348"},{"key":"10402_CR66","unstructured":"Saha D, Schumann C, McElfresh DC, Dickerson JP, Mazurek ML, Tschantz MC (2020) Measuring non-expert comprehension of machine learning fairness metrics. In: Proceedings of the 37th international conference on machine learning, ICML\u201920. JMLR.org"},{"issue":"4","key":"10402_CR67","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1109\/32.799955","volume":"25","author":"C Seaman","year":"1999","unstructured":"Seaman C (1999) Qualitative methods in empirical studies of software engineering. IEEE Trans Softw Eng 25(4):557\u2013572. https:\/\/doi.org\/10.1109\/32.799955","journal-title":"IEEE Trans Softw Eng"},{"key":"10402_CR68","doi-asserted-by":"crossref","unstructured":"Shrestha S, Das S (2022) Exploring gender biases in ml and ai academic research through systematic literature review. Front Artif Intell 5","DOI":"10.3389\/frai.2022.976838"},{"issue":"1","key":"10402_CR69","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12874-019-0868-8","volume":"19","author":"MG Smith","year":"2019","unstructured":"Smith MG, Witte M, Rocha S, Basner M (2019) Effectiveness of incentives and follow-up on increasing survey response rates and participation in field studies. BMC Med Res Methodol 19(1):1\u201313","journal-title":"BMC Med Res Methodol"},{"key":"10402_CR70","doi-asserted-by":"publisher","unstructured":"Soremekun E, Udeshi S, Chattopadhyay S (2022) Astraea: grammar-based fairness testing. IEEE Trans Softw Eng 48(12):5188\u20135211. https:\/\/doi.org\/10.1109\/TSE.2022.3141758","DOI":"10.1109\/TSE.2022.3141758"},{"issue":"2","key":"10402_CR71","doi-asserted-by":"publisher","first-page":"392","DOI":"10.3390\/make3020020","volume":"3","author":"S Studer","year":"2021","unstructured":"Studer S, Bui TB, Drescher C, Hanuschkin A, Winkler L, Peters S, M\u00fcller KR (2021) Towards crisp-ml (q): a machine learning process model with quality assurance methodology. Mach Learn Knowl Extraction 3(2):392\u2013413","journal-title":"Mach Learn Knowl Extraction"},{"key":"10402_CR72","doi-asserted-by":"publisher","unstructured":"Udeshi S, Arora P, Chattopadhyay S (2018) Automated directed fairness testing. In: Proceedings of the 33rd ACM\/IEEE international conference on automated software engineering, ASE \u201918, Association for Computing Machinery, New York, NY, USA, pp\u00a098-108. https:\/\/doi.org\/10.1145\/3238147.3238165","DOI":"10.1145\/3238147.3238165"},{"key":"10402_CR73","doi-asserted-by":"crossref","unstructured":"Vasudevan S, Kenthapadi K (2020) Lift: a scalable framework for measuring fairness in ml applications. In: Proceedings of the 29th ACM international conference on information & knowledge management, pp\u00a02773\u20132780","DOI":"10.1145\/3340531.3412705"},{"key":"10402_CR74","doi-asserted-by":"publisher","unstructured":"Verma S, Rubin J (2018) Fairness definitions explained. In: 2018 IEEE\/ACM international workshop on software fairness (FairWare), pp\u00a01\u20137. https:\/\/doi.org\/10.23919\/FAIRWARE.2018.8452913","DOI":"10.23919\/FAIRWARE.2018.8452913"},{"key":"10402_CR75","doi-asserted-by":"crossref","unstructured":"Wohlin C, Runeson P, H\u00f6st M, Ohlsson MC, Regnell B, Wessl\u00e9n A (2012) Experimentation in software engineering. Springer Science and Business Media","DOI":"10.1007\/978-3-642-29044-2"},{"key":"10402_CR76","doi-asserted-by":"crossref","unstructured":"Xivuri K, Twinomurinzi H (2021) A systematic review of fairness in artificial intelligence algorithms. In: Responsible AI and analytics for an ethical and inclusive digitized society: 20th IFIP WG 6.11 Conference on e-Business, e-Services and e-Society, I3E 2021, Galway, Ireland, September 1\u20133, 2021, Proceedings 20, Springer, pp\u00a0271\u2013284","DOI":"10.1007\/978-3-030-85447-8_24"},{"key":"10402_CR77","doi-asserted-by":"publisher","unstructured":"Xu D, Yuan S, Zhang L, Wu X (2019) Fairgan+: Achieving fair data generation and classification through generative adversarial nets. In: 2019 IEEE international conference on big data (Big Data), pp\u00a01401\u20131406. https:\/\/doi.org\/10.1109\/BigData47090.2019.9006322","DOI":"10.1109\/BigData47090.2019.9006322"},{"key":"10402_CR78","doi-asserted-by":"publisher","unstructured":"Zhang BH, Lemoine B, Mitchell M (2018) Mitigating unwanted biases with adversarial learning. In: Proceedings of the 2018 AAAI\/ACM conference on AI, ethics, and society, AIES \u201918, Association for Computing Machinery, New York, NY, USA, pp\u00a0335-340. https:\/\/doi.org\/10.1145\/3278721.3278779","DOI":"10.1145\/3278721.3278779"},{"key":"10402_CR79","doi-asserted-by":"publisher","unstructured":"Zhang JM, Harman M (2021) \u201cignorance and prejudice\" in software fairness. In: 2021 IEEE\/ACM 43rd international conference on software engineering (ICSE), pp\u00a01436\u20131447. https:\/\/doi.org\/10.1109\/ICSE43902.2021.00129","DOI":"10.1109\/ICSE43902.2021.00129"},{"key":"10402_CR80","doi-asserted-by":"publisher","unstructured":"Zhang M, Sun J (2022) Adaptive fairness improvement based on causality analysis. In: Proceedings of the 30th ACM joint European software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2022, Association for Computing Machinery, New York, NY, USA, pp\u00a06-17. https:\/\/doi.org\/10.1145\/3540250.3549103","DOI":"10.1145\/3540250.3549103"},{"key":"10402_CR81","unstructured":"Zhou J, Chen F (2018) Human and machine learning: Visible, explainable, trustworthy and transparent"},{"key":"10402_CR82","doi-asserted-by":"crossref","unstructured":"Zhou Y, Yu Y, Ding B (2020) Towards mlops: a case study of ml pipeline platform. In: 2020 international conference on artificial intelligence and computer engineering (ICAICE), IEEE, pp\u00a0494\u2013500","DOI":"10.1109\/ICAICE51518.2020.00102"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10402-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10402-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10402-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T13:25:28Z","timestamp":1711545928000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10402-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,24]]},"references-count":82,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["10402"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10402-y","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,24]]},"assertion":[{"value":"3 October 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interest"}}],"article-number":"9"}}