{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T04:43:20Z","timestamp":1789101800750,"version":"build-2803163510"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"DOI":"10.13039\/501100000266","name":"UK Engineering and Physical Sciences Research Council","doi-asserted-by":"crossref","award":["EP\/R513143\/1"],"award-info":[{"award-number":["EP\/R513143\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]},{"name":"University College London Interaction Centre"},{"name":"UCLIC Ethics Committee"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>The increasing adoption of machine learning (ML) raises ethical concerns, particularly regarding bias. This study explores how ML practitioners with limited experience in bias understand and apply bias definitions, detection measures, and mitigation methods. Through a take-home task, exercises, and interviews with 22 participants, we identified five key themes: sources of bias, selecting bias metrics, detecting bias, mitigating bias, and ethical considerations. Participants faced unresolved conflicts, such as applying fairness definitions in practice, selecting context-dependent bias metrics, addressing real-world biases, balancing model performance with bias mitigation, and relying on personal perspectives over data-driven metrics. While bias mitigation techniques helped identify biases in two datasets, participants could not fully eliminate bias, citing the oversimplification of complex processes into models with limited variables. We propose designing bias detection tools that encourage practitioners to focus on the underlying assumptions and integrating bias concepts into ML practices, such as using a harmonic mean-based approach, akin to the F1 score, to balance bias and accuracy.<\/jats:p>","DOI":"10.1145\/3733838","type":"journal-article","created":{"date-parts":[[2025,5,5]],"date-time":"2025-05-05T15:05:18Z","timestamp":1746457518000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Practitioners and Bias in Machine Learning: A Study"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4326-261X","authenticated-orcid":false,"given":"Robert","family":"Cinca","sequence":"first","affiliation":[{"name":"University College London, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1842-8342","authenticated-orcid":false,"given":"Enrico","family":"Costanza","sequence":"additional","affiliation":[{"name":"University College London, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9712-4090","authenticated-orcid":false,"given":"Mirco","family":"Musolesi","sequence":"additional","affiliation":[{"name":"University College London, London, United Kingdom of Great Britain and Northern Ireland and University of Bologna, Bologna, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,9]]},"reference":[{"issue":"1","key":"e_1_3_2_2_2","first-page":"139","volume":"23","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica 23, 1 (2016), 139\u2013159. Retrieved from https:\/\/www.propublica.org\/article\/machine-bias-risk-assessments-in-criminal-sentencing","journal-title":"Machine bias. ProPublica"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942288"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209581"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3600211.3604674"},{"key":"e_1_3_2_6_2","volume-title":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks","volume":"1","author":"Bao Michelle","year":"2021","unstructured":"Michelle Bao, Angela Zhou, Samantha Zottola, Brian Brubach, Brian Brubach, Sarah Desmarais, Aaron Horowitz, Kristian Lum, and Suresh Venkatasubramanian. 2021. It's COMPASlicated: The messy relationship between RAI datasets and algorithmic fairness benchmarks. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, Vol. 1. MIT Press. Retrieved from https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper_files\/paper\/2021\/file\/92cc227532d17e56e07902b254dfad10-Paper-round1.pdf"},{"key":"e_1_3_2_7_2","unstructured":"Solon Barocas Moritz Hardt and Arvind Narayanan. 2019. Fairness and Machine Learning. fairmlbook.org Retrieved from http:\/\/www.fairmlbook.org"},{"key":"e_1_3_2_8_2","first-page":"15849","volume-title":"Proceedings of the National Academy of Sciences","volume":"116","author":"Belkin Mikhail","year":"2019","unstructured":"Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal. 2019. Reconciling modern machine-learning practice and the classical bias\u2013variance trade-off. Proceedings of the National Academy of Sciences 116, 32 (2019), 15849\u201315854."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942287"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1177\/0049124118782533"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372864"},{"key":"e_1_3_2_12_2","unstructured":"Sarah Bird Miro Dud\u00edk Richard Edgar Brandon Horn Roman Lutz Vanessa Milan Mehrnoosh Sameki Hanna Wallach and Kathleen Walker. 2020. Fairlearn: A Toolkit for Assessing and Improving Fairness in AI. Technical Report MSR-TR-2020-32. Microsoft. Retrieved from https:\/\/www.microsoft.com\/en-us\/research\/publication\/fairlearn-a-toolkit-for-assessing-and-improving-fairness-in-ai\/"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706QP063OA"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1080\/2159676X.2019.1628806"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445308"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2024.103438"},{"issue":"2","key":"e_1_3_2_19_2","first-page":"120","article-title":"Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning","volume":"26","author":"Clarke Victoria","year":"2013","unstructured":"Victoria Clarke and Virginia Braun. 2013. Teaching thematic analysis: Overcoming challenges and developing strategies for effective learning. The Psychologist 26, 2 (2013), 120\u2013123.","journal-title":"The Psychologist"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3516392"},{"key":"e_1_3_2_21_2","unstructured":"European Commission. 2024. Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act). Retrieved form https:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?uri=CELEX:32024R1689"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145396"},{"key":"e_1_3_2_24_2","first-page":"473","volume-title":"Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency","author":"Hanwen Deng Wesley","year":"2022","unstructured":"Wesley Hanwen Deng, Manish Nagireddy, Michelle Seng Ah Lee, Jatinder Singh, Zhiwei Steven Wu, Kenneth Holstein, and Haiyi Zhu. 2022. Exploring how machine learning practitioners (try to) use fairness toolkits. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, 473\u2013484."},{"key":"e_1_3_2_25_2","first-page":"1","article-title":"COMPAS risk scales: Demonstrating accuracy equity and predictive parity","volume":"1","author":"Dieterich William","year":"2016","unstructured":"William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc 1 (2016), 1\u201339.","journal-title":"Northpointe Inc"},{"key":"e_1_3_2_26_2","unstructured":"Dheeru Dua and Casey Graff. 2017. UCI machine learning repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278733"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375828"},{"key":"e_1_3_2_30_2","first-page":"1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"5","author":"Ferreira Isabella","year":"2021","unstructured":"Isabella Ferreira, Jinghui Cheng, and Bram Adams. 2021. The \u201cshut the f** k up\u201d phenomenon: Characterizing incivility in open source code review discussions. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1\u201335."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3433949"},{"key":"e_1_3_2_32_2","unstructured":"Timnit Gebru Jamie Morgenstern Briana Vecchione Jennifer Wortman Vaughan Hanna Wallach Hal Daum\u00e9 III and Kate Crawford. 2018. Datasheets for datasets. arXiv:1803.09010. Retrieved from https:\/\/arxiv.org\/abs\/1803.09010"},{"key":"e_1_3_2_33_2","unstructured":"Google. 2020. Tensorfow\u2019s fairness evaluation and visualization toolkit. Retrieved from https:\/\/github.com\/tensorflow\/fairness-indicators"},{"key":"e_1_3_2_34_2","volume-title":"Proceedings of the NIPS Symposium on Machine Learning and the Law","volume":"1","author":"Grgic-Hlaca Nina","year":"2016","unstructured":"Nina Grgic-Hlaca, Muhammad Bilal Zafar, Krishna P. Gummadi, and Adrian Weller. 2016. The case for process fairness in learning: Feature selection for fair decision making. In Proceedings of the NIPS Symposium on Machine Learning and the Law, Vol. 1. ACM, 2."},{"key":"e_1_3_2_35_2","first-page":"304","volume-title":"Proceedings of the 29th International Conference on Intelligent User Interfaces","author":"Mohammad Rakin Haider Chowdhury","year":"2024","unstructured":"Chowdhury Mohammad Rakin Haider, Christopher Clifton, and Ming Yin. 2024. Do crowdsourced fairness preferences correlate with risk perceptions?. In Proceedings of the 29th International Conference on Intelligent User Interfaces. ACM, 304\u2013324."},{"key":"e_1_3_2_36_2","first-page":"3315","article-title":"Equality of opportunity in supervised learning","volume":"29","author":"Hardt Moritz","year":"2016","unstructured":"Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, Vol. 29, 3315\u20133323.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_3_2_38_2","first-page":"1","volume-title":"Proceedings of the ACM on Human-Computer Interaction 2","author":"Keyes Os","year":"2018","unstructured":"Os Keyes. 2018. The misgendering machines: Trans\/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1\u201322."},{"issue":"1","key":"e_1_3_2_39_2","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1017\/jme.2022.13","article-title":"Mitigating racial bias in machine learning","volume":"50","author":"Kostick-Quenet Kristin M.","year":"2022","unstructured":"Kristin M. Kostick-Quenet, I. Glenn Cohen, Sara Gerke, Bernard Lo, James Antaki, Faezah Movahedi, Hasna Njah, Lauren Schoen, Jerry E. Estep, and J. S. Blumenthal-Barby. 2022. Mitigating racial bias in machine learning. Journal of Law, Medicine & Ethics 50, 1 (2022), 92\u2013100.","journal-title":"Journal of Law, Medicine & Ethics"},{"key":"e_1_3_2_40_2","unstructured":"Cassie Kozyrkov. 2018. Advice for finding AI use cases. Retrieved from https:\/\/hackernoon.com\/imagine-a-drunk-island-advice-for-finding-ai-use-cases-8d47495d4c3f"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858529"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2207676.2207678"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.5555\/3294996.3295162"},{"key":"e_1_3_2_44_2","first-page":"1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Seng Michelle","year":"2021","unstructured":"Michelle Seng,Ah Lee, and Jat Singh. 2021. The landscape and gaps in open source fairness toolkits. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, 1\u201313."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","unstructured":"Annie Liang Jay Lu and Xiaosheng Mu. 2021. Algorithm design: A fairness-accuracy frontier. arXiv:2112.09975. Retrieved from 10.48550\/arXiv.2112.09975","DOI":"10.48550\/arXiv.2112.09975"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1519023"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236386.3241340"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3512899"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2017.05.009"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_3_2_51_2","unstructured":"Michele Merler Nalini Ratha Rogerio S. Feris and John R. Smith. 2019. Diversity in faces. arXiv:1901.10436. Retrieved from https:\/\/arxiv.org\/abs\/1901.10436"},{"key":"e_1_3_2_52_2","volume-title":"Artificial Intelligence: A Guide for Thinking Humans","author":"Mitchell Melanie","year":"2019","unstructured":"Melanie Mitchell. 2019. Artificial Intelligence: A Guide for Thinking Humans. Penguin, United Kingdom."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-042720-125902"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2022.2067936"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3514258"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341525.3393997"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.18574\/nyu\/9781479833641.001.0001"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2016.2598828"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445604"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/1754428.1754432"},{"key":"e_1_3_2_62_2","first-page":"8377","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Saha Debjani","year":"2020","unstructured":"Debjani Saha, Candice Schumann, Duncan Mcelfresh, John Dickerson, Michelle Mazurek, and Michael Tschantz. 2020. Measuring non-expert comprehension of machine learning fairness metrics. In Proceedings of the International Conference on Machine Learning. PMLR, 8377\u20138387."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.1811.05577"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314314"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3583204"},{"issue":"49","key":"e_1_3_2_66_2","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.21105\/joss.01904","article-title":"FAT forensics: A python toolbox for implementing and deploying fairness, accountability and transparency algorithms in predictive systems","volume":"5","author":"Sokol Kacper","year":"2020","unstructured":"Kacper Sokol, Alexander Hepburn, Rafael Poyiadzi, Matthew Clifford, Raul Santos-Rodriguez, and Peter Flach. 2020. FAT forensics: A python toolbox for implementing and deploying fairness, accountability and transparency algorithms in predictive systems. Journal of Open Source Software 5, 49 (2020), 1904.","journal-title":"Journal of Open Source Software"},{"key":"e_1_3_2_67_2","doi-asserted-by":"crossref","first-page":"2239","DOI":"10.1145\/3219819.3220046","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD \u201918)","author":"Speicher Till","year":"2018","unstructured":"Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P. Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018. A unified approach to quantifying algorithmic unfairness: Measuring individual & group unfairness via inequality indices. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD \u201918). ACM, 2239\u20132248."},{"issue":"2","key":"e_1_3_2_68_2","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1145\/3466132.3466134","article-title":"Biases in AI systems: A survey for practitioners","volume":"19","author":"Srinivasan Ramya","year":"2021","unstructured":"Ramya Srinivasan and Ajay Chander. 2021. Biases in AI systems: A survey for practitioners. Queue 19, 2 (2021), 45\u201364.","journal-title":"Queue"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330664"},{"key":"e_1_3_2_70_2","unstructured":"VentureBeat Staff. 2019. Why do 87% of data science projects never make it into production. Retrieved from https:\/\/venturebeat.com\/ai\/why-do-87-of-data-science-projects-never-make-it-into-production\/"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465416.3483305"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12880-015-0068-x"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1747-9991.2012.00500.x"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3580903"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174156"},{"issue":"1","key":"e_1_3_2_76_2","first-page":"1","article-title":"Mitigating bias in machine learning for medicine","volume":"1","author":"Vokinger Kerstin N.","year":"2021","unstructured":"Kerstin N. Vokinger, Stefan Feuerriegel, and Aaron S. Kesselheim. 2021. Mitigating bias in machine learning for medicine. Communications Medicine 1, 1 (2021), 1\u20133.","journal-title":"Communications Medicine"},{"issue":"1","key":"e_1_3_2_77_2","doi-asserted-by":"crossref","first-page":"i","DOI":"10.1007\/978-3-031-01685-1","article-title":"Why AI\/data science projects fail: How to avoid project pitfalls","volume":"1","author":"Weiner Joyce","year":"2020","unstructured":"Joyce Weiner. 2020. Why AI\/data science projects fail: How to avoid project pitfalls. Synthesis Lectures on Computation and Analytics 1, 1 (2020), i\u201377.","journal-title":"Synthesis Lectures on Computation and Analytics"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209897"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196729"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3733838","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,9]],"date-time":"2025-06-09T14:06:59Z","timestamp":1749478019000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3733838"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,9]]},"references-count":78,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3733838"],"URL":"https:\/\/doi.org\/10.1145\/3733838","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,9]]},"assertion":[{"value":"2024-06-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}