{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T14:31:59Z","timestamp":1781015519636,"version":"3.54.1"},"reference-count":25,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,4,30]],"date-time":"2021-04-30T00:00:00Z","timestamp":1619740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Queue"],"published-print":{"date-parts":[[2021,4,30]]},"abstract":"<jats:p>This article provides an organization of various kinds of biases that can occur in the AI pipeline starting from dataset creation and problem formulation to data analysis and evaluation. It highlights the challenges associated with the design of bias-mitigation strategies, and it outlines some best practices suggested by researchers. Finally, a set of guidelines is presented that could aid ML developers in identifying potential sources of bias, as well as avoiding the introduction of unwanted biases. The work is meant to serve as an educational resource for ML developers in handling and addressing issues related to bias in AI systems.<\/jats:p>","DOI":"10.1145\/3466132.3466134","type":"journal-article","created":{"date-parts":[[2021,5,12]],"date-time":"2021-05-12T22:07:50Z","timestamp":1620857270000},"page":"45-64","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":38,"title":["Biases in AI Systems"],"prefix":"10.1145","volume":"19","author":[{"given":"Ramya","family":"Srinivasan","sequence":"first","affiliation":[{"name":"Fujitsu Laboratories of America"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ajay","family":"Chander","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,5,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359301"},{"key":"e_1_2_1_2_1","unstructured":"Amatriain X. 2015. What does the concept of presentation-feedback bias refer to in the context of machine learning? Quora; https:\/\/www.quora.com\/What-does-the-concept-of-presentation-feedback-bias-refer-to-in-the-context-of-machine-learning."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2009.05.009"},{"key":"e_1_2_1_4_1","unstructured":"Barocas S. Hardt M. Narayanan A. 2019. Fairness and machine learning: limitations and opportunities; https:\/\/fairmlbook.org."},{"key":"e_1_2_1_5_1","volume-title":"AI Fairness 360: an extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv","author":"Bellamy R. K. E.","year":"1810","unstructured":"Bellamy, R. K. E., Dey, K., Hind, M., Hoffman, S. C., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K. N., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K. R., Zhang, Y. 2018. AI Fairness 360: an extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias. arXiv; https:\/\/arxiv.org\/abs\/1810.01943."},{"key":"e_1_2_1_6_1","first-page":"1","article-title":"Gender shades: intersectional accuracy disparities in commercial gender classification","volume":"81","author":"Buolamwini J.","year":"2018","unstructured":"Buolamwini, J., Gebru, T. 2018. Gender shades: intersectional accuracy disparities in commercial gender classification. In Proceedings of Machine Learning Research 81, 1-15; http:\/\/proceedings.mlr.press\/v81\/buolamwini18a\/buolamwini18a.pdf.","journal-title":"Proceedings of Machine Learning Research"},{"key":"e_1_2_1_7_1","volume-title":"Excavating AI: the politics of images in machine learning training sets","author":"Crawford K.","unstructured":"Crawford, K., Paglen, T. 2019. Excavating AI: the politics of images in machine learning training sets. The AI Now Institute, New York University; https:\/\/www.excavating.ai."},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Dressel J. Farid H. 2018. The accuracy fairness and limits of predicting recidivism. Science Advances 4(1); https:\/\/advances.sciencemag.org\/content\/4\/1\/eaao5580.","DOI":"10.1126\/sciadv.aao5580"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-soc-071913-043455"},{"key":"e_1_2_1_10_1","first-page":"230561","article-title":"Bias in computer systems","volume":"1145","author":"Friedman B.","year":"1996","unstructured":"Friedman, B., Nissenbaum, H. 1996. Bias in computer systems. In ACM Transactions on Information Systems 14(3), https:\/\/dl.acm.org\/doi\/10.1145\/230538.230561.","journal-title":"ACM Transactions on Information Systems 14(3), https:\/\/dl.acm.org\/doi\/10."},{"key":"e_1_2_1_11_1","unstructured":"Gajane P. Pechenizkiy M. 2018. On formalizing fairness in prediction with machine learning. In Proceedings of the International Conference on Machine Learning Fairness Accountability and Transparency Workshop; https:\/\/www.fatml.org\/media\/documents\/formalizing_fairness_in_prediction_with_ml.pdf."},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the Fifth Workshop on Fairness, Accountability, and Transparency in Machine Learning; https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/01\/1803","author":"Gebru T.","year":"2018","unstructured":"Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daum\u00e9 III, H., Crawford, K. 2018. Datasheets for datasets. In Proceedings of the Fifth Workshop on Fairness, Accountability, and Transparency in Machine Learning; https:\/\/www.microsoft.com\/en-us\/research\/uploads\/prod\/2019\/01\/1803.09010.pdf."},{"key":"e_1_2_1_13_1","volume-title":"This is how AI bias really happens?and why it's so hard to fix. MIT Technology Review","author":"Hao K.","year":"2019","unstructured":"Hao, K. 2019. This is how AI bias really happens?and why it's so hard to fix. MIT Technology Review; https:\/\/www.technologyreview.com\/2019\/02\/04\/137602\/this-is-how-ai-bias-really-happensand-why-its-so-hard-to-fix\/."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511803475.039"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the Conference on Uncertainty in Artificial Intelligence; http:\/\/auai.org\/uai2019\/proceedings\/papers\/213","author":"Kilbertus N.","year":"2019","unstructured":"Kilbertus, N., Ball, P. J., Kusner, M. J., Weller, A., Silva, R. 2019. The sensitivity of counterfactual fairness to unmeasured confounding. In Proceedings of the Conference on Uncertainty in Artificial Intelligence; http:\/\/auai.org\/uai2019\/proceedings\/papers\/213.pdf."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.296.6623.657"},{"key":"e_1_2_1_18_1","volume-title":"A survey on bias and fairness in machine learning. arXiv","author":"Mehrabi N.","year":"1908","unstructured":"Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A. 2019. A survey on bias and fairness in machine learning. arXiv; https:\/\/arxiv.org\/abs\/1908.09635."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_2_1_20_1","volume-title":"The Book of Why: The New Science of Cause and Effect","author":"Pearl J.","unstructured":"Pearl, J., Mackenzie, D. 2018. The Book of Why: The New Science of Cause and Effect. Basic Books."},{"key":"e_1_2_1_21_1","volume-title":"The Psychology of Judgment and Decision Making","author":"Plous S.","unstructured":"Plous, S. 1993. The Psychology of Judgment and Decision Making. McGraw-Hill."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314244"},{"key":"e_1_2_1_23_1","unstructured":"Small Z. 2019. 600 000 images removed from AI database after art project exposes racist bias. Hyperallergic; https:\/\/hyperallergic.com\/518822\/600000-imagesremoved-from-ai-database-after-art-project-exposesracist-bias\/."},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Conference on Machine Learning Workshop on Human in the Loop Learning; https:\/\/arxiv.org\/abs\/1906","author":"Srinivasan R.","year":"2019","unstructured":"Srinivasan, R., Chander, A. 2019. Crowdsourcing in the absence of ground truth?a case study. In Proceedings of the International Conference on Machine Learning Workshop on Human in the Loop Learning; https:\/\/arxiv.org\/abs\/1906.07254."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995347"}],"container-title":["Queue"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3466132.3466134","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3466132.3466134","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:18Z","timestamp":1750191498000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3466132.3466134"}},"subtitle":["A survey for practitioners"],"short-title":[],"issued":{"date-parts":[[2021,4,30]]},"references-count":25,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4,30]]}},"alternative-id":["10.1145\/3466132.3466134"],"URL":"https:\/\/doi.org\/10.1145\/3466132.3466134","relation":{},"ISSN":["1542-7730","1542-7749"],"issn-type":[{"value":"1542-7730","type":"print"},{"value":"1542-7749","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,30]]},"assertion":[{"value":"2021-05-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}