{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T19:02:40Z","timestamp":1776193360087,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,5,13]]},"DOI":"10.1145\/3308560.3317590","type":"proceedings-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T12:17:59Z","timestamp":1557749879000},"page":"539-544","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":179,"title":["Managing Bias in AI"],"prefix":"10.1145","author":[{"given":"Drew","family":"Roselli","sequence":"first","affiliation":[{"name":"ParallelM"}]},{"given":"Jeanna","family":"Matthews","sequence":"additional","affiliation":[{"name":"Clarkson Univ."}]},{"given":"Nisha","family":"Talagala","sequence":"additional","affiliation":[{"name":"ParallelM"}]}],"member":"320","published-online":{"date-parts":[[2019,5,13]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Witten H. Frank E. and Hall M.A. (2011). Data Mining: Practical Machine Learning Tools and Techniques 3rd ed. Morgan Kaufmann Publishers Inc. San Francisco CA.   Witten H. Frank E. and Hall M.A. (2011). Data Mining: Practical Machine Learning Tools and Techniques 3rd ed. Morgan Kaufmann Publishers Inc. San Francisco CA.","DOI":"10.1016\/B978-0-12-374856-0.00001-8"},{"key":"e_1_3_2_1_2_1","volume-title":"Artificial Intelligence: The Next Digital Frontier?","author":"McKinsey Global Institute","year":"2017","unstructured":"McKinsey Global Institute ( 2017 ). \u201c Artificial Intelligence: The Next Digital Frontier? \u201d McKinsey Global Institute (2017). \u201cArtificial Intelligence: The Next Digital Frontier?\u201d"},{"key":"e_1_3_2_1_3_1","volume-title":"ICML Workshop on Human Interpretability in Machine Learning","author":"Goodman B.","year":"2016","unstructured":"Goodman , B. and Flaxman , S . ( 2016 ). \u201c European Union regulations on algorithmic decision-making and a 'right to explanation '\u201d. ICML Workshop on Human Interpretability in Machine Learning , New York, NY. Goodman, B. and Flaxman, S. (2016). \u201cEuropean Union regulations on algorithmic decision-making and a 'right to explanation'\u201d. ICML Workshop on Human Interpretability in Machine Learning, New York, NY."},{"key":"e_1_3_2_1_4_1","volume-title":"Bloomberg Businessweek","author":"Kahn J.","year":"2018","unstructured":"Kahn , J. ( 2018 ). \u201c Artificial Intelligence Has Some Explaining to Do \u201d. Bloomberg Businessweek , 12 Dec 2018. Kahn, J. (2018). \u201cArtificial Intelligence Has Some Explaining to Do\u201d. Bloomberg Businessweek, 12 Dec 2018."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157584"},{"key":"e_1_3_2_1_6_1","volume-title":"ProPublica","author":"Angwin J.","year":"2016","unstructured":"Angwin , J. , Larson , J , Mattu , S. , and Kirchner , L . ( 2016 ). \u201c Machine Bias: There's software used across the country to predict future criminals. And it's biased against blacks \u201d. ProPublica , 23 May 2016. Angwin, J., Larson, J, Mattu, S., and Kirchner, L. (2016). \u201cMachine Bias: There's software used across the country to predict future criminals. And it's biased against blacks\u201d. ProPublica, 23 May 2016."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of Machine Learning Research.","author":"Ensign D.","year":"2018","unstructured":"Ensign , D. , Friedler , S.A. , Neville , S. , Scheidegger , C. and Venkatasubramanian , S . ( 2018 ). \u201c Runaway Feedback Loops in Predictive Policing \u201d. Proceedings of Machine Learning Research. Ensign, D., Friedler, S.A., Neville, S., Scheidegger, C. and Venkatasubramanian, S. (2018). \u201cRunaway Feedback Loops in Predictive Policing\u201d. Proceedings of Machine Learning Research."},{"key":"e_1_3_2_1_8_1","article-title":"This is How AI Bias Really Happens -\u2014 and Why It's so Hard to Fix","author":"Hao K.","year":"2019","unstructured":"Hao , K. ( 2019 ). \u201c This is How AI Bias Really Happens -\u2014 and Why It's so Hard to Fix \u201d . MIT Technology Review , 4 Feb 2019. Hao, K. (2019). \u201cThis is How AI Bias Really Happens -\u2014 and Why It's so Hard to Fix\u201d. MIT Technology Review, 4 Feb 2019.","journal-title":". MIT Technology Review"},{"key":"e_1_3_2_1_9_1","volume-title":"Amazon scraps secret AI recruiting tool that showed bias against women","author":"Dastin J.","year":"2018","unstructured":"Dastin , J. ( 2018 ). \u201c Amazon scraps secret AI recruiting tool that showed bias against women \u201d. Reuters Business News , 10 Oct 2018. Dastin, J. (2018). \u201cAmazon scraps secret AI recruiting tool that showed bias against women\u201d. Reuters Business News, 10 Oct 2018."},{"key":"e_1_3_2_1_10_1","volume-title":"Uber Engineering","author":"Del Balso M.","year":"2017","unstructured":"Del Balso , M. and Hermann , J . ( 2017 ). \u201c Meet Michelangelo: Uber's Machine Learning Platform \u201d. Uber Engineering , 5 Sep 2017. Del Balso, M. and Hermann, J. (2017). \u201cMeet Michelangelo: Uber's Machine Learning Platform\u201d. Uber Engineering, 5 Sep 2017."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098021"},{"key":"e_1_3_2_1_12_1","volume-title":"The Verge","author":"Vincent J.","year":"2016","unstructured":"Vincent , J. ( 2016 ). \u201c Twitter taught Microsoft's AI chatbot to be a racist asshole in less than a day \u201d. The Verge , 24 Mar 2016. Vincent, J. (2016). \u201cTwitter taught Microsoft's AI chatbot to be a racist asshole in less than a day\u201d. The Verge, 24 Mar 2016."},{"key":"e_1_3_2_1_13_1","volume-title":"Attacks against machine learning - an overview\u201d. https:\/\/elie.net\/blog\/ai\/attacks-against-machine-learning-an-overview\/","author":"Bursztein E.","year":"2018","unstructured":"Bursztein , E. ( 2018 ). \u201c Attacks against machine learning - an overview\u201d. https:\/\/elie.net\/blog\/ai\/attacks-against-machine-learning-an-overview\/ . Bursztein, E. (2018). \u201cAttacks against machine learning - an overview\u201d. https:\/\/elie.net\/blog\/ai\/attacks-against-machine-learning-an-overview\/."},{"key":"e_1_3_2_1_14_1","volume-title":"TIME","author":"Buolamwini J.","year":"2019","unstructured":"Buolamwini , J. ( 2019 ). \u201c Aritificial Intelligence Has a Problem with Gender and Racial Bias \u201d. TIME , 7 Feb 2019. Buolamwini, J. (2019). \u201cAritificial Intelligence Has a Problem with Gender and Racial Bias\u201d. TIME, 7 Feb 2019."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_16_1","unstructured":"Google. \"TensorFlow Hub \" https:\/\/www.tensorflow.org\/hub.  Google. \"TensorFlow Hub \" https:\/\/www.tensorflow.org\/hub."},{"key":"e_1_3_2_1_17_1","unstructured":"H2O. \u201cOpen Source Leader in AI and ML\u201d. h2o https:\/\/www.h2o.ai\/.  H2O. \u201cOpen Source Leader in AI and ML\u201d. h2o https:\/\/www.h2o.ai\/."},{"key":"e_1_3_2_1_18_1","unstructured":"European Commission. \u201cRights for Citizens - European Commission\u201d. https:\/\/ec.europa.eu\/info\/law\/law-topic\/data-protection\/reform\/rights-citizens_en.  European Commission. \u201cRights for Citizens - European Commission\u201d. https:\/\/ec.europa.eu\/info\/law\/law-topic\/data-protection\/reform\/rights-citizens_en."},{"key":"e_1_3_2_1_19_1","volume-title":"The Fed - Supervisory Letter SR 11-7 on Guidance on Model Risk Management\u201d. https:\/\/www.federalreserve.gov\/supervisionreg\/srletters\/sr1107.htm","author":"Board","year":"2011","unstructured":"Board of Governors of the Federal Reserve System ( 2011 ). \u201c The Fed - Supervisory Letter SR 11-7 on Guidance on Model Risk Management\u201d. https:\/\/www.federalreserve.gov\/supervisionreg\/srletters\/sr1107.htm . Board of Governors of the Federal Reserve System (2011). \u201cThe Fed - Supervisory Letter SR 11-7 on Guidance on Model Risk Management\u201d. https:\/\/www.federalreserve.gov\/supervisionreg\/srletters\/sr1107.htm."},{"key":"e_1_3_2_1_20_1","volume-title":"USENIX Security Symposium","author":"Suciu O.","year":"2018","unstructured":"Suciu , O. , Marginean , R. , Kaya , Y. , Daume . H. III , and Dumitras , T . ( 2018 ). \u201c When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks\u201d.27th USENIX Security Symposium , Baltimore, MD. Suciu, O., Marginean, R., Kaya, Y., Daume. H. III, and Dumitras, T. (2018). \u201cWhen Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks\u201d.27th USENIX Security Symposium, Baltimore, MD."},{"key":"e_1_3_2_1_21_1","unstructured":"IBM. \u201cAI Fairness 360\u201d. http:\/\/aif360.mybluemix.net\/.  IBM. \u201cAI Fairness 360\u201d. http:\/\/aif360.mybluemix.net\/."},{"key":"e_1_3_2_1_22_1","unstructured":"H2O. \u201cInterpreting a Model \u2014 Using Driverless AI\u201d. https:\/\/www.h2o.ai\/wp-content\/uploads\/2017\/09\/driverlessai\/interpreting.html.  H2O. \u201cInterpreting a Model \u2014 Using Driverless AI\u201d. https:\/\/www.h2o.ai\/wp-content\/uploads\/2017\/09\/driverlessai\/interpreting.html."},{"key":"e_1_3_2_1_23_1","volume-title":"ITOpsTimes","author":"Ghanta S.","year":"2018","unstructured":"Ghanta , S. and Talagala N . ( 2018 ). \u201c MLOps Health: Taking the Pulse of ML in Production \u201d. ITOpsTimes , 10 July 2018. Ghanta, S. and Talagala N. (2018). \u201cMLOps Health: Taking the Pulse of ML in Production\u201d. ITOpsTimes, 10 July 2018."},{"key":"e_1_3_2_1_24_1","volume-title":"Conference on Neural Information Processing Systems","author":"Chen I.Y.","year":"2018","unstructured":"Chen , I.Y. , Johansson , F.D. , and Sontag , D . ( 2018 ). \u201c Why Is My Classifier Discriminatory?\u201d.32nd Conference on Neural Information Processing Systems , Montreal, Canada. Chen, I.Y., Johansson, F.D., and Sontag, D. (2018). \u201cWhy Is My Classifier Discriminatory?\u201d.32nd Conference on Neural Information Processing Systems, Montreal, Canada."},{"key":"e_1_3_2_1_25_1","first-page":"633","article-title":"Accountable Algorithms","volume":"165","author":"Kroll J.","year":"2017","unstructured":"Kroll , J. , Huey , J. , Barocas , S , Felten , E. , Reidenberg , J. , Robinson , D. , and Yu , H. ( 2017 ). \u201c Accountable Algorithms \u201d. University of Pennsylvania Law Review , vol. 165 , pp. 633 - 705 . Kroll, J., Huey, J., Barocas, S, Felten, E., Reidenberg, J., Robinson, D., and Yu, H. (2017). \u201cAccountable Algorithms\u201d. University of Pennsylvania Law Review, vol. 165, pp. 633-705.","journal-title":"University of Pennsylvania Law Review"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278156"},{"key":"e_1_3_2_1_27_1","volume-title":"E-Verify Has Delayed or Cost Half a Million Jobs for Legal Workers","author":"Bier D.","year":"2017","unstructured":"Bier , D. ( 2017 ). \u201c E-Verify Has Delayed or Cost Half a Million Jobs for Legal Workers \u201d. CATO Institute , 16 May 2017. https:\/\/www.cato.org\/blog\/e-verify-has-held-or-cost-jobs-half-million-legal-workers. Bier, D. (2017). \u201cE-Verify Has Delayed or Cost Half a Million Jobs for Legal Workers\u201d. CATO Institute, 16 May 2017. https:\/\/www.cato.org\/blog\/e-verify-has-held-or-cost-jobs-half-million-legal-workers."},{"key":"e_1_3_2_1_28_1","volume-title":"Bloomberg","author":"Suhartono H.","year":"2018","unstructured":"Suhartono , H. , Levin , A. , and Johnsson , J . ( 2018 ). \u201c Why Did Lion Air Flight 610 Crash? New Report Details Struggle \u201d. Bloomberg , 27 Nov 2018. https:\/\/www.bloomberg.com\/news\/articles\/2018-11-27\/lion-air-pilots-struggle-detailed-in-preliminary-crash-report. Suhartono, H., Levin, A., and Johnsson, J. (2018). \u201cWhy Did Lion Air Flight 610 Crash? New Report Details Struggle\u201d. Bloomberg, 27 Nov 2018. https:\/\/www.bloomberg.com\/news\/articles\/2018-11-27\/lion-air-pilots-struggle-detailed-in-preliminary-crash-report."},{"key":"e_1_3_2_1_29_1","volume-title":"Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy","author":"Neil C.","year":"2016","unstructured":"0' Neil , C. ( 2016 ). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy . Crown . New York, NY . 0'Neil, C. (2016). Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown. New York, NY."},{"key":"e_1_3_2_1_30_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency","author":"Buolamwini J.","year":"2018","unstructured":"Buolamwini , J. and Gebru , T . ( 2018 ). \u201c Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification \u201d. Proceedings of the 1st Conference on Fairness, Accountability and Transparency , New York, NY. Buolamwini, J. and Gebru, T. (2018). \u201cGender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification\u201d. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, New York, NY."},{"key":"e_1_3_2_1_31_1","volume-title":"The Guardian","author":"Buranyi S.","year":"2017","unstructured":"Buranyi , S. ( 2017 ). \u201c Rise of the racist robots - how AI is learning all our worse impulses \u201d. The Guardian , 8 Aug 2017. https:\/\/www.theguardian.com\/inequality\/2017\/aug\/08\/rise-of-the-racist-robots-how-ai-is-learning-all-our-worst-impulses. Buranyi, S. (2017). \u201cRise of the racist robots - how AI is learning all our worse impulses\u201d. The Guardian, 8 Aug 2017. https:\/\/www.theguardian.com\/inequality\/2017\/aug\/08\/rise-of-the-racist-robots-how-ai-is-learning-all-our-worst-impulses."},{"key":"e_1_3_2_1_32_1","volume-title":"MIT Technology Review","author":"Nicholas Diakopoulos","year":"2016","unstructured":"Nicholas Diakopoulos and Sorelle Friedler ( 2016 ). How to Hold Algorithms Accountable , MIT Technology Review , November 17, 2016. https:\/\/www.technologyreview.com\/s\/602933\/how-to-hold-algorithms-accountable\/ Nicholas Diakopoulos and Sorelle Friedler (2016). How to Hold Algorithms Accountable, MIT Technology Review, November 17, 2016. https:\/\/www.technologyreview.com\/s\/602933\/how-to-hold-algorithms-accountable\/"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3125780"},{"key":"e_1_3_2_1_34_1","first-page":"2017","volume":"12","author":"ACM US Public Policy Council (USACM)","year":"2017","unstructured":"ACM US Public Policy Council (USACM) ( 2017 ). Statement on Algorithmic Transparency and Accountability . January 12 , 2017 . https:\/\/www.acm.org\/binaries\/content\/assets\/public-policy\/2017_usacm_statement_algorithms.pdf ACM US Public Policy Council (USACM) (2017). Statement on Algorithmic Transparency and Accountability. January 12, 2017. https:\/\/www.acm.org\/binaries\/content\/assets\/public-policy\/2017_usacm_statement_algorithms.pdf","journal-title":"Statement on Algorithmic Transparency and Accountability"},{"key":"e_1_3_2_1_35_1","volume-title":"Algorithmic accountability report","author":"World Wide Web Foundation","year":"2017","unstructured":"World Wide Web Foundation ( 2017 ). Algorithmic accountability report , 2017. https:\/\/webfoundation.org\/docs\/2017\/07\/Algorithms_Report_WF.pdf World Wide Web Foundation (2017). Algorithmic accountability report, 2017. https:\/\/webfoundation.org\/docs\/2017\/07\/Algorithms_Report_WF.pdf"},{"key":"e_1_3_2_1_36_1","volume-title":"Algorithmic impact assessments: A practical framework for public agency accountability. AI Now institute","author":"Reisman D.","year":"2018","unstructured":"Reisman , D. , Schultz , J. , Crawford , K. , and Whittaker , M . ( 2018 ). Algorithmic impact assessments: A practical framework for public agency accountability. AI Now institute . Apr. 2018. https:\/\/ainowinstitute.org\/aiareport2018.pdf Reisman, D., Schultz, J., Crawford, K., and Whittaker, M. (2018). Algorithmic impact assessments: A practical framework for public agency accountability. AI Now institute. Apr. 2018. https:\/\/ainowinstitute.org\/aiareport2018.pdf"},{"key":"e_1_3_2_1_37_1","volume-title":"Algorithmic accountability primer","author":"Caplan R.","year":"2018","unstructured":"Caplan , R. , Donovan , J. , Hanson , L. , and Matthews , J . ( 2018 ). Algorithmic accountability primer . Data & Society . Apr. 2018. https:\/\/datasociety.net\/wp-content\/uploads\/2018\/04\/Data_Society_Algorithmic_Accountability_Primer_FINAL-4.pdf Caplan, R., Donovan, J., Hanson, L., and Matthews, J. (2018). Algorithmic accountability primer. Data & Society. Apr. 2018. https:\/\/datasociety.net\/wp-content\/uploads\/2018\/04\/Data_Society_Algorithmic_Accountability_Primer_FINAL-4.pdf"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/230538.230561"},{"key":"e_1_3_2_1_39_1","volume-title":"Social data: Biases, methodological pitfalls, and ethical boundaries","author":"Olteanu A.","year":"2016","unstructured":"Olteanu , A. , Castillo , C , Diaz , F. , and Kiciman , E . ( 2016 ). Social data: Biases, methodological pitfalls, and ethical boundaries . Olteanu, A., Castillo, C, Diaz, F., and Kiciman, E. (2016). Social data: Biases, methodological pitfalls, and ethical boundaries."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908131.2908135"}],"event":{"name":"WWW '19: The Web Conference","location":"San Francisco USA","acronym":"WWW '19","sponsor":["IW3C2 International World Wide Web Conference Committee"]},"container-title":["Companion Proceedings of The 2019 World Wide Web Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308560.3317590","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3308560.3317590","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:35Z","timestamp":1750204415000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308560.3317590"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,13]]},"references-count":40,"alternative-id":["10.1145\/3308560.3317590","10.1145\/3308560"],"URL":"https:\/\/doi.org\/10.1145\/3308560.3317590","relation":{},"subject":[],"published":{"date-parts":[[2019,5,13]]},"assertion":[{"value":"2019-05-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}