{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T01:08:56Z","timestamp":1777770536112,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,5]],"date-time":"2021-10-05T00:00:00Z","timestamp":1633392000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,5]]},"DOI":"10.1145\/3465416.3483305","type":"proceedings-article","created":{"date-parts":[[2021,11,4]],"date-time":"2021-11-04T22:07:16Z","timestamp":1636063636000},"page":"1-9","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":346,"title":["A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle"],"prefix":"10.1145","author":[{"given":"Harini","family":"Suresh","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, USA"}]},{"given":"John","family":"Guttag","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,11,4]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Julia Angwin Jeff Larson Surya Mattu and Lauren Kirchner. 2016. Machine bias. ProPublica May 23(2016).  Julia Angwin Jeff Larson Surya Mattu and Lauren Kirchner. 2016. Machine bias. ProPublica May 23(2016)."},{"key":"e_1_3_2_1_2_1","volume-title":"The NeurIPS\u201918 Competition","author":"Atwood James","unstructured":"James Atwood , Yoni Halpern , Pallavi Baljekar , Eric Breck , D Sculley , Pavel Ostyakov , Sergey\u00a0 I Nikolenko , Igor Ivanov , Roman Solovyev , Weimin Wang , 2020. The Inclusive Images Competition . In The NeurIPS\u201918 Competition . Springer , 155\u2013186. James Atwood, Yoni Halpern, Pallavi Baljekar, Eric Breck, D Sculley, Pavel Ostyakov, Sergey\u00a0I Nikolenko, Igor Ivanov, Roman Solovyev, Weimin Wang, 2020. The Inclusive Images Competition. In The NeurIPS\u201918 Competition. Springer, 155\u2013186."},{"key":"e_1_3_2_1_3_1","first-page":"15479","article-title":"Differential privacy has disparate impact on model accuracy","volume":"32","author":"Bagdasaryan Eugene","year":"2019","unstructured":"Eugene Bagdasaryan , Omid Poursaeed , and Vitaly Shmatikov . 2019 . Differential privacy has disparate impact on model accuracy . Advances in Neural Information Processing Systems 32 (2019), 15479 \u2013 15488 . Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019. Differential privacy has disparate impact on model accuracy. Advances in Neural Information Processing Systems 32 (2019), 15479\u201315488.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_4_1","volume-title":"The problem with bias: from allocative to representational harms in machine learning","author":"Barocas Solon","year":"2017","unstructured":"Solon Barocas , Kate Crawford , Aaron Shapiro , and Hanna Wallach . 2017. The problem with bias: from allocative to representational harms in machine learning . Special Interest Group for Computing , Information and Society (SIGCIS) ( 2017 ). Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: from allocative to representational harms in machine learning. Special Interest Group for Computing, Information and Society (SIGCIS) (2017)."},{"key":"e_1_3_2_1_5_1","first-page":"671","article-title":"Big data\u2019s disparate impact","volume":"104","author":"Barocas Solon","year":"2016","unstructured":"Solon Barocas and Andrew\u00a0 D Selbst . 2016 . Big data\u2019s disparate impact . Cal. L. Rev. 104 (2016), 671 . Solon Barocas and Andrew\u00a0D Selbst. 2016. Big data\u2019s disparate impact. Cal. L. Rev. 104(2016), 671.","journal-title":"Cal. L. Rev."},{"key":"e_1_3_2_1_6_1","volume-title":"Conference on Fairness, Accountability and Transparency. 77\u201391","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on Fairness, Accountability and Transparency. 77\u201391 . Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. 77\u201391."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00289259"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_3_2_1_9_1","unstructured":"Irene Chen Fredrik\u00a0D Johansson and David Sontag. 2018. Why Is My Classifier Discriminatory?. In Advances in Neural Information Processing Systems. 3539\u20133550.  Irene Chen Fredrik\u00a0D Johansson and David Sontag. 2018. Why Is My Classifier Discriminatory?. In Advances in Neural Information Processing Systems. 3539\u20133550."},{"key":"e_1_3_2_1_10_1","first-page":"57","article-title":"Punishing Risk. Geo","volume":"107","author":"Collins Erin","year":"2018","unstructured":"Erin Collins . 2018 . Punishing Risk. Geo . LJ 107 (2018), 57 . Erin Collins. 2018. Punishing Risk. Geo. LJ 107(2018), 57.","journal-title":"LJ"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219166.3277556"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287572"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"J. Deng W. Dong R. Socher L.-J. Li K. Li and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.  J. Deng W. Dong R. Socher L.-J. Li K. Li and L. Fei-Fei. 2009. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_1_14_1","volume-title":"Hilary Nicole, and Morgan\u00a0Klaus Scheuerman.","author":"Denton Emily","year":"2020","unstructured":"Emily Denton , Alex Hanna , Razvan Amironesei , Andrew Smart , Hilary Nicole, and Morgan\u00a0Klaus Scheuerman. 2020 . Bringing the People Back In: Contesting Benchmark Machine Learning Datasets . arxiv:2007.07399\u00a0[cs.CY] Emily Denton, Alex Hanna, Razvan Amironesei, Andrew Smart, Hilary Nicole, and Morgan\u00a0Klaus Scheuerman. 2020. Bringing the People Back In: Contesting Benchmark Machine Learning Datasets. arxiv:2007.07399\u00a0[cs.CY]"},{"key":"e_1_3_2_1_15_1","volume-title":"The accuracy, fairness, and limits of predicting recidivism. Science advances 4, 1","author":"Dressel Julia","year":"2018","unstructured":"Julia Dressel and Hany Farid . 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances 4, 1 ( 2018 ), eaao5580. Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances 4, 1 (2018), eaao5580."},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081)","author":"Dwork Cynthia","year":"2018","unstructured":"Cynthia Dwork , Nicole Immorlica , Adam\u00a0Tauman Kalai , and Max Leiserson . 2018 . Decoupled Classifiers for Group-Fair and Efficient Machine Learning . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081) , Sorelle\u00a0A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 119\u2013133. http:\/\/proceedings.mlr.press\/v81\/dwork18a.html Cynthia Dwork, Nicole Immorlica, Adam\u00a0Tauman Kalai, and Max Leiserson. 2018. Decoupled Classifiers for Group-Fair and Efficient Machine Learning. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081), Sorelle\u00a0A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 119\u2013133. http:\/\/proceedings.mlr.press\/v81\/dwork18a.html"},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081)","author":"Ensign Danielle","year":"2018","unstructured":"Danielle Ensign , Sorelle\u00a0 A. Friedler , Scott Neville , Carlos Scheidegger , and Suresh Venkatasubramanian . 2018 . Runaway Feedback Loops in Predictive Policing . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081) , Sorelle\u00a0A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 160\u2013171. http:\/\/proceedings.mlr.press\/v81\/ensign18a.html Danielle Ensign, Sorelle\u00a0A. Friedler, Scott Neville, Carlos Scheidegger, and Suresh Venkatasubramanian. 2018. Runaway Feedback Loops in Predictive Policing. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol.\u00a081), Sorelle\u00a0A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 160\u2013171. http:\/\/proceedings.mlr.press\/v81\/ensign18a.html"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445912"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1177\/0894439318788314"},{"key":"e_1_3_2_1_20_1","volume-title":"ACM Conference on Fairness, Accountability and Transparency (FAT*). ACM. http:\/\/arxiv.org\/abs\/1802","year":"2019","unstructured":"Friedler, Scheidegger, Venkatasubramanian, Choudhary, Hamilton, and Roth. 2019 . A comparative study of fairness-enhancing interventions in machine learning . In ACM Conference on Fairness, Accountability and Transparency (FAT*). ACM. http:\/\/arxiv.org\/abs\/1802 .04422 Friedler, Scheidegger, Venkatasubramanian, Choudhary, Hamilton, and Roth. 2019. A comparative study of fairness-enhancing interventions in machine learning. In ACM Conference on Fairness, Accountability and Transparency (FAT*). ACM. http:\/\/arxiv.org\/abs\/1802.04422"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/230538.230561"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1720347115"},{"key":"e_1_3_2_1_23_1","volume-title":"Risk Assessment: Explained. The Marshall Project 25(2019).","author":"Henry Matt","year":"2019","unstructured":"Matt Henry . 2019 . Risk Assessment: Explained. The Marshall Project 25(2019). Matt Henry. 2019. Risk Assessment: Explained. The Marshall Project 25(2019)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1516047113"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1748-720X.2001.tb00037.x"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2021.100241"},{"key":"e_1_3_2_1_27_1","unstructured":"Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio and Emily Denton. 2020. Characterising Bias in Compressed Models. https:\/\/arxiv.org\/abs\/2010.03058  Sara Hooker Nyalleng Moorosi Gregory Clark Samy Bengio and Emily Denton. 2020. Characterising Bias in Compressed Models. https:\/\/arxiv.org\/abs\/2010.03058"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445923"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00159"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1257\/pandp.20181018"},{"key":"e_1_3_2_1_31_1","volume-title":"8th Innovations in Theoretical Computer Science Conference (ITCS 2017)(Leibniz International Proceedings in Informatics (LIPIcs), Vol.\u00a067), Christos\u00a0H","author":"Kleinberg Jon","year":"2017","unstructured":"Jon Kleinberg , Sendhil Mullainathan , and Manish Raghavan . 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores . In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017)(Leibniz International Proceedings in Informatics (LIPIcs), Vol.\u00a067), Christos\u00a0H . Papadimitriou (Ed.). Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany , 43:1\u201343:23. https:\/\/doi.org\/10.4230\/LIPIcs.ITCS. 2017 .43 Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent Trade-Offs in the Fair Determination of Risk Scores. In 8th Innovations in Theoretical Computer Science Conference (ITCS 2017)(Leibniz International Proceedings in Informatics (LIPIcs), Vol.\u00a067), Christos\u00a0H. Papadimitriou (Ed.). Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 43:1\u201343:23. https:\/\/doi.org\/10.4230\/LIPIcs.ITCS.2017.43"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/10691898.2018.1434342"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_3_2_1_35_1","volume-title":"Defining racial and ethnic disparities in pain management. Clinical Orthopaedics and Related Research\u00ae 469, 7(2011)","author":"Mossey M","year":"1859","unstructured":"Jana\u00a0 M Mossey . 2011. Defining racial and ethnic disparities in pain management. Clinical Orthopaedics and Related Research\u00ae 469, 7(2011) , 1859 \u20131870. Jana\u00a0M Mossey. 2011. Defining racial and ethnic disparities in pain management. Clinical Orthopaedics and Related Research\u00ae 469, 7(2011), 1859\u20131870."},{"key":"e_1_3_2_1_36_1","volume-title":"21 fairness definitions and their politics","author":"Narayanan Arvind","year":"2018","unstructured":"Arvind Narayanan . 2018. FAT* tutorial : 21 fairness definitions and their politics . New York, NY, USA ( 2018 ). Arvind Narayanan. 2018. FAT* tutorial: 21 fairness definitions and their politics. New York, NY, USA (2018)."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1356"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375841"},{"key":"e_1_3_2_1_39_1","volume-title":"Impact of weight bias and stigma on quality of care and outcomes for patients with obesity. obesity reviews 16, 4","author":"Phelan M","year":"2015","unstructured":"Sean\u00a0 M Phelan , Diane\u00a0 J Burgess , Mark\u00a0 W Yeazel , Wendy\u00a0 L Hellerstedt , Joan\u00a0 M Griffin , and Michelle van Ryn . 2015. Impact of weight bias and stigma on quality of care and outcomes for patients with obesity. obesity reviews 16, 4 ( 2015 ), 319\u2013326. Sean\u00a0M Phelan, Diane\u00a0J Burgess, Mark\u00a0W Yeazel, Wendy\u00a0L Hellerstedt, Joan\u00a0M Griffin, and Michelle van Ryn. 2015. Impact of weight bias and stigma on quality of care and outcomes for patients with obesity. obesity reviews 16, 4 (2015), 319\u2013326."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372828"},{"key":"e_1_3_2_1_41_1","volume-title":"Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT\/ML).","author":"Ryu Hee\u00a0Jung","year":"2018","unstructured":"Hee\u00a0Jung Ryu , Hartwig Adam , and Margaret Mitchell . 2018 . Inclusivefacenet: Improving face attribute detection with race and gender diversity . In Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT\/ML). Hee\u00a0Jung Ryu, Hartwig Adam, and Margaret Mitchell. 2018. Inclusivefacenet: Improving face attribute detection with race and gender diversity. In Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT\/ML)."},{"key":"e_1_3_2_1_42_1","volume-title":"On comparing classifiers: Pitfalls to avoid and a recommended approach. Data mining and knowledge discovery 1, 3","author":"Salzberg L","year":"1997","unstructured":"Steven\u00a0 L Salzberg . 1997. On comparing classifiers: Pitfalls to avoid and a recommended approach. Data mining and knowledge discovery 1, 3 ( 1997 ), 317\u2013328. Steven\u00a0L Salzberg. 1997. On comparing classifiers: Pitfalls to avoid and a recommended approach. Data mining and knowledge discovery 1, 3 (1997), 317\u2013328."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287598"},{"key":"e_1_3_2_1_44_1","volume-title":"NIPS 2017 workshop: Machine Learning for the Developing World.","author":"Shankar Shreya","unstructured":"Shreya Shankar , Yoni Halpern , Eric Breck , James Atwood , Jimbo Wilson , and D. Sculley . 2017. No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World . In NIPS 2017 workshop: Machine Learning for the Developing World. Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, and D. Sculley. 2017. No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World. In NIPS 2017 workshop: Machine Learning for the Developing World."},{"key":"e_1_3_2_1_45_1","first-page":"303","article-title":"Assessing risk assessment in action","volume":"103","author":"Stevenson Megan","year":"2018","unstructured":"Megan Stevenson . 2018 . Assessing risk assessment in action . Minn. L. Rev. 103 (2018), 303 . Megan Stevenson. 2018. Assessing risk assessment in action. Minn. L. Rev. 103(2018), 303.","journal-title":"Minn. L. Rev."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219930"},{"key":"e_1_3_2_1_47_1","volume-title":"International Conference on Machine Learning. 325\u2013333","author":"Zemel Rich","year":"2013","unstructured":"Rich Zemel , Yu Wu , Kevin Swersky , Toni Pitassi , and Cynthia Dwork . 2013 . Learning fair representations . In International Conference on Machine Learning. 325\u2013333 . Rich Zemel, Yu Wu, Kevin Swersky, Toni Pitassi, and Cynthia Dwork. 2013. Learning fair representations. In International Conference on Machine Learning. 325\u2013333."}],"event":{"name":"EAAMO '21: Equity and Access in Algorithms, Mechanisms, and Optimization","location":"-- NY USA","acronym":"EAAMO '21","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","SIGecom Special Interest Group on Economics and Computation"]},"container-title":["Equity and Access in Algorithms, Mechanisms, and Optimization"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3465416.3483305","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3465416.3483305","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:31Z","timestamp":1750191511000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3465416.3483305"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,5]]},"references-count":47,"alternative-id":["10.1145\/3465416.3483305","10.1145\/3465416"],"URL":"https:\/\/doi.org\/10.1145\/3465416.3483305","relation":{},"subject":[],"published":{"date-parts":[[2021,10,5]]},"assertion":[{"value":"2021-11-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}