{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T17:12:48Z","timestamp":1784308368939,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":70,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,1,29]],"date-time":"2019-01-29T00:00:00Z","timestamp":1548720000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,1,29]]},"DOI":"10.1145\/3287560.3287600","type":"proceedings-article","created":{"date-parts":[[2019,1,9]],"date-time":"2019-01-09T21:42:15Z","timestamp":1547070135000},"page":"49-58","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":266,"title":["50 Years of Test (Un)fairness"],"prefix":"10.1145","author":[{"given":"Ben","family":"Hutchinson","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Margaret","family":"Mitchell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,1,29]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Psychological tests: Uses and abuses","author":"Anastasi Anne","year":"1961"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0023906"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-842X.2004.00360.x"},{"key":"e_1_3_2_1_4_1","unstructured":"Solon Barocas Moritz Hardt and Arvind Naranayan. 2018. Fairness in Machine Learning. http:\/\/fairmlbook.org. (2018).  Solon Barocas Moritz Hardt and Arvind Naranayan. 2018. Fairness in Machine Learning. http:\/\/fairmlbook.org. (2018)."},{"key":"e_1_3_2_1_5_1","volume-title":"Fairness in criminal justice risk assessments: the state of the art. arXiv preprint arXiv:1703.09207","author":"Berk Richard","year":"2017"},{"key":"e_1_3_2_1_6_1","volume-title":"Chi","author":"Beutel Alex","year":"2017"},{"key":"e_1_3_2_1_7_1","volume-title":"Conference on Fairness, Accountability and Transparency. 77--91","author":"Buolamwini Joy","year":"2018"},{"key":"e_1_3_2_1_8_1","volume-title":"Ranking with fairness constraints. arXiv preprint arXiv:1704.06840","author":"Celis L Elisa","year":"2017"},{"key":"e_1_3_2_1_9_1","volume-title":"Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5, 2","author":"Chouldechova Alexandra","year":"2017"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2333-8504.1966.tb00541.x"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1968.tb00613.x"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446802800106"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1111\/joop.12023"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1973.tb00802.x"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.2001.tb01132.x"},{"key":"e_1_3_2_1_16_1","unstructured":"Sam Corbett-Davies Emma Pierson Avi Feller and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. Its actually not that clear. https:\/www.washingtonpost.com\/news\/monkey-cage\/wp\/2016\/10\/17\/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas\/.(2016).  Sam Corbett-Davies Emma Pierson Avi Feller and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. Its actually not that clear. https:\/www.washingtonpost.com\/news\/monkey-cage\/wp\/2016\/10\/17\/can-an-algorithm-be-racist-our-analysis-is-more-cautious-than-propublicas\/.(2016)."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_3_2_1_18_1","unstructured":"National Research Council et al. 1989. Fairness in employment testing: Validity generalization minority issues and the General Aptitude Test Battery. National Academies Press.  National Research Council et al. 1989. Fairness in employment testing: Validity generalization minority issues and the General Aptitude Test Battery. National Academies Press."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1971.tb00908.x"},{"key":"e_1_3_2_1_20_1","unstructured":"William Dieterich Christina Mendoza and Tim Brennan. 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. http:\/\/go.volarisgroup.com\/rs\/430-MBX-989\/images\/ProPublica_Commentary_Final_070616.pdf. (2016).  William Dieterich Christina Mendoza and Tim Brennan. 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. http:\/\/go.volarisgroup.com\/rs\/430-MBX-989\/images\/ProPublica_Commentary_Final_070616.pdf. (2016)."},{"key":"e_1_3_2_1_21_1","volume-title":"Advancing Human Assessment","author":"Dorans Neil J"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/j.2333-8504.1992.tb01440.x"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0030871"},{"key":"e_1_3_2_1_25_1","volume-title":"Runaway feedback loops in predictive policing. arXiv preprint arXiv: 1706.09847","author":"Ensign Danielle","year":"2017"},{"key":"e_1_3_2_1_27_1","volume-title":"Foulds and Shimei Pan","author":"James","year":"2018"},{"key":"e_1_3_2_1_28_1","volume-title":"On the (im) possibility of fairness. arXiv preprint arXiv: 1609.07236","author":"Friedler Sorelle A","year":"2016"},{"key":"e_1_3_2_1_29_1","unstructured":"Gabriel Goh Andrew Cotter Maya Gupta and Michael P Friedlander. 2016. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems. 2415--2423.   Gabriel Goh Andrew Cotter Maya Gupta and Michael P Friedlander. 2016. Satisfying real-world goals with dataset constraints. In Advances in Neural Information Processing Systems. 2415--2423."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-232X.1966.tb00449.x"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173582"},{"key":"e_1_3_2_1_32_1","volume-title":"Advances in Neural Information Processing Systems 29","author":"Hardt Moritz"},{"key":"e_1_3_2_1_33_1","volume-title":"Spaces for the Future","author":"Hoffmann Anna Lauren"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.83.6.1053"},{"key":"e_1_3_2_1_35_1","volume-title":"Racial bias in testing. The Black-White test score gap 55","author":"Jencks Christopher","year":"1998"},{"key":"e_1_3_2_1_36_1","unstructured":"Arthur R Jensen. 1980. Bias in mental testing. (1980).  Arthur R Jensen. 1980. Bias in mental testing. (1980)."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316447303300306"},{"key":"e_1_3_2_1_38_1","unstructured":"Jerome Karabel. 2006. The chosen: The hidden history of admission and exclusion at Harvard Yale and Princeton. Houghton Mifflin Harcourt.  Jerome Karabel. 2006. The chosen: The hidden history of admission and exclusion at Harvard Yale and Princeton. Houghton Mifflin Harcourt."},{"key":"e_1_3_2_1_39_1","unstructured":"Michael Kearns Seth Neel Aaron Roth and Zhiwei Steven Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In ICML.  Michael Kearns Seth Neel Aaron Roth and Zhiwei Steven Wu. 2018. Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. In ICML."},{"key":"e_1_3_2_1_40_1","volume-title":"Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807","author":"Kleinberg Jon","year":"2016"},{"key":"e_1_3_2_1_41_1","unstructured":"Matt J Kusner Joshua Loftus Chris Russell and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems. 4066--4076.   Matt J Kusner Joshua Loftus Chris Russell and Ricardo Silva. 2017. Counterfactual fairness. In Advances in Neural Information Processing Systems. 4066--4076."},{"key":"e_1_3_2_1_42_1","unstructured":"Jeff Larson Surya Mau Lauren Kirchner and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. https:\/www.propublica.org\/article\/how-we-analyzed-the-compas-recidivism-algorithm. (2016).  Jeff Larson Surya Mau Lauren Kirchner and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. https:\/www.propublica.org\/article\/how-we-analyzed-the-compas-recidivism-algorithm. (2016)."},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.3102\/00346543043002139"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1976.tb00181.x"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273571"},{"key":"e_1_3_2_1_46_1","volume-title":"Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions. arXiv:1811.07867","author":"Mitchell Shira","year":"2018"},{"key":"e_1_3_2_1_47_1","unstructured":"National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976).  National Council on Measurement in Education NCME (Ed.). 1976. Journal of Education Measurement. 13 1 (1976)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1976.tb00183.x"},{"key":"e_1_3_2_1_49_1","volume-title":"Weapons of math destruction: How big data increases inequality and threatens democracy","author":"O'Neil Cathy"},{"key":"e_1_3_2_1_50_1","volume-title":"Fairness in Educational Assessment and Measurement","author":"Penfield Randall D"},{"key":"e_1_3_2_1_51_1","first-page":"333","article-title":"An expected utility model for \"optimal\" selection","volume":"1","author":"Petersen Nancy S","year":"1976","journal-title":"Journal of Educational Statistics"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1976.tb00178.x"},{"key":"e_1_3_2_1_53_1","volume-title":"Fairness in Educational Assessment and Measurement","author":"Phillips S E"},{"key":"e_1_3_2_1_54_1","first-page":"451","article-title":"Race Norming, Validity Generalization, and Employment Testing","volume":"58","author":"Rice Mitchell F","year":"1994","journal-title":"Handbook of Public Personnel Administration"},{"key":"e_1_3_2_1_55_1","volume-title":"InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity. In Workshop on Fairness, Accountability and Transparency in Machine Learning.","author":"Ryu Hee Jung","year":"2018"},{"key":"e_1_3_2_1_56_1","volume-title":"Psychological testing of American minorities: Issues and consequences","author":"Samuda Ronald J"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1976.tb00182.x"},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1979.tb00095.x"},{"key":"e_1_3_2_1_59_1","volume-title":"The Hardness of Conditional Independence Testing and the Generalised Covariance Measure. arXiv preprint arXiv:1804.07203","author":"Shah Rajen D","year":"2018"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1214\/17-AOAS1058"},{"key":"e_1_3_2_1_61_1","unstructured":"Charles L Thomas. 1973. The Overprediction Phenomenon among Black Collegians: Some Prelinimary Considerations. (1973).  Charles L Thomas. 1973. The Overprediction Phenomenon among Black Collegians: Some Prelinimary Considerations. (1973)."},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1971.tb00907.x"},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.3102\/10769986021003264"},{"key":"e_1_3_2_1_64_1","volume-title":"Scholastic aptitude test scores, race, and academic performance in selective colleges and universities. The Black-White test score gap","author":"Vars Frederick E","year":"1998"},{"key":"e_1_3_2_1_65_1","first-page":"1035","article-title":"Fairness Feuds: Competing Conceptions of Title VII Discriminatory Testing","volume":"46","author":"West-Faulcon Kimberly","year":"2011","journal-title":"Wake Forest L. Rev."},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.2307\/2295085"},{"key":"e_1_3_2_1_67_1","doi-asserted-by":"crossref","volume-title":"Gender and fair assessment","author":"Willingham Warren W","DOI":"10.4324\/9781315045115"},{"key":"e_1_3_2_1_68_1","unstructured":"Muhammad Bilal Zafar Isabel Valera Manuel Rodriguez Krishna Gummadi and Adrian Weller. 2017. From parity to preference-based notions of fairness in classification. In Advances in Neural Information Processing Systems. 229--239.  Muhammad Bilal Zafar Isabel Valera Manuel Rodriguez Krishna Gummadi and Adrian Weller. 2017. From parity to preference-based notions of fairness in classification. In Advances in Neural Information Processing Systems. 229--239."},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"crossref","unstructured":"Brian Hu Zhang Blake Lemoine and Margaret Mitchell. 2018. Mitigating Unwanted Biases with Adversarial Learning. (2018).  Brian Hu Zhang Blake Lemoine and Margaret Mitchell. 2018. Mitigating Unwanted Biases with Adversarial Learning. (2018).","DOI":"10.1145\/3278721.3278779"},{"key":"e_1_3_2_1_70_1","volume-title":"Proceedings of the 20th International conference on Machine learning (ICML-03)","author":"Zhu Xiaojin","year":"2003"},{"key":"e_1_3_2_1_71_1","volume-title":"Fairness in Educational Assessment and Measurement","author":"Zwick Rebecca"}],"event":{"name":"FAT* '19: Conference on Fairness, Accountability, and Transparency","location":"Atlanta GA USA","acronym":"FAT* '19","sponsor":["ACM Association for Computing Machinery"]},"container-title":["Proceedings of the Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3287560.3287600","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3287560.3287600","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:08:04Z","timestamp":1750208884000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3287560.3287600"}},"subtitle":["Lessons for Machine Learning"],"short-title":[],"issued":{"date-parts":[[2019,1,29]]},"references-count":70,"alternative-id":["10.1145\/3287560.3287600","10.1145\/3287560"],"URL":"https:\/\/doi.org\/10.1145\/3287560.3287600","relation":{},"subject":[],"published":{"date-parts":[[2019,1,29]]},"assertion":[{"value":"2019-01-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}