{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:53:59Z","timestamp":1776113639682,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":55,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,3,1]],"date-time":"2021-03-01T00:00:00Z","timestamp":1614556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,3,3]]},"DOI":"10.1145\/3442188.3445901","type":"proceedings-article","created":{"date-parts":[[2021,2,25]],"date-time":"2021-02-25T01:45:48Z","timestamp":1614217548000},"page":"375-385","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":196,"title":["Measurement and Fairness"],"prefix":"10.1145","author":[{"given":"Abigail Z.","family":"Jacobs","sequence":"first","affiliation":[{"name":"University of Michigan"}]},{"given":"Hanna","family":"Wallach","sequence":"additional","affiliation":[{"name":"Microsoft Research"}]}],"member":"320","published-online":{"date-parts":[[2021,3]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Ken Alder. 2002. The measure of all things: The seven-year odyssey and hidden error that transformed the world.","DOI":"10.1119\/1.1631891"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.3102\/0013189X08316420"},{"key":"e_1_3_2_1_3_1","first-page":"215824402092222","article-title":"Methodological Concerns About the Education Value-Added Assessment System (EVAAS)","volume":"10","author":"Amrein-Beardsley Audrey","year":"2020","unstructured":"Audrey Amrein-Beardsley and Tray Geiger. 2020. Methodological Concerns About the Education Value-Added Assessment System (EVAAS): Validity, Reliability, and Bias. SAGE Open 10, 2 (2020), 2158244020922224.","journal-title":"Validity, Reliability, and Bias. SAGE Open"},{"key":"e_1_3_2_1_4_1","volume-title":"Machine Bias. ProPublica (May","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (May 2016)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/ecoj.12531"},{"key":"e_1_3_2_1_6_1","volume-title":"The problem with bias: from allocative to representational harms in machine learning","author":"Barocas Solon","year":"2017","unstructured":"Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: from allocative to representational harms in machine learning. Special Interest Group for Computing. Information and Society (SIGCIS) (2017)."},{"key":"e_1_3_2_1_7_1","unstructured":"Ruha Benjamin. 2019. Race after Technology: Abolitionist Tools for the New Jim Code."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386383"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","unstructured":"Richard Berk Hoda Heidari Shahin Jabbari Michael Kearns and Aaron Roth. 2018. Fairness in criminal justice risk assessments: the state of the art. Soc. Meth. Res. (2018). https:\/\/doi.org\/10.1177\/0049124118782533","DOI":"10.1177\/0049124118782533"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/380681"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_2_1_12_1","volume-title":"The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810","author":"Chouldechova Alexandra","year":"2018","unstructured":"Alexandra Chouldechova and Aaron Roth. 2018. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810 (2018)."},{"key":"e_1_3_2_1_13_1","volume-title":"The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. Preprint, arXiv:1808.00023","author":"Corbett-Davies Sam","year":"2018","unstructured":"Sam Corbett-Davies and Sharad Goel. 2018. The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. Preprint, arXiv:1808.00023 (2018)."},{"key":"e_1_3_2_1_14_1","volume-title":"A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Washington Post (October","author":"Corbett-Davies Sam","year":"2016","unstructured":"Sam Corbett-Davies, Emma Pierson, Avi Feller, and Sharad Goel. 2016. A computer program used for bail and sentencing decisions was labeled biased against blacks. It's actually not that clear. Washington Post (October 2016)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Sam Corbett-Davies Emma Pierson Avi Feller Sharad Goel and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness.","DOI":"10.1145\/3097983.3098095"},{"key":"e_1_3_2_1_16_1","unstructured":"Kate Crawford. 2017. The Trouble with Bias. NIPS Keynote https:\/\/www.youtube.com\/watch?v=fMym_BKWQzk accessed 08-22-2018."},{"key":"e_1_3_2_1_17_1","volume-title":"Fairness through awareness. ITCS","author":"Dwork Cynthia","year":"2012","unstructured":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. ITCS (2012)."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1177\/0093854818811379"},{"key":"e_1_3_2_1_19_1","unstructured":"equivant. 2017. Practitioner's Guide to COMPAS Core. http:\/\/www.equivant.com\/assets\/img\/content\/Practitioners_Guide_COMPASCore_121917.pdf http:\/\/www.equivant.com\/assets\/img\/content\/Practitioners_Guide_COMPASCore_121917.pdf accessed 08-22-2018."},{"key":"e_1_3_2_1_20_1","volume-title":"On the (im)possibility of fairness. Preprint, arXiv:1609.07236","author":"Friedler Sorelle A","year":"2016","unstructured":"Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian. 2016. On the (im)possibility of fairness. Preprint, arXiv:1609.07236 (2016)."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of the Aristotelian Society","volume":"56","author":"Gallie Walter Bryce","year":"1955","unstructured":"Walter Bryce Gallie. 1955. Essentially contested concepts. In Proceedings of the Aristotelian Society, Vol. 56. JSTOR, 167--198."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287568"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372869"},{"key":"e_1_3_2_1_24_1","volume-title":"Comment: Evaluating model performance in fictitious prediction problems. Technical Report.","author":"Grimmer Justin","year":"2012","unstructured":"Justin Grimmer. 2012. Comment: Evaluating model performance in fictitious prediction problems. Technical Report."},{"key":"e_1_3_2_1_25_1","unstructured":"David J Hand. 2004. Measurement theory and practice. Arnold Hodder Headline."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1093\/actrade\/9780198779568.001.0001"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372826"},{"key":"e_1_3_2_1_28_1","volume-title":"Advances in Neural Information Processing Systems","volume":"29","author":"Hardt Mortiz","year":"2016","unstructured":"Mortiz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In Advances in Neural Information Processing Systems, Vol. 29."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Anna Lauren Hoffmann. 2020. Rawls Information Technology and the Sociotechnical Bases of Self-Respect. In The Oxford Handbook of Philosophy of Technology.","DOI":"10.1093\/oxfordhb\/9780190851187.013.15"},{"key":"e_1_3_2_1_30_1","volume-title":"The Oxford Handbook of Political Methodology","author":"Jackman Simon","unstructured":"Simon Jackman. 2008. Measurement. In The Oxford Handbook of Political Methodology. Oxford Univ. Press, Chapter 9."},{"key":"e_1_3_2_1_31_1","volume-title":"Fair algorithms for infinite and contextual bandits. FATML, preprint arXiv:1610.09559","author":"Joseph Matthew","year":"2017","unstructured":"Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. Fair algorithms for infinite and contextual bandits. FATML, preprint arXiv:1610.09559 (2017)."},{"key":"e_1_3_2_1_32_1","volume-title":"Fairness in learning: Classic and contextual bandits. NIPS","author":"Joseph Matthew","year":"2016","unstructured":"Matthew Joseph, Michael Kearns, Jamie H Morgenstern, and Aaron Roth. 2016. Fairness in learning: Classic and contextual bandits. NIPS (2016)."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274357"},{"key":"e_1_3_2_1_34_1","volume-title":"Inherent trade-offs in the fair determination of risk scores. ITCS","author":"Kleinberg Jon","year":"2017","unstructured":"Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-offs in the fair determination of risk scores. ITCS (2017)."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1601"},{"key":"e_1_3_2_1_36_1","first-page":"635","article-title":"Objective tests as instruments of psychological theory","volume":"3","author":"Loevinger Jane","year":"1957","unstructured":"Jane Loevinger. 1957. Objective tests as instruments of psychological theory. Psychological Reports 3, 3 (1957), 635--694.","journal-title":"Psychological Reports"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1740-9713.2016.00960.x"},{"key":"e_1_3_2_1_38_1","volume-title":"Andrew Smart, and William S Isaac","author":"Martin Donald","year":"2020","unstructured":"Donald Martin, Jr, Vinodkumar Prabhakaran, Jill Kuhlberg, Andrew Smart, and William S Isaac. 2020. Extending the Machine Learning Abstraction Boundary: A Complex Systems Approach to Incorporate Societal Context. arXiv preprint arXiv:2006.09663 (2020)."},{"key":"e_1_3_2_1_39_1","first-page":"2218","article-title":"Bias in, bias out","volume":"128","author":"Mayson Sandra G","year":"2018","unstructured":"Sandra G Mayson. 2018. Bias in, bias out. Yale Law Journal 128 (2018), 2218.","journal-title":"Yale Law Journal"},{"key":"e_1_3_2_1_40_1","series-title":"ETS Research Report Series.","volume-title":"Validity","author":"Messick Samuel","unstructured":"Samuel Messick. 1987. Validity. In ETS Research Report Series."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.31.2.87"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2016.0118"},{"key":"e_1_3_2_1_43_1","volume-title":"Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464","author":"Obermeyer Ziad","year":"2019","unstructured":"Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 6464 (2019), 447--453."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2019.00013"},{"key":"e_1_3_2_1_45_1","unstructured":"National Committee on Vital and Health Statistics (NCVHS). 2012. Re: Development of Standards for the Collection of Socioeconomic Status in Health Surveys Conducted by the Department of Health and Human Services. https:\/\/www.ncvhs.hhs.gov\/wp-content\/uploads\/2014\/05\/120622lt.pdf"},{"key":"e_1_3_2_1_46_1","volume-title":"Weapons of Math Destruction","author":"O'Neil Cathy","unstructured":"Cathy O'Neil. 2016. Weapons of Math Destruction. Broadway Books."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287567"},{"key":"e_1_3_2_1_48_1","volume-title":"On fairness and calibration. NIPS","author":"Pleiss Geoff","year":"2017","unstructured":"Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. 2017. On fairness and calibration. NIPS (2017)."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-5907.2009.00427.x"},{"key":"e_1_3_2_1_50_1","first-page":"987","article-title":"Arrests as guilt. Ala","volume":"70","author":"Roberts Anna","year":"2018","unstructured":"Anna Roberts. 2018. Arrests as guilt. Ala. L. Rev. 70 (2018), 987.","journal-title":"L. Rev."},{"key":"e_1_3_2_1_51_1","unstructured":"SAS. [n. d.]. EVAAS for K-12 Statistical Models. Available at https:\/\/www.sas.com\/content\/dam\/SAS\/en_us\/doc\/whitepaper1\/sas-evaas-k12-statistical-models-107411.pdf accessed 08-23-2018."},{"key":"e_1_3_2_1_52_1","volume-title":"Fairness and Abstraction in Sociotechnical Systems. In ACM Conference on Fairness, Accountability, and Transparency (FAT*).","author":"Selbst Andrew D","year":"2019","unstructured":"Andrew D Selbst, Sorelle Friedler, Suresh Venkatasubramanian, Janet Vertesi, et al. 2019. Fairness and Abstraction in Sociotechnical Systems. In ACM Conference on Fairness, Accountability, and Transparency (FAT*)."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1006985528729"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1177\/0146167208318067"},{"key":"e_1_3_2_1_55_1","volume-title":"UK: Automated Benefits System Failing People in Need. https:\/\/www.hrw.org\/news\/2020\/09\/29\/uk-automated-benefits-system-failing-people-need","author":"Watch Human Rights","year":"2020","unstructured":"Human Rights Watch. 2020. UK: Automated Benefits System Failing People in Need. https:\/\/www.hrw.org\/news\/2020\/09\/29\/uk-automated-benefits-system-failing-people-need"}],"event":{"name":"FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency","location":"Virtual Event Canada","acronym":"FAccT '21","sponsor":["ACM Association for Computing Machinery"]},"container-title":["Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3442188.3445901","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3442188.3445901","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:57Z","timestamp":1750193337000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3442188.3445901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3]]},"references-count":55,"alternative-id":["10.1145\/3442188.3445901","10.1145\/3442188"],"URL":"https:\/\/doi.org\/10.1145\/3442188.3445901","relation":{},"subject":[],"published":{"date-parts":[[2021,3]]},"assertion":[{"value":"2021-03-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}