{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T17:23:22Z","timestamp":1774373002326,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["IIS2040929"],"award-info":[{"award-number":["IIS2040929"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,21]]},"DOI":"10.1145\/3461702.3462583","type":"proceedings-article","created":{"date-parts":[[2021,7,31]],"date-time":"2021-07-31T01:21:32Z","timestamp":1627694492000},"page":"1023-1033","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["A Human-in-the-loop Framework to Construct Context-aware Mathematical Notions of Outcome Fairness"],"prefix":"10.1145","author":[{"given":"Mohammad","family":"Yaghini","sequence":"first","affiliation":[{"name":"University of Toronto, Toronto, ON, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Krause","sequence":"additional","affiliation":[{"name":"ETH Z\u00fcrich, Z\u00fcrich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hoda","family":"Heidari","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,30]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Machine Bias. Propublica","author":"Angwin Julia","year":"2016","unstructured":"Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine Bias. Propublica ( 2016 ). Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. Propublica (2016)."},{"key":"e_1_3_2_2_2_1","volume-title":"Nature","volume":"563","author":"Awad Edmond","year":"2018","unstructured":"Edmond Awad , Sohan Dsouza , Richard Kim , Jonathan Schulz , Joseph Henrich , Azim Shariff , Jean-Francois Bonnefon , and Iyad Rahwan . 2018 . The moral machine experiment . Nature , Vol. 563 , 7729 (2018), 59. Edmond Awad, Sohan Dsouza, Richard Kim, Jonathan Schulz, Joseph Henrich, Azim Shariff, Jean-Francois Bonnefon, and Iyad Rahwan. 2018. The moral machine experiment. Nature, Vol. 563, 7729 (2018), 59."},{"key":"e_1_3_2_2_3_1","volume-title":"Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arXiv preprint arXiv:1703.00056","author":"Chouldechova Alexandra","year":"2017","unstructured":"Alexandra Chouldechova . 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arXiv preprint arXiv:1703.00056 ( 2017 ). Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arXiv preprint arXiv:1703.00056 (2017)."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11140"},{"key":"e_1_3_2_2_5_1","volume-title":"COMPAS risk scales: Demonstrating accuracy equity and predictive parity","author":"Dieterich William","year":"2016","unstructured":"William Dieterich , Christina Mendoza , and Tim Brennan . 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity . Northpoint Inc ( 2016 ). William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpoint Inc (2016)."},{"key":"e_1_3_2_2_6_1","volume-title":"The accuracy, fairness, and limits of predicting recidivism. Science advances","author":"Dressel Julia","year":"2018","unstructured":"Julia Dressel and Hany Farid . 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances , Vol. 4 , 1 ( 2018 ). Julia Dressel and Hany Farid. 2018. The accuracy, fairness, and limits of predicting recidivism. Science advances, Vol. 4, 1 (2018)."},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090236.2090255"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2020.103261"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186138"},{"key":"e_1_3_2_2_10_1","volume-title":"Proceedings of Advances in Neural Information Processing Systems. 3315--3323","author":"Hardt Moritz","year":"2016","unstructured":"Moritz Hardt , Eric Price , and Nati Srebro . 2016 . Equality of opportunity in supervised learning . In Proceedings of Advances in Neural Information Processing Systems. 3315--3323 . Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. In Proceedings of Advances in Neural Information Processing Systems. 3315--3323."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287584"},{"key":"e_1_3_2_2_12_1","volume-title":"Proceedings of the 2018 CHI conference on human factors in computing systems.","author":"Holstein Kenneth","year":"2018","unstructured":"Kenneth Holstein , Jennifer Wortman Vaughan , Hal Daum\u00e9 III , Miro Dud'ik , and Hanna Wallach . 2018 . Improving fairness in machine learning systems: What do industry practitioners need? . In Proceedings of the 2018 CHI conference on human factors in computing systems. Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daum\u00e9 III, Miro Dud'ik, and Hanna Wallach. 2018. Improving fairness in machine learning systems: What do industry practitioners need?. In Proceedings of the 2018 CHI conference on human factors in computing systems."},{"key":"e_1_3_2_2_13_1","volume-title":"Proceedings of the Innovations in Theoretical Computer Science Conference.","author":"Kleinberg Jon","year":"2017","unstructured":"Jon Kleinberg , Sendhil Mullainathan , and Manish Raghavan . 2017 . Inherent trade-offs in the fair determination of risk scores . In Proceedings of the Innovations in Theoretical Computer Science Conference. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. 2017. Inherent trade-offs in the fair determination of risk scores. In Proceedings of the Innovations in Theoretical Computer Science Conference."},{"key":"e_1_3_2_2_14_1","unstructured":"Jeff Larson Surya Mattu Lauren Kirchner and Julia Angwin. 2016. Data and analysis for 'How we analyzed the COMPAS recidivism algorithm'. https:\/\/github.com\/propublica\/compas-analysis .  Jeff Larson Surya Mattu Lauren Kirchner and Julia Angwin. 2016. Data and analysis for 'How we analyzed the COMPAS recidivism algorithm'. https:\/\/github.com\/propublica\/compas-analysis ."},{"key":"e_1_3_2_2_15_1","volume-title":"Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, et almbox.","author":"Lee Min Kyung","year":"2019","unstructured":"Min Kyung Lee , Daniel Kusbit , Anson Kahng , Ji Tae Kim , Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, et almbox. 2019 . WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction, Vol. 3 , CSCW ( 2019), 1--35. Min Kyung Lee, Daniel Kusbit, Anson Kahng, Ji Tae Kim, Xinran Yuan, Allissa Chan, Daniel See, Ritesh Noothigattu, Siheon Lee, Alexandros Psomas, et almbox. 2019. WeBuildAI: Participatory framework for algorithmic governance. Proceedings of the ACM on Human-Computer Interaction, Vol. 3, CSCW (2019), 1--35."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpubeco.2009.07.008"},{"key":"e_1_3_2_2_17_1","volume-title":"Skeem","author":"Monahan John","year":"2016","unstructured":"John Monahan and Jennifer L . Skeem . 2016 . Risk assessment in criminal sentencing. Annual review of clinical psychology, Vol. 12 (2016), 489--513. John Monahan and Jennifer L. Skeem. 2016. Risk assessment in criminal sentencing. Annual review of clinical psychology, Vol. 12 (2016), 489--513."},{"key":"e_1_3_2_2_18_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence .","author":"Noothigattu Ritesh","unstructured":"Ritesh Noothigattu , Snehalkumar 'Neil' S. Gaikwad , Edmond Awad , Sohan Dsouza , Iyad Rahwan , Pradeep Ravikumar , and Ariel D. Procaccia . 2018. A voting-based system for ethical decision making . In Proceedings of the AAAI Conference on Artificial Intelligence . Ritesh Noothigattu, Snehalkumar 'Neil' S. Gaikwad, Edmond Awad, Sohan Dsouza, Iyad Rahwan, Pradeep Ravikumar, and Ariel D. Procaccia. 2018. A voting-based system for ethical decision making. In Proceedings of the AAAI Conference on Artificial Intelligence ."},{"key":"e_1_3_2_2_19_1","volume-title":"Conversational UI is Becoming the Preferred Consumer Interface: 12 Reasons Why. https:\/\/clinc.com\/conversational-ui-preferred-consumer-interface\/. Clinc","author":"Rice Bill","year":"2019","unstructured":"Bill Rice . 2019. Conversational UI is Becoming the Preferred Consumer Interface: 12 Reasons Why. https:\/\/clinc.com\/conversational-ui-preferred-consumer-interface\/. Clinc ( 2019 ). Bill Rice. 2019. Conversational UI is Becoming the Preferred Consumer Interface: 12 Reasons Why. https:\/\/clinc.com\/conversational-ui-preferred-consumer-interface\/. Clinc (2019)."},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s003550100123"},{"key":"e_1_3_2_2_21_1","volume-title":"The age of secrecy and unfairness in recidivism prediction. arXiv preprint arXiv:1811.00731","author":"Rudin Cynthia","year":"2018","unstructured":"Cynthia Rudin , Caroline Wang , and Beau Coker . 2018. The age of secrecy and unfairness in recidivism prediction. arXiv preprint arXiv:1811.00731 ( 2018 ). Cynthia Rudin, Caroline Wang, and Beau Coker. 2018. The age of secrecy and unfairness in recidivism prediction. arXiv preprint arXiv:1811.00731 (2018)."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375819"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314248"},{"key":"e_1_3_2_2_24_1","volume-title":"Social choice theory. Handbook of mathematical economics","author":"Sen Amartya","year":"1986","unstructured":"Amartya Sen . 1986. Social choice theory. Handbook of mathematical economics , Vol. 3 ( 1986 ), 1073--1181. Amartya Sen. 1986. Social choice theory. Handbook of mathematical economics, Vol. 3 (1986), 1073--1181."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330664"},{"key":"e_1_3_2_2_26_1","first-page":"303","article-title":"Assessing risk assessment in action","volume":"103","author":"Stevenson Megan","year":"2018","unstructured":"Megan Stevenson . 2018 . Assessing risk assessment in action . Minn. L. Rev. , Vol. 103 (2018), 303 . Megan Stevenson. 2018. Assessing risk assessment in action. Minn. L. Rev., Vol. 103 (2018), 303.","journal-title":"Minn. L. Rev."},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174014"},{"key":"e_1_3_2_2_28_1","volume-title":"Proceedings of Advances in Neural Information Processing Systems.","author":"Wick Michael","year":"2019","unstructured":"Michael Wick , Swetasudha Panda , and Jean-Baptiste Tristan . 2019 . Unlocking fairness: a trade-off revisited . In Proceedings of Advances in Neural Information Processing Systems. Michael Wick, Swetasudha Panda, and Jean-Baptiste Tristan. 2019. Unlocking fairness: a trade-off revisited. In Proceedings of Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174230"},{"key":"e_1_3_2_2_30_1","volume-title":"Proceedings of the World Wide Web Conference. 1171--1180","author":"Zafar Muhammad Bilal","unstructured":"Muhammad Bilal Zafar , Isabel Valera , Manuel Gomez Rodriguez , and Krishna P. Gummadi . 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment . In Proceedings of the World Wide Web Conference. 1171--1180 . Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the World Wide Web Conference. 1171--1180."}],"event":{"name":"AIES '21: AAAI\/ACM Conference on AI, Ethics, and Society","location":"Virtual Event USA","acronym":"AIES '21","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","AAAI"]},"container-title":["Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462583","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461702.3462583","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461702.3462583","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:05Z","timestamp":1750191425000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462583"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":30,"alternative-id":["10.1145\/3461702.3462583","10.1145\/3461702"],"URL":"https:\/\/doi.org\/10.1145\/3461702.3462583","relation":{},"subject":[],"published":{"date-parts":[[2021,7,21]]},"assertion":[{"value":"2021-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}