{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,14]],"date-time":"2026-06-14T05:39:56Z","timestamp":1781415596335,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":82,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,4,29]],"date-time":"2022-04-29T00:00:00Z","timestamp":1651190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2040942"],"award-info":[{"award-number":["2040942"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Cisco"},{"name":"Amazon"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,29]]},"DOI":"10.1145\/3491102.3517441","type":"proceedings-article","created":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T16:34:58Z","timestamp":1651163698000},"page":"1-19","source":"Crossref","is-referenced-by-count":84,"title":["Toward User-Driven Algorithm Auditing: Investigating users\u2019 strategies for uncovering harmful algorithmic behavior"],"prefix":"10.1145","author":[{"given":"Alicia","family":"DeVos","sequence":"first","affiliation":[{"name":"Human-Computer Interaction Institute, Carnegie Mellon University, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aditi","family":"Dhabalia","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong","family":"Shen","sequence":"additional","affiliation":[{"name":"Human-Computer Interaction Institute, Carnegie Mellon University, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kenneth","family":"Holstein","sequence":"additional","affiliation":[{"name":"Human-Computer Interaction Institute, Carnegie Mellon University, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Motahhare","family":"Eslami","sequence":"additional","affiliation":[{"name":"School of Computer Science, Carnegie Mellon University, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,29]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"[n.d.]. Google image search cements national stereotypes of \u2019racy\u2019 women. ([n. d.]). https:\/\/www.dw.com\/en\/google-image-search-cements-national-stereotypes-of-racy-women\/a-56767605  [n.d.]. Google image search cements national stereotypes of \u2019racy\u2019 women. ([n. d.]). https:\/\/www.dw.com\/en\/google-image-search-cements-national-stereotypes-of-racy-women\/a-56767605"},{"key":"e_1_3_2_2_2_1","volume-title":"Do Google\u2019s \u2019unprofessional hair","year":"2016","unstructured":"2016. Do Google\u2019s \u2019unprofessional hair \u2019 results show it is racist?( 2016 ). https:\/\/www.theguardian.com\/technology\/2016\/apr\/08\/does-google-unprofessional-hair-results-prove-algorithms-racist- 2016. Do Google\u2019s \u2019unprofessional hair\u2019 results show it is racist?(2016). https:\/\/www.theguardian.com\/technology\/2016\/apr\/08\/does-google-unprofessional-hair-results-prove-algorithms-racist-"},{"key":"e_1_3_2_2_3_1","volume-title":"I saw a tweet saying \u201dGoogle unprofessional hairstyles for work\u201d. I did. Then I checked the \u2019professional","year":"2016","unstructured":"2016. I saw a tweet saying \u201dGoogle unprofessional hairstyles for work\u201d. I did. Then I checked the \u2019professional \u2019 ones. ( 2016 ). https:\/\/twitter.com\/HereroRocher\/status\/717457819864272896 2016. I saw a tweet saying \u201dGoogle unprofessional hairstyles for work\u201d. I did. Then I checked the \u2019professional\u2019 ones. (2016). https:\/\/twitter.com\/HereroRocher\/status\/717457819864272896"},{"key":"e_1_3_2_2_4_1","unstructured":"2018. Gender and Jobs in Online Image Searches. (2018). https:\/\/www.pewresearch.org\/social-trends\/2018\/12\/17\/gender-and-jobs-in-online-image-searches\/  2018. Gender and Jobs in Online Image Searches. (2018). https:\/\/www.pewresearch.org\/social-trends\/2018\/12\/17\/gender-and-jobs-in-online-image-searches\/"},{"key":"e_1_3_2_2_5_1","volume-title":"Religion & Conscious Language. panel at ACES Annual Conference. https:\/\/aceseditors","author":"Holy","year":"2019","unstructured":"2019. Holy $%#@ : Religion & Conscious Language. panel at ACES Annual Conference. https:\/\/aceseditors .org\/conference\/past-conferences\/ 2019 \/sessions 2019. Holy $%#@: Religion & Conscious Language. panel at ACES Annual Conference. https:\/\/aceseditors.org\/conference\/past-conferences\/2019\/sessions"},{"key":"e_1_3_2_2_6_1","volume-title":"Search Engine Bias. (May","year":"2019","unstructured":"2019. Search Engine Bias. (May 2019 ). https:\/\/therepproject.org\/search-engine-bias\/ 2019. Search Engine Bias. (May 2019). https:\/\/therepproject.org\/search-engine-bias\/"},{"key":"e_1_3_2_2_7_1","volume-title":"https:\/\/www2.census.gov\/programs-surveys\/decennial\/2020\/technical-documentation\/questionnaires-and-instructions\/questionnaires\/2020-informational-questionnaire.pdf","year":"2020","unstructured":"[ 7 ] 2020. ( 2020 ). https:\/\/www2.census.gov\/programs-surveys\/decennial\/2020\/technical-documentation\/questionnaires-and-instructions\/questionnaires\/2020-informational-questionnaire.pdf [7] 2020. (2020). https:\/\/www2.census.gov\/programs-surveys\/decennial\/2020\/technical-documentation\/questionnaires-and-instructions\/questionnaires\/2020-informational-questionnaire.pdf"},{"key":"e_1_3_2_2_8_1","volume-title":"To Live in Their Utopia: Why Algorithmic Systems Create Absurd Outcomes","author":"Alkhatib Ali","unstructured":"Ali Alkhatib . 2021. To Live in Their Utopia: Why Algorithmic Systems Create Absurd Outcomes . Association for Computing Machinery , New York, NY, USA . https:\/\/doi.org\/10.1145\/3411764.3445740 Ali Alkhatib. 2021. To Live in Their Utopia: Why Algorithmic Systems Create Absurd Outcomes. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3411764.3445740"},{"key":"e_1_3_2_2_9_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media 14","author":"Asplund Joshua","year":"2020","unstructured":"Joshua Asplund , Motahhare Eslami , Hari Sundaram , Christian Sandvig , and Karrie Karahalios . 2020 . Auditing Race and Gender Discrimination in Online Housing Markets . Proceedings of the International AAAI Conference on Web and Social Media 14 , 1 (May 2020), 24\u201335. https:\/\/ojs.aaai.org\/index.php\/ICWSM\/article\/view\/7276 Joshua Asplund, Motahhare Eslami, Hari Sundaram, Christian Sandvig, and Karrie Karahalios. 2020. Auditing Race and Gender Discrimination in Online Housing Markets. Proceedings of the International AAAI Conference on Web and Social Media 14, 1 (May 2020), 24\u201335. https:\/\/ojs.aaai.org\/index.php\/ICWSM\/article\/view\/7276"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2700832"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979275"},{"key":"e_1_3_2_2_12_1","volume-title":"\u2019It\u2019s Reducing a Human Being to a Percentage\u2019","author":"Binns Reuben","year":"1801","unstructured":"Reuben Binns , Max\u00a0Van Kleek , Michael Veale , Ulrik Lyngs , Jun Zhao , and Nigel Shadbolt . 2018. \u2019It\u2019s Reducing a Human Being to a Percentage\u2019 ; Perceptions of Justice in Algorithmic Decisions. CoRR abs\/ 1801 .10408(2018). arxiv:1801.10408http:\/\/arxiv.org\/abs\/1801.10408 Reuben Binns, Max\u00a0Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. \u2019It\u2019s Reducing a Human Being to a Percentage\u2019; Perceptions of Justice in Algorithmic Decisions. CoRR abs\/1801.10408(2018). arxiv:1801.10408http:\/\/arxiv.org\/abs\/1801.10408"},{"key":"e_1_3_2_2_13_1","volume-title":"A critical survey of","author":"Blodgett Su\u00a0Lin","year":"2005","unstructured":"Su\u00a0Lin Blodgett , Solon Barocas , Hal Daum\u00e9\u00a0III, and Hanna Wallach . 2020. Language (technology) is power : A critical survey of \u201d bias\u201d in nlp. arXiv preprint arXiv: 2005 .14050(2020). Su\u00a0Lin Blodgett, Solon Barocas, Hal Daum\u00e9\u00a0III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of\u201d bias\u201d in nlp. arXiv preprint arXiv:2005.14050(2020)."},{"key":"e_1_3_2_2_14_1","volume-title":"Twitter users say the platform crops out Black faces. CBS News (Sep","author":"Brooks J","year":"2020","unstructured":"Khristopher\u00a0 J Brooks . 2020. Twitter users say the platform crops out Black faces. CBS News (Sep 2020 ). https:\/\/www.cbsnews.com\/news\/twitter-image-cropping-algorithm-racial-profiling\/ Khristopher\u00a0J Brooks. 2020. Twitter users say the platform crops out Black faces. CBS News (Sep 2020). https:\/\/www.cbsnews.com\/news\/twitter-image-cropping-algorithm-racial-profiling\/"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Anna Brown Alexandra Chouldechova Emily Putnam-Hornstein Andrew Tobin and Rhema Vaithianathan. 2019. Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-Making in Child Welfare Services. (2019) 1\u201312. https:\/\/doi.org\/10.1145\/3290605.3300271  Anna Brown Alexandra Chouldechova Emily Putnam-Hornstein Andrew Tobin and Rhema Vaithianathan. 2019. Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-Making in Child Welfare Services. (2019) 1\u201312. https:\/\/doi.org\/10.1145\/3290605.3300271","DOI":"10.1145\/3290605.3300271"},{"key":"e_1_3_2_2_16_1","volume-title":"Making sense of sensemaking narratives. Human relations 61, 8","author":"Brown D","year":"2008","unstructured":"Andrew\u00a0 D Brown , Patrick Stacey , and Joe Nandhakumar . 2008. Making sense of sensemaking narratives. Human relations 61, 8 ( 2008 ), 1035\u20131062. Andrew\u00a0D Brown, Patrick Stacey, and Joe Nandhakumar. 2008. Making sense of sensemaking narratives. Human relations 61, 8 (2008), 1035\u20131062."},{"key":"e_1_3_2_2_17_1","volume-title":"Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. 81 (23\u201324","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. 81 (23\u201324 Feb 2018), 77\u201391. http:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. 81 (23\u201324 Feb 2018), 77\u201391. http:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479569"},{"key":"e_1_3_2_2_19_1","unstructured":"Metz Cade. [n.d.]. \u2018Nerd \u2019 \u2018Nonsmoker \u2019 \u2018Wrongdoer\u2019: How Might A.I. Label You? ImageNet Roulette a digital art project and viral selfie app exposes how biases have crept into the artificial-intelligence technologies changing our lives. ([n. d.]). https:\/\/www.nytimes.com\/2019\/09\/20\/arts\/design\/imagenet-trevor-paglen-ai-facial-recognition.html  Metz Cade. [n.d.]. \u2018Nerd \u2019 \u2018Nonsmoker \u2019 \u2018Wrongdoer\u2019: How Might A.I. Label You? ImageNet Roulette a digital art project and viral selfie app exposes how biases have crept into the artificial-intelligence technologies changing our lives. ([n. d.]). https:\/\/www.nytimes.com\/2019\/09\/20\/arts\/design\/imagenet-trevor-paglen-ai-facial-recognition.html"},{"key":"e_1_3_2_2_20_1","volume-title":"Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights.","author":"J\u00a0PATIL","year":"2016","unstructured":"D J\u00a0PATIL CECILIA\u00a0MU\u00d1OZ, MEGAN\u00a0 SMITH. 2016 . Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights. (2016). https:\/\/obamawhitehouse.archives.gov\/sites\/default\/files\/microsites\/ostp\/2016_0504_data_discrimination.pdf DJ\u00a0PATIL CECILIA\u00a0MU\u00d1OZ, MEGAN\u00a0SMITH. 2016. Big Data: A Report on Algorithmic Systems, Opportunity, and Civil Rights. (2016). https:\/\/obamawhitehouse.archives.gov\/sites\/default\/files\/microsites\/ostp\/2016_0504_data_discrimination.pdf"},{"key":"e_1_3_2_2_21_1","volume-title":"Investigating the Impact of Gender on Rank in Resume Search Engines","author":"Chen Le","unstructured":"Le Chen , Ruijun Ma , Anik\u00f3 Hann\u00e1k , and Christo Wilson . 2018. Investigating the Impact of Gender on Rank in Resume Search Engines . Association for Computing Machinery , New York, NY, USA , 1\u201314. https:\/\/doi.org\/10.1145\/3173574.3174225 Le Chen, Ruijun Ma, Anik\u00f3 Hann\u00e1k, and Christo Wilson. 2018. Investigating the Impact of Gender on Rank in Resume Search Engines. Association for Computing Machinery, New York, NY, USA, 1\u201314. https:\/\/doi.org\/10.1145\/3173574.3174225"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278156"},{"key":"e_1_3_2_2_23_1","volume-title":"Excavating AI: The politics of images in machine learning training sets","author":"Crawford Kate","year":"2021","unstructured":"Kate Crawford and Trevor Paglen . 2021 . Excavating AI: The politics of images in machine learning training sets . AI & Society ( 2021), 1\u201312. Kate Crawford and Trevor Paglen. 2021. Excavating AI: The politics of images in machine learning training sets. AI & Society (2021), 1\u201312."},{"key":"e_1_3_2_2_24_1","unstructured":"Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. (2018). https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight\/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G  Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. (2018). https:\/\/www.reuters.com\/article\/us-amazon-com-jobs-automation-insight\/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"crossref","unstructured":"MICHAEL\u00a0ANN DEVITO. 2021. Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing Platforms. (2021). https:\/\/doi.org\/10.1145\/3476080  MICHAEL\u00a0ANN DEVITO. 2021. Adaptive Folk Theorization as a Path to Algorithmic Literacy on Changing Platforms. (2021). https:\/\/doi.org\/10.1145\/3476080","DOI":"10.1145\/3476080"},{"key":"e_1_3_2_2_26_1","volume-title":"How People Form Folk Theories of Social Media Feeds and What It Means for How We Study Self-Presentation","author":"DeVito A.","unstructured":"Michael\u00a0 A. DeVito , Jeremy Birnholtz , Jeffery\u00a0 T. Hancock , Megan French , and Sunny Liu . 2018. How People Form Folk Theories of Social Media Feeds and What It Means for How We Study Self-Presentation . Association for Computing Machinery , New York, NY, USA , 1\u201312. https:\/\/doi.org\/10.1145\/3173574.3173694 Michael\u00a0A. DeVito, Jeremy Birnholtz, Jeffery\u00a0T. Hancock, Megan French, and Sunny Liu. 2018. How People Form Folk Theories of Social Media Feeds and What It Means for How We Study Self-Presentation. Association for Computing Machinery, New York, NY, USA, 1\u201312. https:\/\/doi.org\/10.1145\/3173574.3173694"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025659"},{"key":"e_1_3_2_2_28_1","volume-title":"Then I Hide It: Folk Theories of Social Feeds","author":"Eslami Motahhare","unstructured":"Motahhare Eslami , Karrie Karahalios , Christian Sandvig , Kristen Vaccaro , Aimee Rickman , Kevin Hamilton , and Alex Kirlik . 2016. First I \u201dlike\u201d It , Then I Hide It: Folk Theories of Social Feeds . Association for Computing Machinery , New York, NY, USA , 2371\u20132382. https:\/\/doi.org\/10.1145\/2858036.2858494 Motahhare Eslami, Karrie Karahalios, Christian Sandvig, Kristen Vaccaro, Aimee Rickman, Kevin Hamilton, and Alex Kirlik. 2016. First I \u201dlike\u201d It, Then I Hide It: Folk Theories of Social Feeds. Association for Computing Machinery, New York, NY, USA, 2371\u20132382. https:\/\/doi.org\/10.1145\/2858036.2858494"},{"key":"e_1_3_2_2_29_1","volume-title":"Communicating Algorithmic Process in Online Behavioral Advertising","author":"Eslami Motahhare","unstructured":"Motahhare Eslami , Sneha\u00a0 R. Krishna\u00a0Kumaran , Christian Sandvig , and Karrie Karahalios . 2018. Communicating Algorithmic Process in Online Behavioral Advertising . Association for Computing Machinery , New York, NY, USA , 1\u201313. https:\/\/doi.org\/10.1145\/3173574.3174006 Motahhare Eslami, Sneha\u00a0R. Krishna\u00a0Kumaran, Christian Sandvig, and Karrie Karahalios. 2018. Communicating Algorithmic Process in Online Behavioral Advertising. Association for Computing Machinery, New York, NY, USA, 1\u201313. https:\/\/doi.org\/10.1145\/3173574.3174006"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702556"},{"key":"e_1_3_2_2_31_1","volume-title":"Be careful","author":"Eslami Motahhare","year":"2017","unstructured":"Motahhare Eslami , Kristen Vaccaro , Karrie Karahalios , and Kevin Hamilton . 2017. Be careful ; Things can be worse than they appear - Understanding biased algorithms and users\u2019 behavior around them in rating platforms\u201d. ( 2017 ), 62\u201371. Funding Information : This work was funded by NSF grant CHS-1564041. Publisher Copyright : \u00a9 Copyright 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.; 11th International Conference on Web and Social Media, ICWSM 2017 ; Conference date: 15-05-2017 Through 18-05-2017. Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. Be careful; Things can be worse than they appear - Understanding biased algorithms and users\u2019 behavior around them in rating platforms\u201d. (2017), 62\u201371. Funding Information: This work was funded by NSF grant CHS-1564041. Publisher Copyright: \u00a9 Copyright 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.; 11th International Conference on Web and Social Media, ICWSM 2017 ; Conference date: 15-05-2017 Through 18-05-2017."},{"key":"e_1_3_2_2_32_1","volume-title":"Eric Gilbert, and Karrie Karahalios.","author":"Eslami Motahhare","year":"2019","unstructured":"Motahhare Eslami , Kristen Vaccaro , Min\u00a0Kyung Lee , Amit Elazari Bar\u00a0On , Eric Gilbert, and Karrie Karahalios. 2019 . User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing Platforms . (2019), 1\u201314. https:\/\/doi.org\/10.1145\/3290605.3300724 Motahhare Eslami, Kristen Vaccaro, Min\u00a0Kyung Lee, Amit Elazari Bar\u00a0On, Eric Gilbert, and Karrie Karahalios. 2019. User Attitudes towards Algorithmic Opacity and Transparency in Online Reviewing Platforms. (2019), 1\u201314. https:\/\/doi.org\/10.1145\/3290605.3300724"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300724"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Nancy Fraser. 1990. Rethinking the public sphere: A contribution to the critique of actually existing democracy. Social text25\/26(1990) 56\u201380.  Nancy Fraser. 1990. Rethinking the public sphere: A contribution to the critique of actually existing democracy. Social text25\/26(1990) 56\u201380.","DOI":"10.2307\/466240"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/230538.230561"},{"key":"e_1_3_2_2_36_1","volume-title":"Towards a theory of thinking, Goel Vinod von M\u00fcller\u00a0Albrecht Glatzeder","author":"Goldstone S.","unstructured":"Day\u00a0 S. Goldstone , R.\u00a0 L. and J.\u00a0Y. Son . 2010. Comparison . In Towards a theory of thinking, Goel Vinod von M\u00fcller\u00a0Albrecht Glatzeder , Britt (Ed.). Springer, Berlin , Heidelberg , 103\u2013121. Day\u00a0S. Goldstone, R.\u00a0L. and J.\u00a0Y. Son. 2010. Comparison. In Towards a theory of thinking, Goel Vinod von M\u00fcller\u00a0Albrecht Glatzeder, Britt (Ed.). Springer, Berlin, Heidelberg, 103\u2013121."},{"key":"e_1_3_2_2_37_1","volume-title":"The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality","author":"Gordon L.","unstructured":"Mitchell\u00a0 L. Gordon , Kaitlyn Zhou , Kayur Patel , Tatsunori Hashimoto , and Michael\u00a0 S. Bernstein . 2021. The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality . Association for Computing Machinery , New York, NY, USA . https:\/\/doi.org\/10.1145\/3411764.3445423 Mitchell\u00a0L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael\u00a0S. Bernstein. 2021. The Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3411764.3445423"},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Nina Grgic-Hlaca Elissa\u00a0M. Redmiles Krishna\u00a0P. Gummadi and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction. (2018) 903\u2013912. https:\/\/doi.org\/10.1145\/3178876.3186138  Nina Grgic-Hlaca Elissa\u00a0M. Redmiles Krishna\u00a0P. Gummadi and Adrian Weller. 2018. Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction. (2018) 903\u2013912. https:\/\/doi.org\/10.1145\/3178876.3186138","DOI":"10.1145\/3178876.3186138"},{"key":"e_1_3_2_2_39_1","unstructured":"Ben Guarino. [n.d.]. Google faulted for racial bias in image search results for black teenagers. ([n. d.]). https:\/\/www.dw.com\/en\/google-image-search-cements-national-stereotypes-of-racy-women\/a-56767605  Ben Guarino. [n.d.]. Google faulted for racial bias in image search results for black teenagers. ([n. d.]). https:\/\/www.dw.com\/en\/google-image-search-cements-national-stereotypes-of-racy-women\/a-56767605"},{"key":"e_1_3_2_2_40_1","volume-title":"Google photos labeled black people \u2018gorillas\u2019. USA Today (June","author":"Guynn Jessica","year":"2015","unstructured":"Jessica Guynn . 2015. Google photos labeled black people \u2018gorillas\u2019. USA Today (June 2015 ). https:\/\/www.usatoday.com\/story\/tech\/2015\/07\/01\/google-apologizes-after-photos-identify-black-people-as-gorillas\/29567465 Jessica Guynn. 2015. Google photos labeled black people \u2018gorillas\u2019. USA Today (June 2015). https:\/\/www.usatoday.com\/story\/tech\/2015\/07\/01\/google-apologizes-after-photos-identify-black-people-as-gorillas\/29567465"},{"key":"e_1_3_2_2_41_1","unstructured":"Anik\u00f3 Hann\u00e1k Piotr Sapiezynski Arash\u00a0Molavi Khaki David Lazer Alan Mislove and Christo Wilson. 2017. Measuring Personalization of Web Search. CoRR abs\/1706.05011(2017). arXiv:1706.05011http:\/\/arxiv.org\/abs\/1706.05011  Anik\u00f3 Hann\u00e1k Piotr Sapiezynski Arash\u00a0Molavi Khaki David Lazer Alan Mislove and Christo Wilson. 2017. Measuring Personalization of Web Search. CoRR abs\/1706.05011(2017). arXiv:1706.05011http:\/\/arxiv.org\/abs\/1706.05011"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"crossref","unstructured":"Aniko Hannak Gary Soeller David Lazer Alan Mislove and Christo Wilson. 2014. Measuring Price Discrimination and Steering on E-Commerce Web Sites. (2014) 305\u2013318. https:\/\/doi.org\/10.1145\/2663716.2663744  Aniko Hannak Gary Soeller David Lazer Alan Mislove and Christo Wilson. 2014. Measuring Price Discrimination and Steering on E-Commerce Web Sites. (2014) 305\u2013318. https:\/\/doi.org\/10.1145\/2663716.2663744","DOI":"10.1145\/2663716.2663744"},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"crossref","unstructured":"Aniko Hannak Gary Soeller David Lazer Alan Mislove and Christo Wilson. 2014. Measuring Price Discrimination and Steering on E-Commerce Web Sites. (2014) 305\u2013318. https:\/\/doi.org\/10.1145\/2663716.2663744  Aniko Hannak Gary Soeller David Lazer Alan Mislove and Christo Wilson. 2014. Measuring Price Discrimination and Steering on E-Commerce Web Sites. (2014) 305\u2013318. https:\/\/doi.org\/10.1145\/2663716.2663744","DOI":"10.1145\/2663716.2663744"},{"key":"e_1_3_2_2_44_1","volume-title":"Twitter apologises for \u2019racist","author":"Hern Alex","year":"2020","unstructured":"Alex Hern . 2020. Twitter apologises for \u2019racist \u2019 image-cropping algorithm. The Guardian (Sept . 2020 ). https:\/\/www.theguardian.com\/technology\/2020\/sep\/21\/twitter-apologises-for-racist-image-cropping-algorithm Alex Hern. 2020. Twitter apologises for \u2019racist\u2019 image-cropping algorithm. The Guardian (Sept. 2020). https:\/\/www.theguardian.com\/technology\/2020\/sep\/21\/twitter-apologises-for-racist-image-cropping-algorithm"},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395427"},{"key":"e_1_3_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"crossref","unstructured":"Samuel\u00a0GB Johnson Greeshma Rajeev-Kumar and Frank\u00a0C Keil. 2016. Sense-making under ignorance. Cognitive psychology 89(2016) 39\u201370.  Samuel\u00a0GB Johnson Greeshma Rajeev-Kumar and Frank\u00a0C Keil. 2016. Sense-making under ignorance. Cognitive psychology 89(2016) 39\u201370.","DOI":"10.1016\/j.cogpsych.2016.06.004"},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3476046"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"crossref","unstructured":"Matthew Kay Cynthia Matuszek and Sean\u00a0A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. (2015) 3819\u20133828. https:\/\/doi.org\/10.1145\/2702123.2702520  Matthew Kay Cynthia Matuszek and Sean\u00a0A. Munson. 2015. Unequal Representation and Gender Stereotypes in Image Search Results for Occupations. (2015) 3819\u20133828. https:\/\/doi.org\/10.1145\/2702123.2702520","DOI":"10.1145\/2702123.2702520"},{"key":"e_1_3_2_2_50_1","unstructured":"Anna Kramer. 2021. Twitter\u2019s image cropping was biased so it dumped the algorithm. https:\/\/www.protocol.com\/twitter-image-cropping-algorithm-biased  Anna Kramer. 2021. Twitter\u2019s image cropping was biased so it dumped the algorithm. https:\/\/www.protocol.com\/twitter-image-cropping-algorithm-biased"},{"key":"e_1_3_2_2_51_1","volume-title":"The international encyclopedia of organizational communication","author":"Kramer W","year":"2017","unstructured":"Michael\u00a0 W Kramer . 2017. Sensemaking. The international encyclopedia of organizational communication ( 2017 ), 1\u201310. Michael\u00a0W Kramer. 2017. Sensemaking. The international encyclopedia of organizational communication (2017), 1\u201310."},{"key":"e_1_3_2_2_52_1","unstructured":"Juhi Kulshrestha Motahhare Eslami Johnnatan Messias Muhammad\u00a0Bilal Zafar Saptarshi Ghosh Krishna\u00a0P. Gummadi and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. CoRR abs\/1704.01347(2017). arxiv:1704.01347http:\/\/arxiv.org\/abs\/1704.01347  Juhi Kulshrestha Motahhare Eslami Johnnatan Messias Muhammad\u00a0Bilal Zafar Saptarshi Ghosh Krishna\u00a0P. Gummadi and Karrie Karahalios. 2017. Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. CoRR abs\/1704.01347(2017). arxiv:1704.01347http:\/\/arxiv.org\/abs\/1704.01347"},{"key":"e_1_3_2_2_53_1","volume-title":"The Landscape and Gaps in Open Source Fairness Toolkits (September 18","author":"Seng\u00a0Ah Lee Michelle","year":"2020","unstructured":"Michelle Seng\u00a0Ah Lee and Jatinder Singh . [n.d.]. The Landscape and Gaps in Open Source Fairness Toolkits (September 18 , 2020 ), biburl = https:\/\/doi.org\/10.1145\/3411764.3445261. ([n. d.]). Michelle Seng\u00a0Ah Lee and Jatinder Singh. [n.d.]. The Landscape and Gaps in Open Source Fairness Toolkits (September 18, 2020), biburl = https:\/\/doi.org\/10.1145\/3411764.3445261. ([n. d.])."},{"key":"e_1_3_2_2_54_1","unstructured":"Rosina Lippi-Green. [n.d.]. English with an Accent: Language Ideology and Discrimination in the United States. ([n. d.]).  Rosina Lippi-Green. [n.d.]. English with an Accent: Language Ideology and Discrimination in the United States. ([n. d.])."},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"crossref","unstructured":"Michael\u00a0A. Madaio Luke Stark Jennifer Wortman\u00a0Vaughan and Hanna Wallach. 2020. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. (2020) 1\u201314. https:\/\/doi.org\/10.1145\/3313831.3376445  Michael\u00a0A. Madaio Luke Stark Jennifer Wortman\u00a0Vaughan and Hanna Wallach. 2020. Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI. (2020) 1\u201314. https:\/\/doi.org\/10.1145\/3313831.3376445","DOI":"10.1145\/3313831.3376445"},{"key":"e_1_3_2_2_56_1","unstructured":"Bugcrowd\u00a0Product Marketing. 2020. Reducing Noise in Crowdsourced Security. https:\/\/www.bugcrowd.com\/blog\/reducing-noise-in-crowdsourced-security\/  Bugcrowd\u00a0Product Marketing. 2020. Reducing Noise in Crowdsourced Security. https:\/\/www.bugcrowd.com\/blog\/reducing-noise-in-crowdsourced-security\/"},{"key":"e_1_3_2_2_57_1","unstructured":"Aarian Marshall. 2021. Gig Workers Gather Their Own Data to Check the Algorithm\u2019s Math. https:\/\/www.wired.com\/story\/gig-workers-gather-data-check-algorithm-math\/  Aarian Marshall. 2021. Gig Workers Gather Their Own Data to Check the Algorithm\u2019s Math. https:\/\/www.wired.com\/story\/gig-workers-gather-data-check-algorithm-math\/"},{"key":"e_1_3_2_2_58_1","unstructured":"Milagros Miceli Julian Posada and Tianling Yang. 2021. Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?arxiv:2109.08131\u00a0[cs.HC]  Milagros Miceli Julian Posada and Tianling Yang. 2021. Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?arxiv:2109.08131\u00a0[cs.HC]"},{"key":"e_1_3_2_2_59_1","unstructured":"Safiya\u00a0Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. (2018). https:\/\/www.jstor.org\/stable\/j.ctt1pwt9w5  Safiya\u00a0Umoja Noble. 2018. Algorithms of Oppression: How Search Engines Reinforce Racism. (2018). https:\/\/www.jstor.org\/stable\/j.ctt1pwt9w5"},{"key":"e_1_3_2_2_60_1","volume-title":"Search Atlas: Visualizing Divergent Search Results Across Geopolitical Borders.","author":"Ochigame Rodrigo","year":"2021","unstructured":"Rodrigo Ochigame and Katherine Ye . 2021 . Search Atlas: Visualizing Divergent Search Results Across Geopolitical Borders. (2021), 1970\u20131983. https:\/\/doi.org\/10.1145\/3461778.3462032 Rodrigo Ochigame and Katherine Ye. 2021. Search Atlas: Visualizing Divergent Search Results Across Geopolitical Borders. (2021), 1970\u20131983. https:\/\/doi.org\/10.1145\/3461778.3462032"},{"key":"e_1_3_2_2_61_1","volume-title":"Proceedings of international conference on intelligence analysis, Vol.\u00a05. McLean, VA, USA, 2\u20134.","author":"Pirolli Peter","year":"2005","unstructured":"Peter Pirolli and Stuart Card . 2005 . The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis . In Proceedings of international conference on intelligence analysis, Vol.\u00a05. McLean, VA, USA, 2\u20134. Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol.\u00a05. McLean, VA, USA, 2\u20134."},{"key":"e_1_3_2_2_62_1","volume-title":"Rebecca\u00a0N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes.","author":"Raji Inioluwa\u00a0Deborah","year":"2020","unstructured":"Inioluwa\u00a0Deborah Raji , Andrew Smart , Rebecca\u00a0N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020 . Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. CoRR abs\/2001.00973(2020). arxiv:2001.00973http:\/\/arxiv.org\/abs\/2001.00973 Inioluwa\u00a0Deborah Raji, Andrew Smart, Rebecca\u00a0N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes. 2020. Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing. CoRR abs\/2001.00973(2020). arxiv:2001.00973http:\/\/arxiv.org\/abs\/2001.00973"},{"key":"e_1_3_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274417"},{"key":"e_1_3_2_2_64_1","unstructured":"Aja Romano. 2019. A group of YouTubers is trying to prove the site systematically demonetizes queer content. (2019). https:\/\/www.vox.com\/culture\/2019\/10\/10\/20893258\/youtube-lgbtq-censorship-demonetization-nerd-city-algorithm-report  Aja Romano. 2019. A group of YouTubers is trying to prove the site systematically demonetizes queer content. (2019). https:\/\/www.vox.com\/culture\/2019\/10\/10\/20893258\/youtube-lgbtq-censorship-demonetization-nerd-city-algorithm-report"},{"key":"e_1_3_2_2_65_1","doi-asserted-by":"crossref","unstructured":"Daniel\u00a0M Russell Mark\u00a0J Stefik Peter Pirolli and Stuart\u00a0K Card. 1993. The cost structure of sensemaking. (1993) 269\u2013276.  Daniel\u00a0M Russell Mark\u00a0J Stefik Peter Pirolli and Stuart\u00a0K Card. 1993. The cost structure of sensemaking. (1993) 269\u2013276.","DOI":"10.1145\/169059.169209"},{"key":"e_1_3_2_2_66_1","volume-title":"Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry","author":"Sandvig Christian","year":"2014","unstructured":"Christian Sandvig , Kevin Hamilton , Karrie Karahalios , and Cedric Langbort . 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry ( 2014 ). Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing algorithms: Research methods for detecting discrimination on internet platforms. Data and Discrimination: Converting Critical Concerns into Productive Inquiry (2014)."},{"key":"e_1_3_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274423"},{"key":"e_1_3_2_2_68_1","unstructured":"[\n  68\n  ]  Sarita Schoenebeck.[n.d.]. ([n. d.]). https:\/\/twitter.com\/syardi\/status\/1276587095767887874?s=20  [68] Sarita Schoenebeck.[n.d.]. ([n. d.]). https:\/\/twitter.com\/syardi\/status\/1276587095767887874?s=20"},{"key":"e_1_3_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951717738104"},{"key":"e_1_3_2_2_70_1","doi-asserted-by":"crossref","unstructured":"Andrew\u00a0D. Selbst Danah Boyd Sorelle\u00a0A. Friedler Suresh Venkatasubramanian and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. (2019) 59\u201368. https:\/\/doi.org\/10.1145\/3287560.3287598  Andrew\u00a0D. Selbst Danah Boyd Sorelle\u00a0A. Friedler Suresh Venkatasubramanian and Janet Vertesi. 2019. Fairness and Abstraction in Sociotechnical Systems. (2019) 59\u201368. https:\/\/doi.org\/10.1145\/3287560.3287598","DOI":"10.1145\/3287560.3287598"},{"key":"e_1_3_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3479577"},{"key":"e_1_3_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951720923377"},{"key":"e_1_3_2_2_73_1","article-title":"AnchorViz: Facilitating Semantic Data Exploration and Concept Discovery for Interactive Machine Learning","volume":"10","author":"Suh Jina","year":"2019","unstructured":"Jina Suh , Soroush Ghorashi , Gonzalo Ramos , Nan-Chen Chen , Steven Drucker , Johan Verwey , and Patrice Simard . 2019 . AnchorViz: Facilitating Semantic Data Exploration and Concept Discovery for Interactive Machine Learning . ACM Trans. Interact. Intell. Syst. 10 , 1, Article 7 (Aug. 2019), 38\u00a0pages. https:\/\/doi.org\/10.1145\/3241379 Jina Suh, Soroush Ghorashi, Gonzalo Ramos, Nan-Chen Chen, Steven Drucker, Johan Verwey, and Patrice Simard. 2019. AnchorViz: Facilitating Semantic Data Exploration and Concept Discovery for Interactive Machine Learning. ACM Trans. Interact. Intell. Syst. 10, 1, Article 7 (Aug. 2019), 38\u00a0pages. https:\/\/doi.org\/10.1145\/3241379","journal-title":"ACM Trans. Interact. Intell. Syst."},{"key":"e_1_3_2_2_74_1","unstructured":"Latanya Sweeney. 2013. Discrimination in Online Ad Delivery. (2013). arxiv:1301.6822\u00a0[cs.IR]  Latanya Sweeney. 2013. Discrimination in Online Ad Delivery. (2013). arxiv:1301.6822\u00a0[cs.IR]"},{"key":"e_1_3_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359130"},{"key":"e_1_3_2_2_76_1","volume-title":"Algorithmic resistance: Media practices and the politics of repair. Information","author":"Velkova Julia","year":"2019","unstructured":"Julia Velkova and Anne Kaun . 2019. Algorithmic resistance: Media practices and the politics of repair. Information , Communication & Society( 2019 ), 1\u201318. Julia Velkova and Anne Kaun. 2019. Algorithmic resistance: Media practices and the politics of repair. Information, Communication & Society(2019), 1\u201318."},{"key":"e_1_3_2_2_77_1","unstructured":"Neil Vigdor. 2019. Apple card investigated after gender discrimination complaints. The New York Times. (2019). https:\/\/www.nytimes.com\/2019\/11\/10\/business\/Apple-credit-card-investigation.html  Neil Vigdor. 2019. Apple card investigated after gender discrimination complaints. The New York Times. (2019). https:\/\/www.nytimes.com\/2019\/11\/10\/business\/Apple-credit-card-investigation.html"},{"key":"e_1_3_2_2_78_1","volume-title":"Twitter taught Microsoft\u2019s AI chatbot to be a racist asshole in less than a day. The Verge (Mar","author":"Vincent James","year":"2016","unstructured":"James Vincent . 2016. Twitter taught Microsoft\u2019s AI chatbot to be a racist asshole in less than a day. The Verge (Mar 2016 ). https:\/\/www.theverge.com\/2016\/3\/24\/11297050\/tay-microsoft-chatbot-racist James Vincent. 2016. Twitter taught Microsoft\u2019s AI chatbot to be a racist asshole in less than a day. The Verge (Mar 2016). https:\/\/www.theverge.com\/2016\/3\/24\/11297050\/tay-microsoft-chatbot-racist"},{"key":"e_1_3_2_2_79_1","volume-title":"HCI Tactics for Politics from Below: Meeting the Challenges of Smart Cities","author":"Whitney Cedric\u00a0Deslandes","unstructured":"Cedric\u00a0Deslandes Whitney , Teresa Naval , Elizabeth Quepons , Simrandeep Singh , Steven\u00a0 R Rick , and Lilly Irani . 2021. HCI Tactics for Politics from Below: Meeting the Challenges of Smart Cities . Association for Computing Machinery , New York, NY, USA . https:\/\/doi.org\/10.1145\/3411764.3445314 Cedric\u00a0Deslandes Whitney, Teresa Naval, Elizabeth Quepons, Simrandeep Singh, Steven\u00a0R Rick, and Lilly Irani. 2021. HCI Tactics for Politics from Below: Meeting the Challenges of Smart Cities. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3411764.3445314"},{"key":"e_1_3_2_2_80_1","doi-asserted-by":"crossref","unstructured":"Allison Woodruff Sarah\u00a0E. Fox Steven Rousso-Schindler and Jeffrey Warshaw. 2018. A Qualitative Exploration of Perceptions of Algorithmic Fairness. (2018) 1\u201314. https:\/\/doi.org\/10.1145\/3173574.3174230  Allison Woodruff Sarah\u00a0E. Fox Steven Rousso-Schindler and Jeffrey Warshaw. 2018. A Qualitative Exploration of Perceptions of Algorithmic Fairness. (2018) 1\u201314. https:\/\/doi.org\/10.1145\/3173574.3174230","DOI":"10.1145\/3173574.3174230"},{"key":"e_1_3_2_2_81_1","unstructured":"Meg Young Lassana Magassa and Batya Friedman. [n.d.]. Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents. Ethics and Information Technology([n. d.]).  Meg Young Lassana Magassa and Batya Friedman. [n.d.]. Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents. Ethics and Information Technology([n. d.])."},{"key":"e_1_3_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-05707-8"}],"event":{"name":"CHI '22: CHI Conference on Human Factors in Computing Systems","location":"New Orleans LA USA","acronym":"CHI '22","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491102.3517441","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491102.3517441","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491102.3517441","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:23Z","timestamp":1750188683000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491102.3517441"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,29]]},"references-count":82,"alternative-id":["10.1145\/3491102.3517441","10.1145\/3491102"],"URL":"https:\/\/doi.org\/10.1145\/3491102.3517441","relation":{},"subject":[],"published":{"date-parts":[[2022,4,29]]}}}