{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:54:00Z","timestamp":1776113640868,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":92,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T00:00:00Z","timestamp":1686528000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,12]]},"DOI":"10.1145\/3593013.3594098","type":"proceedings-article","created":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T14:40:46Z","timestamp":1686580846000},"page":"1531-1552","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Auditing Cross-Cultural Consistency of Human-Annotated Labels for Recommendation Systems"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8613-498X","authenticated-orcid":false,"given":"Rock Yuren","family":"Pang","sequence":"first","affiliation":[{"name":"University of Washington, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1751-6059","authenticated-orcid":false,"given":"Jack","family":"Cenatempo","sequence":"additional","affiliation":[{"name":"Microsoft Research, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3726-2808","authenticated-orcid":false,"given":"Franklyn","family":"Graham","sequence":"additional","affiliation":[{"name":"Microsoft Corporation \u2013 Xbox Division, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-6838-285X","authenticated-orcid":false,"given":"Bridgette","family":"Kuehn","sequence":"additional","affiliation":[{"name":"Microsoft Corporation \u2013 Xbox Division, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0571-8685","authenticated-orcid":false,"given":"Maddy","family":"Whisenant","sequence":"additional","affiliation":[{"name":"Microsoft Corporation \u2013 Xbox Division, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6553-6338","authenticated-orcid":false,"given":"Portia","family":"Botchway","sequence":"additional","affiliation":[{"name":"Microsoft Corporation \u2013 Xbox Division, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0248-566X","authenticated-orcid":false,"given":"Katie","family":"Stone Perez","sequence":"additional","affiliation":[{"name":"Microsoft Corporation \u2013 Xbox Division, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6233-8256","authenticated-orcid":false,"given":"Allison","family":"Koenecke","sequence":"additional","affiliation":[{"name":"Cornell University, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,6,12]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness. CoRR abs\/1907.13158","author":"Abdollahpouri Himan","year":"2019","unstructured":"Himan Abdollahpouri and Robin Burke . 2019. Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness. CoRR abs\/1907.13158 ( 2019 ). arXiv:1907.13158http:\/\/arxiv.org\/abs\/1907.13158 Himan Abdollahpouri and Robin Burke. 2019. Multi-stakeholder Recommendation and its Connection to Multi-sided Fairness. CoRR abs\/1907.13158 (2019). arXiv:1907.13158http:\/\/arxiv.org\/abs\/1907.13158"},{"key":"e_1_3_2_1_2_1","unstructured":"Abhishek. 2022. Video game metadata: All you need to know. https:\/\/www.gameopedia.com\/everything-about-video-game-metadata\/  Abhishek. 2022. Video game metadata: All you need to know. https:\/\/www.gameopedia.com\/everything-about-video-game-metadata\/"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v36i1.2564"},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI \u201919)","author":"Nat\u00e3","unstructured":"Nat\u00e3 M. Barbosa and Monchu Chen. 2019. Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning . In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI \u201919) . Association for Computing Machinery, New York, NY, USA, 1\u201312. https:\/\/doi.org\/10.1145\/3290605.3300773 10.1145\/3290605.3300773 Nat\u00e3 M. Barbosa and Monchu Chen. 2019. Rehumanized Crowdsourcing: A Labeling Framework Addressing Bias and Ethics in Machine Learning. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI \u201919). Association for Computing Machinery, New York, NY, USA, 1\u201312. https:\/\/doi.org\/10.1145\/3290605.3300773"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1067990.1067994"},{"key":"e_1_3_2_1_6_1","volume-title":"A Convex Framework for Fair Regression. CoRR abs\/1706.02409","author":"Berk Richard","year":"2017","unstructured":"Richard Berk , Hoda Heidari , Shahin Jabbari , Matthew Joseph , Michael J. Kearns , Jamie Morgenstern , Seth Neel , and Aaron Roth . 2017. A Convex Framework for Fair Regression. CoRR abs\/1706.02409 ( 2017 ). arXiv:1706.02409http:\/\/arxiv.org\/abs\/1706.02409 Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael J. Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. A Convex Framework for Fair Regression. CoRR abs\/1706.02409 (2017). arXiv:1706.02409http:\/\/arxiv.org\/abs\/1706.02409"},{"key":"e_1_3_2_1_7_1","volume-title":"Beyond the Chinese Face : Insights from Psychology","author":"Bond Michael Harris","unstructured":"Michael Harris Bond . 1991. Beyond the Chinese Face : Insights from Psychology . Oxford University Press , USA , Hong Kong. Michael Harris Bond. 1991. Beyond the Chinese Face : Insights from Psychology. Oxford University Press, USA, Hong Kong."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1177\/135910457000100301"},{"key":"e_1_3_2_1_9_1","first-page":"209","article-title":"Exploring Cultural Differences in Game Reception: JRPGs in Germany and Japan","volume":"4","author":"Br\u00fcckner Stefan","year":"2019","unstructured":"Stefan Br\u00fcckner , Yukiko Sato , Shuichi Kurabayashi , and Ikumi Waragai . 2019 . Exploring Cultural Differences in Game Reception: JRPGs in Germany and Japan . Transactions of the Digital Games Research Association 4 , 3 (2019), 209 \u2013 243 . Stefan Br\u00fcckner, Yukiko Sato, Shuichi Kurabayashi, and Ikumi Waragai. 2019. Exploring Cultural Differences in Game Reception: JRPGs in Germany and Japan. Transactions of the Digital Games Research Association 4, 3 (2019), 209\u2013243.","journal-title":"Transactions of the Digital Games Research Association"},{"key":"e_1_3_2_1_10_1","volume-title":"Gosling","author":"Buhrmester Michael","year":"2011","unstructured":"Michael Buhrmester , Tracy Kwang , and Samuel D . Gosling . 2011 . Amazon\u2019s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?Perspectives on Psychological Science 6, 1 (2011), 3\u20135. https:\/\/doi.org\/10.1177\/1745691610393980 arXiv:https:\/\/doi.org\/10.1177\/1745691610393980PMID: 26162106. 10.1177\/1745691610393980 Michael Buhrmester, Tracy Kwang, and Samuel D. Gosling. 2011. Amazon\u2019s Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?Perspectives on Psychological Science 6, 1 (2011), 3\u20135. https:\/\/doi.org\/10.1177\/1745691610393980 arXiv:https:\/\/doi.org\/10.1177\/1745691610393980PMID: 26162106."},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research","volume":"91","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research , Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77\u2013 91 . https:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77\u201391. https:\/\/proceedings.mlr.press\/v81\/buolamwini18a.html"},{"key":"e_1_3_2_1_12_1","volume-title":"Multisided Fairness for Recommendation. CoRR abs\/1707.00093","author":"Burke Robin","year":"2017","unstructured":"Robin Burke . 2017. Multisided Fairness for Recommendation. CoRR abs\/1707.00093 ( 2017 ). arXiv:1707.00093http:\/\/arxiv.org\/abs\/1707.00093 Robin Burke. 2017. Multisided Fairness for Recommendation. CoRR abs\/1707.00093 (2017). arXiv:1707.00093http:\/\/arxiv.org\/abs\/1707.00093"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-6419.2007.00527.x"},{"key":"e_1_3_2_1_14_1","unstructured":"Edwin Chen. 2022. 30% of Google\u2019s emotions dataset is mislabeled. https:\/\/www.surgehq.ai\/\/blog\/30-percent-of-googles-reddit-emotions-dataset-is-mislabeled  Edwin Chen. 2022. 30% of Google\u2019s emotions dataset is mislabeled. https:\/\/www.surgehq.ai\/\/blog\/30-percent-of-googles-reddit-emotions-dataset-is-mislabeled"},{"key":"e_1_3_2_1_15_1","article-title":"Bias and Debias in Recommender System: A Survey and Future Directions","volume":"41","author":"Chen Jiawei","year":"2023","unstructured":"Jiawei Chen , Hande Dong , Xiang Wang , Fuli Feng , Meng Wang , and Xiangnan He . 2023 . Bias and Debias in Recommender System: A Survey and Future Directions . ACM Trans. Inf. Syst. 41 , 3, Article 67 (feb 2023), 39 pages. https:\/\/doi.org\/10.1145\/3564284 10.1145\/3564284 Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and Debias in Recommender System: A Survey and Future Directions. ACM Trans. Inf. Syst. 41, 3, Article 67 (feb 2023), 39 pages. https:\/\/doi.org\/10.1145\/3564284","journal-title":"ACM Trans. Inf. Syst."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1177\/1461444813497555","article-title":"What\u2019s the cultural difference between the West and the East? The consumption of popular \u201ccute\u201d games in the Taiwanese market","volume":"16","author":"Chen Lai Chi","year":"2014","unstructured":"Lai Chi Chen . 2014 . What\u2019s the cultural difference between the West and the East? The consumption of popular \u201ccute\u201d games in the Taiwanese market . New Media & Society 16 , 6 (2014), 1018 \u2013 1033 . https:\/\/doi.org\/10.1177\/1461444813497555 arXiv:https:\/\/doi.org\/10.1177\/1461444813497555 10.1177\/1461444813497555 Lai Chi Chen. 2014. What\u2019s the cultural difference between the West and the East? The consumption of popular \u201ccute\u201d games in the Taiwanese market. New Media & Society 16, 6 (2014), 1018\u20131033. https:\/\/doi.org\/10.1177\/1461444813497555 arXiv:https:\/\/doi.org\/10.1177\/1461444813497555","journal-title":"New Media & Society"},{"key":"e_1_3_2_1_17_1","unstructured":"Anders Christofferson Andre James Amanda OBrien and Tom Rowland. 2022. Level up: The Future of Video Games Is bright. https:\/\/www.bain.com\/insights\/level-up-the-future-of-video-games-is-bright\/  Anders Christofferson Andre James Amanda OBrien and Tom Rowland. 2022. Level up: The Future of Video Games Is bright. https:\/\/www.bain.com\/insights\/level-up-the-future-of-video-games-is-bright\/"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1177\/1555412015591900"},{"key":"e_1_3_2_1_19_1","volume-title":"Methods to Market Mario: An Analysis of American and Japanese Preference for Control in Video Games. Ph. D. Dissertation","author":"Cook Geoffrey Michael","unstructured":"Geoffrey Michael Cook . 2009. Methods to Market Mario: An Analysis of American and Japanese Preference for Control in Video Games. Ph. D. Dissertation . University of Kansas. Geoffrey Michael Cook. 2009. Methods to Market Mario: An Analysis of American and Japanese Preference for Control in Video Games. Ph. D. Dissertation. University of Kansas."},{"key":"e_1_3_2_1_20_1","volume-title":"Conscientious Classification: A Data Scientist\u2019s Guide to Discrimination-Aware Classification. Big data 5 2","author":"Alessandro Brian","year":"2017","unstructured":"Brian d\u2019 Alessandro , Cathy O\u2019Neil , and Tom LaGatta . 2017 . Conscientious Classification: A Data Scientist\u2019s Guide to Discrimination-Aware Classification. Big data 5 2 (2017), 120\u2013134. Brian d\u2019Alessandro, Cathy O\u2019Neil, and Tom LaGatta. 2017. Conscientious Classification: A Data Scientist\u2019s Guide to Discrimination-Aware Classification. Big data 5 2 (2017), 120\u2013134."},{"key":"e_1_3_2_1_21_1","volume-title":"CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In 2022 ACM Conference on Fairness, Accountability, and Transparency","author":"D\u00edaz Mark","year":"2022","unstructured":"Mark D\u00edaz , Ian Kivlichan , Rachel Rosen , Dylan Baker , Razvan Amironesei , Vinodkumar Prabhakaran , and Emily Denton . 2022 . CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In 2022 ACM Conference on Fairness, Accountability, and Transparency ( Seoul, Republic of Korea) (FAccT \u201922). Association for Computing Machinery, New York, NY, USA, 2342\u20132351. https:\/\/doi.org\/10.1145\/3531146.3534647 10.1145\/3531146.3534647 Mark D\u00edaz, Ian Kivlichan, Rachel Rosen, Dylan Baker, Razvan Amironesei, Vinodkumar Prabhakaran, and Emily Denton. 2022. CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotation. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT \u201922). Association for Computing Machinery, New York, NY, USA, 2342\u20132351. https:\/\/doi.org\/10.1145\/3531146.3534647"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159661"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581347"},{"key":"e_1_3_2_1_24_1","volume-title":"Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research","volume":"186","author":"Ekstrand Michael D.","year":"2018","unstructured":"Michael D. Ekstrand , Mucun Tian , Ion Madrazo Azpiazu , Jennifer D. Ekstrand , Oghenemaro Anuyah , David McNeill , and Maria Soledad Pera . 2018 . All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness . In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research , Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 172\u2013 186 . https:\/\/proceedings.mlr.press\/v81\/ekstrand18b.html Michael D. Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D. Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 172\u2013186. https:\/\/proceedings.mlr.press\/v81\/ekstrand18b.html"},{"key":"e_1_3_2_1_25_1","volume-title":"Machine Learning for Multimodal Interaction","author":"Faria Arlo","unstructured":"Arlo Faria . 2006. Accent Classification for Speech Recognition . In Machine Learning for Multimodal Interaction , Steve Renals and Samy Bengio (Eds.). Springer Berlin Heidelberg, Berlin , Heidelberg , 285\u2013293. Arlo Faria. 2006. Accent Classification for Speech Recognition. In Machine Learning for Multimodal Interaction, Steve Renals and Samy Bengio (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 285\u2013293."},{"key":"e_1_3_2_1_26_1","first-page":"4","article-title":"Algorithmic logics and the construction of cultural taste of the Netflix Recommender System. Media","volume":"44","author":"Gaw Fatima","year":"2021","unstructured":"Fatima Gaw . 2021 . Algorithmic logics and the construction of cultural taste of the Netflix Recommender System. Media , Culture Society 44 , 4 (Oct. 2021), 706\u2013725. https:\/\/doi.org\/10.1177\/01634437211053767 10.1177\/01634437211053767 Fatima Gaw. 2021. Algorithmic logics and the construction of cultural taste of the Netflix Recommender System. Media, Culture Society 44, 4 (Oct. 2021), 706\u2013725. https:\/\/doi.org\/10.1177\/01634437211053767","journal-title":"Culture Society"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.wnut-1.35"},{"key":"e_1_3_2_1_28_1","first-page":"4","article-title":"The Netflix Recommender System","volume":"6","author":"Gomez-Uribe Carlos A.","year":"2015","unstructured":"Carlos A. Gomez-Uribe and Neil Hunt . 2015 . The Netflix Recommender System . ACM Transactions on Management Information Systems 6 , 4 (Dec. 2015), 1\u201319. https:\/\/doi.org\/10.1145\/2843948 10.1145\/2843948 Carlos A. Gomez-Uribe and Neil Hunt. 2015. The Netflix Recommender System. ACM Transactions on Management Information Systems 6, 4 (Dec. 2015), 1\u201319. https:\/\/doi.org\/10.1145\/2843948","journal-title":"ACM Transactions on Management Information Systems"},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Gordon Mitchell L.","year":"2004","unstructured":"Mitchell L. Gordon , Michelle S. Lam , Joon Sung Park , Kayur Patel , Jeff Hancock , Tatsunori Hashimoto , and Michael S. Bernstein . 2022. Jury Learning: Integrating Dissenting Voices into Machine Learning Models . In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems ( New Orleans, LA, USA) (CHI \u201922). Association for Computing Machinery, New York, NY, USA, Article 115, 19 pages. https:\/\/doi.org\/10.1145\/3491102.350 2004 10.1145\/3491102.3502004 Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. 2022. Jury Learning: Integrating Dissenting Voices into Machine Learning Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI \u201922). Association for Computing Machinery, New York, NY, USA, Article 115, 19 pages. https:\/\/doi.org\/10.1145\/3491102.3502004"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.23919\/JSC.2021.0029"},{"key":"e_1_3_2_1_31_1","unstructured":"Kori Hale. 2019. Google & Microsoft Banking On Africa\u2019s AI Labeling Workforce. https:\/\/www.forbes.com\/sites\/korihale\/2019\/05\/28\/google-microsoft-banking-on-africas-ai-labeling-workforce\/?sh=36e98a98541c  Kori Hale. 2019. Google & Microsoft Banking On Africa\u2019s AI Labeling Workforce. https:\/\/www.forbes.com\/sites\/korihale\/2019\/05\/28\/google-microsoft-banking-on-africas-ai-labeling-workforce\/?sh=36e98a98541c"},{"key":"e_1_3_2_1_32_1","volume-title":"Culture\u2019s Consequences: International Differences in Work-Related Values","author":"Hofstede G.","unstructured":"G. Hofstede . 1980. Culture\u2019s Consequences: International Differences in Work-Related Values . SAGE Publications, Beverly Hills, CA , and London . https:\/\/books.google.com\/books?id=hW6AAAAAMAAJ G. Hofstede. 1980. Culture\u2019s Consequences: International Differences in Work-Related Values. SAGE Publications, Beverly Hills, CA, and London. https:\/\/books.google.com\/books?id=hW6AAAAAMAAJ"},{"key":"e_1_3_2_1_33_1","volume-title":"Long-versus short-term orientation: new perspectives. Asia Pacific business review 16, 4","author":"Hofstede Geert","year":"2010","unstructured":"Geert Hofstede and Michael Minkov . 2010. Long-versus short-term orientation: new perspectives. Asia Pacific business review 16, 4 ( 2010 ), 493\u2013504. Geert Hofstede and Michael Minkov. 2010. Long-versus short-term orientation: new perspectives. Asia Pacific business review 16, 4 (2010), 493\u2013504."},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Huffaker Jordan S.","unstructured":"Jordan S. Huffaker , Jonathan K. Kummerfeld , Walter S. Lasecki , and Mark S. Ackerman . 2020. Crowdsourced Detection of Emotionally Manipulative Language . In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems ( Honolulu, HI, USA) (CHI \u201920). Association for Computing Machinery, New York, NY, USA, 1\u201314. https:\/\/doi.org\/10.1145\/3313831.3376375 10.1145\/3313831.3376375 Jordan S. Huffaker, Jonathan K. Kummerfeld, Walter S. Lasecki, and Mark S. Ackerman. 2020. Crowdsourced Detection of Emotionally Manipulative Language. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI \u201920). Association for Computing Machinery, New York, NY, USA, 1\u201314. https:\/\/doi.org\/10.1145\/3313831.3376375"},{"key":"e_1_3_2_1_35_1","first-page":"2","article-title":"Mapping Global Values","volume":"5","author":"Inglehart Ronald","year":"2006","unstructured":"Ronald Inglehart . 2006 . Mapping Global Values . Comparative Sociology 5 , 2 - 3 (2006), 115 \u2013 136. https:\/\/doi.org\/10.1163\/156913306778667401 10.1163\/156913306778667401 Ronald Inglehart. 2006. Mapping Global Values. Comparative Sociology 5, 2-3 (2006), 115 \u2013 136. https:\/\/doi.org\/10.1163\/156913306778667401","journal-title":"Comparative Sociology"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837906"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/00913367.2016.1269304"},{"key":"e_1_3_2_1_38_1","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society","author":"Kim B.","year":"2019","unstructured":"B. Kim , H. Kim , K. Kim , S. Kim , and J. Kim . 2019. Learning Not to Learn: Training Deep Neural Networks With Biased Data . In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society , Los Alamitos, CA, USA, 9004\u20139012. https:\/\/doi.org\/10.1109\/CVPR. 2019 .00922 10.1109\/CVPR.2019.00922 B. Kim, H. Kim, K. Kim, S. Kim, and J. Kim. 2019. Learning Not to Learn: Training Deep Neural Networks With Biased Data. In 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 9004\u20139012. https:\/\/doi.org\/10.1109\/CVPR.2019.00922"},{"key":"e_1_3_2_1_39_1","volume-title":"Sound Event Annotation and Detection with Less Human Effort. Ph. D. Dissertation","year":"1985","unstructured":"Kim, Bongjun. 2021. Sound Event Annotation and Detection with Less Human Effort. Ph. D. Dissertation . Northwestern University . https:\/\/doi.org\/10.2 1985 \/N2-JE7C-8R70 10.21985\/N2-JE7C-8R70 Kim, Bongjun. 2021. Sound Event Annotation and Detection with Less Human Effort. Ph. D. Dissertation. Northwestern University. https:\/\/doi.org\/10.21985\/N2-JE7C-8R70"},{"key":"e_1_3_2_1_40_1","volume-title":"The measurement of meaning. Audiovisual communication review 7","author":"Kittross John Michael","year":"1959","unstructured":"John Michael Kittross . 1959. The measurement of meaning. Audiovisual communication review 7 ( 1959 ), 154\u2013156. John Michael Kittross. 1959. The measurement of meaning. Audiovisual communication review 7 (1959), 154\u2013156."},{"key":"#cr-split#-e_1_3_2_1_41_1.1","unstructured":"Allison Koenecke Eric Giannella Robb Willer and Sharad Goel. 2023. Popular Support for Balancing Equity and Efficiency in Resource Allocation: A Case Study in Online Advertising to Increase Welfare Program Awareness. https:\/\/doi.org\/10.48550\/ARXIV.2304.08530 10.48550\/ARXIV.2304.08530"},{"key":"#cr-split#-e_1_3_2_1_41_1.2","unstructured":"Allison Koenecke Eric Giannella Robb Willer and Sharad Goel. 2023. Popular Support for Balancing Equity and Efficiency in Resource Allocation: A Case Study in Online Advertising to Increase Welfare Program Awareness. https:\/\/doi.org\/10.48550\/ARXIV.2304.08530"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1915768117"},{"key":"e_1_3_2_1_43_1","volume-title":"Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research","volume":"5664","author":"Koh Pang Wei","year":"2021","unstructured":"Pang Wei Koh , Shiori Sagawa , Henrik Marklund , Sang Michael Xie , Marvin Zhang , Akshay Balsubramani , Weihua Hu , Michihiro Yasunaga , Richard Lanas Phillips , Irena Gao , Tony Lee , Etienne David , Ian Stavness , Wei Guo , Berton Earnshaw , Imran Haque , Sara M Beery , Jure Leskovec , Anshul Kundaje , Emma Pierson , Sergey Levine , Chelsea Finn , and Percy Liang . 2021 . WILDS: A Benchmark of in-the-Wild Distribution Shifts . In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research , Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, Virtual, 5637\u2013 5664 . Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2021. WILDS: A Benchmark of in-the-Wild Distribution Shifts. In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, Virtual, 5637\u20135664."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533114"},{"key":"e_1_3_2_1_45_1","first-page":"234","volume-title":"Whittinghill","author":"Kuo Yu-Tung","year":"2022","unstructured":"Yu-Tung Kuo , Yu-Chun Kuo , and David M . Whittinghill . 2022 . Exploring the Reliability of a Cross-Cultural Model for Digital Games: A Systematic Review. International Journal of Emerging Technologies in Learning ( iJET) 17, 13 (Jul. 2022), pp. 217\u2013 234 . https:\/\/doi.org\/10.3991\/ijet.v17i13.29519 10.3991\/ijet.v17i13.29519 Yu-Tung Kuo, Yu-Chun Kuo, and David M. Whittinghill. 2022. Exploring the Reliability of a Cross-Cultural Model for Digital Games: A Systematic Review. International Journal of Emerging Technologies in Learning (iJET) 17, 13 (Jul. 2022), pp. 217\u2013234. https:\/\/doi.org\/10.3991\/ijet.v17i13.29519"},{"key":"e_1_3_2_1_46_1","volume-title":"Human-Computer Interaction \u2013 INTERACT","author":"Lai-Chong Law Effie","year":"2009","unstructured":"Effie Lai-Chong Law , Tim Gamble , and Daniel Schwarz . 2009. Gender and Cultural Differences in Perceiving Game Characters of Digital Educational Games . In Human-Computer Interaction \u2013 INTERACT 2009 , Tom Gross, Jan Gulliksen, Paula Kotz\u00e9, Lars Oestreicher, Philippe Palanque, Raquel Oliveira Prates, and Marco Winckler (Eds.). Springer Berlin Heidelberg , Berlin, Heidelberg, 149\u2013153. Effie Lai-Chong Law, Tim Gamble, and Daniel Schwarz. 2009. Gender and Cultural Differences in Perceiving Game Characters of Digital Educational Games. In Human-Computer Interaction \u2013 INTERACT 2009, Tom Gross, Jan Gulliksen, Paula Kotz\u00e9, Lars Oestreicher, Philippe Palanque, Raquel Oliveira Prates, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 149\u2013153."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"crossref","first-page":"567","DOI":"10.2307\/1860957","article-title":"The Concept of Cultural Hegemony: Problems and Possibilities","volume":"90","author":"Jackson Lears T. J.","year":"1985","unstructured":"T. J. Jackson Lears . 1985 . The Concept of Cultural Hegemony: Problems and Possibilities . The American Historical Review 90 , 3 (1985), 567 \u2013 593 . http:\/\/www.jstor.org\/stable\/1860957 T. J. Jackson Lears. 1985. The Concept of Cultural Hegemony: Problems and Possibilities. The American Historical Review 90, 3 (1985), 567\u2013593. http:\/\/www.jstor.org\/stable\/1860957","journal-title":"The American Historical Review"},{"key":"e_1_3_2_1_48_1","volume-title":"Proceedings of the Ninth Asian Conference on Machine Learning(Proceedings of Machine Learning Research","volume":"342","author":"Lecoutre Adrian","year":"2017","unstructured":"Adrian Lecoutre , Benjamin Negrevergne , and Florian Yger . 2017 . Recognizing Art Style Automatically in Painting with Deep Learning . In Proceedings of the Ninth Asian Conference on Machine Learning(Proceedings of Machine Learning Research , Vol. 77), Min-Ling Zhang and Yung-Kyun Noh (Eds.). PMLR, Yonsei University, Seoul, Republic of Korea, 327\u2013 342 . https:\/\/proceedings.mlr.press\/v77\/lecoutre17a.html Adrian Lecoutre, Benjamin Negrevergne, and Florian Yger. 2017. Recognizing Art Style Automatically in Painting with Deep Learning. In Proceedings of the Ninth Asian Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 77), Min-Ling Zhang and Yung-Kyun Noh (Eds.). PMLR, Yonsei University, Seoul, Republic of Korea, 327\u2013342. https:\/\/proceedings.mlr.press\/v77\/lecoutre17a.html"},{"key":"e_1_3_2_1_49_1","volume-title":"ISMIR","author":"Lee Jin Ha","year":"2005","unstructured":"Jin Ha Lee , J. Stephen Downie , and Sally Jo Cunningham . 2005. Challenges in Cross-Cultural\/Multilingual Music Information Seeking . In ISMIR 2005 , 6th International Conference on Music Information Retrieval, London, UK, 11-15 September 2005, Proceedings. Transactions of the International Society for Music Information Retrieval , London, UK, 1\u20137. http:\/\/ismir2005.ismir.net\/proceedings\/1100.pdf Jin Ha Lee, J. Stephen Downie, and Sally Jo Cunningham. 2005. Challenges in Cross-Cultural\/Multilingual Music Information Seeking. In ISMIR 2005, 6th International Conference on Music Information Retrieval, London, UK, 11-15 September 2005, Proceedings. Transactions of the International Society for Music Information Retrieval, London, UK, 1\u20137. http:\/\/ismir2005.ismir.net\/proceedings\/1100.pdf"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462814"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1993.10476368"},{"key":"e_1_3_2_1_52_1","first-page":"310","article-title":"Gender and Cultural Differences in Game-Based Learning Experiences","volume":"15","author":"Lukosch Heide","year":"2017","unstructured":"Heide Lukosch , Shalini Kurapati , Daan Groen , and Alexander Verbraeck . 2017 . Gender and Cultural Differences in Game-Based Learning Experiences . Electronic Journal of e-Learning 15 , 4 (2017), pp 310 \u2013 319 . Heide Lukosch, Shalini Kurapati, Daan Groen, and Alexander Verbraeck. 2017. Gender and Cultural Differences in Game-Based Learning Experiences. Electronic Journal of e-Learning 15, 4 (2017), pp310\u2013319.","journal-title":"Electronic Journal of e-Learning"},{"key":"e_1_3_2_1_53_1","volume-title":"Proceedings of the National Institute of Science of India","volume":"12","author":"Mahalanobis Prasanta Chandra","year":"1936","unstructured":"Prasanta Chandra Mahalanobis . 1936 . On the generalised distance in statistics . In Proceedings of the National Institute of Science of India , Vol. 12 . National Institute of Science of India, 49\u201355. Prasanta Chandra Mahalanobis. 1936. On the generalised distance in statistics. In Proceedings of the National Institute of Science of India, Vol. 12. National Institute of Science of India, 49\u201355."},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"#cr-split#-e_1_3_2_1_55_1.1","doi-asserted-by":"crossref","unstructured":"Rishabh Mehrotra Ashton Anderson Fernando Diaz Amit Sharma Hanna Wallach and Emine Yilmaz. 2017. Auditing Search Engines for Differential Satisfaction Across Demographics. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth Australia) (WWW '17 Companion). International World Wide Web Conferences Steering Committee Republic and Canton of Geneva CHE 626-633. https:\/\/doi.org\/10.1145\/3041021.3054197 10.1145\/3041021.3054197","DOI":"10.1145\/3041021.3054197"},{"key":"#cr-split#-e_1_3_2_1_55_1.2","doi-asserted-by":"crossref","unstructured":"Rishabh Mehrotra Ashton Anderson Fernando Diaz Amit Sharma Hanna Wallach and Emine Yilmaz. 2017. Auditing Search Engines for Differential Satisfaction Across Demographics. In Proceedings of the 26th International Conference on World Wide Web Companion (Perth Australia) (WWW '17 Companion). International World Wide Web Conferences Steering Committee Republic and Canton of Geneva CHE 626-633. https:\/\/doi.org\/10.1145\/3041021.3054197","DOI":"10.1145\/3041021.3054197"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-042720-125902"},{"key":"e_1_3_2_1_57_1","volume-title":"Proceedings of the International AAAI Conference on Web and Social Media 9, 1 (Aug.","author":"Mitra Tanushree","year":"2021","unstructured":"Tanushree Mitra and Eric Gilbert . 2021 . CREDBANK: A Large-Scale Social Media Corpus With Associated Credibility Annotations . Proceedings of the International AAAI Conference on Web and Social Media 9, 1 (Aug. 2021), 258\u2013267. https:\/\/doi.org\/10.1609\/icwsm.v9i1.14625 10.1609\/icwsm.v9i1.14625 Tanushree Mitra and Eric Gilbert. 2021. CREDBANK: A Large-Scale Social Media Corpus With Associated Credibility Annotations. Proceedings of the International AAAI Conference on Web and Social Media 9, 1 (Aug. 2021), 258\u2013267. https:\/\/doi.org\/10.1609\/icwsm.v9i1.14625"},{"key":"e_1_3_2_1_58_1","unstructured":"Madhumita Murgia. 2019. AI\u2019s new workforce: the data-labelling industry spreads globally. https:\/\/www.ft.com\/content\/56dde36c-aa40-11e9-984c-fac8325aaa04  Madhumita Murgia. 2019. AI\u2019s new workforce: the data-labelling industry spreads globally. https:\/\/www.ft.com\/content\/56dde36c-aa40-11e9-984c-fac8325aaa04"},{"key":"e_1_3_2_1_59_1","volume-title":"Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. CoRR abs\/2103.14749","author":"Northcutt Curtis G.","year":"2021","unstructured":"Curtis G. Northcutt , Anish Athalye , and Jonas Mueller . 2021. Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. CoRR abs\/2103.14749 ( 2021 ). arXiv:2103.14749https:\/\/arxiv.org\/abs\/2103.14749 Curtis G. Northcutt, Anish Athalye, and Jonas Mueller. 2021. Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks. CoRR abs\/2103.14749 (2021). arXiv:2103.14749https:\/\/arxiv.org\/abs\/2103.14749"},{"key":"e_1_3_2_1_60_1","first-page":"1","article-title":"A Re-Inquiry of Hofstede\u2019s Cultural Dimensions: A Call for 21st Century Cross-Cultural Research","volume":"18","author":"Orr Linda M","year":"2008","unstructured":"Linda M Orr and William J Hauser . 2008 . A Re-Inquiry of Hofstede\u2019s Cultural Dimensions: A Call for 21st Century Cross-Cultural Research . Marketing Management Journal 18 , 2 (2008), 1 \u2013 19 . Linda M Orr and William J Hauser. 2008. A Re-Inquiry of Hofstede\u2019s Cultural Dimensions: A Call for 21st Century Cross-Cultural Research. Marketing Management Journal 18, 2 (2008), 1\u201319.","journal-title":"Marketing Management Journal"},{"key":"e_1_3_2_1_61_1","unstructured":"Jennifer Pak. 2021. The human labor behind artificial intelligence. https:\/\/www.marketplace.org\/2021\/05\/04\/the-human-labor-behind-artificial-intelligence\/  Jennifer Pak. 2021. The human labor behind artificial intelligence. https:\/\/www.marketplace.org\/2021\/05\/04\/the-human-labor-behind-artificial-intelligence\/"},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mph024"},{"key":"e_1_3_2_1_63_1","volume-title":"Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics","author":"Parmar Mihir","year":"2023","unstructured":"Mihir Parmar , Swaroop Mishra , Mor Geva , and Chitta Baral . 2023 . Don\u2019t Blame the Annotator: Bias Already Starts in the Annotation Instructions . In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics , Dubrovnik, Croatia, 1779\u20131789. https:\/\/aclanthology.org\/ 2023.eacl-main.130 Mihir Parmar, Swaroop Mishra, Mor Geva, and Chitta Baral. 2023. Don\u2019t Blame the Annotator: Bias Already Starts in the Annotation Instructions. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Dubrovnik, Croatia, 1779\u20131789. https:\/\/aclanthology.org\/2023.eacl-main.130"},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0695-z"},{"key":"e_1_3_2_1_65_1","first-page":"4","article-title":"Reputation as a sufficient condition for data quality on Amazon Mechanical Turk","volume":"46","author":"Peer Eyal","year":"2013","unstructured":"Eyal Peer , Joachim Vosgerau , and Alessandro Acquisti . 2013 . Reputation as a sufficient condition for data quality on Amazon Mechanical Turk . Behavior Research Methods 46 , 4 (Dec. 2013), 1023\u20131031. https:\/\/doi.org\/10.3758\/s13428-013-0434-y 10.3758\/s13428-013-0434-y Eyal Peer, Joachim Vosgerau, and Alessandro Acquisti. 2013. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods 46, 4 (Dec. 2013), 1023\u20131031. https:\/\/doi.org\/10.3758\/s13428-013-0434-y","journal-title":"Behavior Research Methods"},{"key":"e_1_3_2_1_66_1","volume-title":"Amazon launches \u2018Themes","author":"Perez Sarah","year":"2017","unstructured":"Sarah Perez . 2017. Amazon launches \u2018Themes ,\u2019 a feature for filtering Customer Reviews by popular terms. https:\/\/techcrunch.com\/ 2017 \/08\/04\/amazon-launches-themes-a-feature-for-filtering-customer-reviews-by-popular-terms\/ Sarah Perez. 2017. Amazon launches \u2018Themes,\u2019 a feature for filtering Customer Reviews by popular terms. https:\/\/techcrunch.com\/2017\/08\/04\/amazon-launches-themes-a-feature-for-filtering-customer-reviews-by-popular-terms\/"},{"key":"e_1_3_2_1_67_1","first-page":"315","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Peskov Denis","year":"2021","unstructured":"Denis Peskov , Viktor Hangya , Jordan Boyd-Graber , and Alexander Fraser . 2021 . Adapting Entities across Languages and Cultures . In Findings of the Association for Computational Linguistics: EMNLP 2021 . Association for Computational Linguistics, Punta Cana, Dominican Republic, 3725\u20133750. https:\/\/doi.org\/10. 18653\/v1\/2021.findings-emnlp. 315 10.18653\/v1 Denis Peskov, Viktor Hangya, Jordan Boyd-Graber, and Alexander Fraser. 2021. Adapting Entities across Languages and Cultures. In Findings of the Association for Computational Linguistics: EMNLP 2021. Association for Computational Linguistics, Punta Cana, Dominican Republic, 3725\u20133750. https:\/\/doi.org\/10.18653\/v1\/2021.findings-emnlp.315"},{"key":"e_1_3_2_1_68_1","first-page":"1","article-title":"Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics","volume":"9","author":"Posch Lisa","year":"2022","unstructured":"Lisa Posch , Arnim Bleier , Fabian Fl\u00f6ck , Clemens M. Lechner , Katharina Kinder-Kurlanda , Denis Helic , and Markus Strohmaier . 2022 . Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics . Human Computation 9 , 1 (Aug. 2022), 22\u201357. https:\/\/doi.org\/10.15346\/hc.v9i1.106 10.15346\/hc.v9i1.106 Lisa Posch, Arnim Bleier, Fabian Fl\u00f6ck, Clemens M. Lechner, Katharina Kinder-Kurlanda, Denis Helic, and Markus Strohmaier. 2022. Characterizing the Global Crowd Workforce: A Cross-Country Comparison of Crowdworker Demographics. Human Computation 9, 1 (Aug. 2022), 22\u201357. https:\/\/doi.org\/10.15346\/hc.v9i1.106","journal-title":"Human Computation"},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-011-9115-7"},{"key":"e_1_3_2_1_70_1","volume-title":"Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing","author":"Reinecke Katharina","unstructured":"Katharina Reinecke and Krzysztof Z. Gajos . 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples . In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing ( Vancouver, BC, Canada) (CSCW \u201915). Association for Computing Machinery, New York, NY, USA, 1364\u20131378. https:\/\/doi.org\/10.1145\/2675133.2675246 10.1145\/2675133.2675246 Katharina Reinecke and Krzysztof Z. Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (Vancouver, BC, Canada) (CSCW \u201915). Association for Computing Machinery, New York, NY, USA, 1364\u20131378. https:\/\/doi.org\/10.1145\/2675133.2675246"},{"key":"e_1_3_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/192844.192905"},{"key":"e_1_3_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/70.1.41"},{"key":"e_1_3_2_1_73_1","volume-title":"CHI \u201910 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI EA \u201910)","author":"Ross Joel","unstructured":"Joel Ross , Lilly Irani , M. Six Silberman , Andrew Zaldivar , and Bill Tomlinson . 2010. Who Are the Crowdworkers? Shifting Demographics in Mechanical Turk . In CHI \u201910 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI EA \u201910) . Association for Computing Machinery , New York, NY, USA , 2863\u20132872. https:\/\/doi.org\/10.1145\/1753846.1753873 10.1145\/1753846.1753873 Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who Are the Crowdworkers? Shifting Demographics in Mechanical Turk. In CHI \u201910 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI EA \u201910). Association for Computing Machinery, New York, NY, USA, 2863\u20132872. https:\/\/doi.org\/10.1145\/1753846.1753873"},{"key":"e_1_3_2_1_74_1","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1177\/1470595804047820","article-title":"Contextual Influences on Culture Research: Shifting Assumptions for New Workplace Realities","volume":"4","author":"Sackmann Sonja A.","year":"2004","unstructured":"Sonja A. Sackmann and Margaret E. Phillips . 2004 . Contextual Influences on Culture Research: Shifting Assumptions for New Workplace Realities . International Journal of Cross Cultural Management 4 , 3 (2004), 370 \u2013 390 . https:\/\/doi.org\/10.1177\/1470595804047820 arXiv:https:\/\/doi.org\/10.1177\/1470595804047820 10.1177\/1470595804047820 Sonja A. Sackmann and Margaret E. Phillips. 2004. Contextual Influences on Culture Research: Shifting Assumptions for New Workplace Realities. International Journal of Cross Cultural Management 4, 3 (2004), 370\u2013390. https:\/\/doi.org\/10.1177\/1470595804047820 arXiv:https:\/\/doi.org\/10.1177\/1470595804047820","journal-title":"International Journal of Cross Cultural Management"},{"key":"e_1_3_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460112.3471954"},{"key":"e_1_3_2_1_76_1","volume-title":"Collaborative Filtering Recommender Systems","author":"Schafer J. Ben","year":"2079","unstructured":"J. Ben Schafer , Dan Frankowski , Jon Herlocker , and Shilad Sen . 2007. Collaborative Filtering Recommender Systems . Springer Berlin Heidelberg , Berlin, Heidelberg , 291\u2013324. https:\/\/doi.org\/10.1007\/978-3-540-7 2079 -9_9 10.1007\/978-3-540-72079-9_9 J. Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative Filtering Recommender Systems. Springer Berlin Heidelberg, Berlin, Heidelberg, 291\u2013324. https:\/\/doi.org\/10.1007\/978-3-540-72079-9_9"},{"key":"e_1_3_2_1_77_1","volume-title":"Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics","author":"Snow Rion","year":"2008","unstructured":"Rion Snow , Brendan O\u2019Connor , Daniel Jurafsky , and Andrew Ng . 2008 . Cheap and Fast \u2013 But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks . In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics , Honolulu, Hawaii, 254\u2013263. https:\/\/aclanthology.org\/D08-1027 Rion Snow, Brendan O\u2019Connor, Daniel Jurafsky, and Andrew Ng. 2008. Cheap and Fast \u2013 But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Honolulu, Hawaii, 254\u2013263. https:\/\/aclanthology.org\/D08-1027"},{"key":"e_1_3_2_1_78_1","unstructured":"Steam. 2023. Introducing steam tags a powerful new way to shop for games. https:\/\/store.steampowered.com\/tag\/  Steam. 2023. Introducing steam tags a powerful new way to shop for games. https:\/\/store.steampowered.com\/tag\/"},{"key":"e_1_3_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533101"},{"key":"e_1_3_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijforecast.2014.06.001"},{"key":"e_1_3_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijresmar.2016.05.003"},{"key":"e_1_3_2_1_82_1","volume-title":"Do Artifacts Have Politics?Daedalus 109, 1","author":"Winner Langdon","year":"1980","unstructured":"Langdon Winner . 1980. Do Artifacts Have Politics?Daedalus 109, 1 ( 1980 ), 121\u2013136. http:\/\/www.jstor.org\/stable\/20024652 Langdon Winner. 1980. Do Artifacts Have Politics?Daedalus 109, 1 (1980), 121\u2013136. http:\/\/www.jstor.org\/stable\/20024652"},{"key":"#cr-split#-e_1_3_2_1_83_1.1","doi-asserted-by":"crossref","unstructured":"Ellery Wulczyn Nithum Thain and Lucas Dixon. 2017. Ex Machina: Personal Attacks Seen at Scale. In Proceedings of the 26th International Conference on World Wide Web (Perth Australia) (WWW '17). International World Wide Web Conferences Steering Committee Republic and Canton of Geneva CHE 1391-1399. https:\/\/doi.org\/10.1145\/3038912.3052591 10.1145\/3038912.3052591","DOI":"10.1145\/3038912.3052591"},{"key":"#cr-split#-e_1_3_2_1_83_1.2","doi-asserted-by":"crossref","unstructured":"Ellery Wulczyn Nithum Thain and Lucas Dixon. 2017. Ex Machina: Personal Attacks Seen at Scale. In Proceedings of the 26th International Conference on World Wide Web (Perth Australia) (WWW '17). International World Wide Web Conferences Steering Committee Republic and Canton of Geneva CHE 1391-1399. https:\/\/doi.org\/10.1145\/3038912.3052591","DOI":"10.1145\/3038912.3052591"},{"key":"e_1_3_2_1_84_1","unstructured":"Xbox. 2023. Xbox Ambassadors. https:\/\/ambassadors.microsoft.com\/xbox  Xbox. 2023. Xbox Ambassadors. https:\/\/ambassadors.microsoft.com\/xbox"},{"key":"e_1_3_2_1_85_1","volume-title":"Proceedings of the 8th International Conference on the Foundations of Digital Games, FDG 2013","author":"Jos\u00e9","year":"2013","unstructured":"Jos\u00e9 P. Zagal and Noriko Tomuro. 2013. Cultural differences in game appreciation: A study of player game reviews . In Proceedings of the 8th International Conference on the Foundations of Digital Games, FDG 2013 , Chania, Crete, Greece , May 14-17, 2013 , Georgios N. Yannakakis, Espen Aarseth, Kristine J\u00f8rgensen, and James C. Lester (Eds.). Society for the Advancement of the Science of Digital Games, Chania, Greece, 86\u201393. http:\/\/www.fdg2013.org\/program\/papers\/paper12_zagal_tomuro.pdf Jos\u00e9 P. Zagal and Noriko Tomuro. 2013. Cultural differences in game appreciation: A study of player game reviews. In Proceedings of the 8th International Conference on the Foundations of Digital Games, FDG 2013, Chania, Crete, Greece, May 14-17, 2013, Georgios N. Yannakakis, Espen Aarseth, Kristine J\u00f8rgensen, and James C. Lester (Eds.). Society for the Advancement of the Science of Digital Games, Chania, Greece, 86\u201393. http:\/\/www.fdg2013.org\/program\/papers\/paper12_zagal_tomuro.pdf"},{"key":"e_1_3_2_1_86_1","volume-title":"Cross-cultural patterns in mobile playtime: an analysis of 118 billion hours of human data. Scientific reports 13, 1 (January","author":"Zendle David","year":"2023","unstructured":"David Zendle , Catherine Flick , Darel Halgarth , Nick Ballou , Simon Demediuk , and Anders Drachen . 2023. Cross-cultural patterns in mobile playtime: an analysis of 118 billion hours of human data. Scientific reports 13, 1 (January 2023 ), 386. https:\/\/doi.org\/10.1038\/s41598-022-26730-w 10.1038\/s41598-022-26730-w David Zendle, Catherine Flick, Darel Halgarth, Nick Ballou, Simon Demediuk, and Anders Drachen. 2023. Cross-cultural patterns in mobile playtime: an analysis of 118 billion hours of human data. Scientific reports 13, 1 (January 2023), 386. https:\/\/doi.org\/10.1038\/s41598-022-26730-w"},{"key":"e_1_3_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498427"},{"key":"e_1_3_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412210"},{"key":"e_1_3_2_1_89_1","doi-asserted-by":"crossref","unstructured":"James Zou and Londa Schiebinger. 2018. AI can be sexist and racist\u2014it\u2019s time to make it fair.  James Zou and Londa Schiebinger. 2018. AI can be sexist and racist\u2014it\u2019s time to make it fair.","DOI":"10.1038\/d41586-018-05707-8"}],"event":{"name":"FAccT '23: the 2023 ACM Conference on Fairness, Accountability, and Transparency","location":"Chicago IL USA","acronym":"FAccT '23"},"container-title":["2023 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593013.3594098","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3593013.3594098","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:19Z","timestamp":1750178239000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593013.3594098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,12]]},"references-count":92,"alternative-id":["10.1145\/3593013.3594098","10.1145\/3593013"],"URL":"https:\/\/doi.org\/10.1145\/3593013.3594098","relation":{},"subject":[],"published":{"date-parts":[[2023,6,12]]},"assertion":[{"value":"2023-06-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}