{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T05:19:32Z","timestamp":1776489572038,"version":"3.51.2"},"publisher-location":"New York, NY, USA","reference-count":72,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,21]]},"DOI":"10.1145\/3461702.3462536","type":"proceedings-article","created":{"date-parts":[[2021,7,31]],"date-time":"2021-07-31T01:21:38Z","timestamp":1627694498000},"page":"122-133","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":102,"title":["Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases"],"prefix":"10.1145","author":[{"given":"Wei","family":"Guo","sequence":"first","affiliation":[{"name":"George Washington University, Washington, DC, USA"}]},{"given":"Aylin","family":"Caliskan","sequence":"additional","affiliation":[{"name":"George Washington University and Institute for Data, Democracy & Politics, Washington, DC, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,7,30]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","first-page":"S7","DOI":"10.1016\/j.jadohealth.2014.10.016","article-title":"13. Intersecting Identities in Black Gay and Bisexual Young Men: A Potential Framework for HIV Risk","volume":"56","author":"Arrington-Sanders Renata","year":"2015","unstructured":"Renata Arrington-Sanders , Jessica Oidtman , Anthony Morgan , Gary Harper , Maria Trent , and J Dennis Fortenberry . 2015 . 13. Intersecting Identities in Black Gay and Bisexual Young Men: A Potential Framework for HIV Risk . Journal of Adolescent Health , Vol. 56 , 2 (2015), S7 -- S8 . Renata Arrington-Sanders, Jessica Oidtman, Anthony Morgan, Gary Harper, Maria Trent, and J Dennis Fortenberry. 2015. 13. Intersecting Identities in Black Gay and Bisexual Young Men: A Potential Framework for HIV Risk. Journal of Adolescent Health, Vol. 56, 2 (2015), S7--S8.","journal-title":"Journal of Adolescent Health"},{"key":"e_1_3_2_1_2_1","volume-title":"On the comparability of pre-trained language models. arXiv preprint arXiv:2001.00781","author":"A\u00dfenmacher Matthias","year":"2020","unstructured":"Matthias A\u00dfenmacher and Christian Heumann . 2020. On the comparability of pre-trained language models. arXiv preprint arXiv:2001.00781 ( 2020 ). Matthias A\u00dfenmacher and Christian Heumann. 2020. On the comparability of pre-trained language models. arXiv preprint arXiv:2001.00781 (2020)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_3_2_1_4_1","volume-title":"Hal Daum\u00e9 III, and Hanna Wallach","author":"Blodgett Su Lin","year":"2020","unstructured":"Su Lin Blodgett , Solon Barocas , Hal Daum\u00e9 III, and Hanna Wallach . 2020 . Language (Technology) is Power : A Critical Survey of\" Bias\" in NLP. arXiv preprint arXiv:2005.14050 (2020). Su Lin Blodgett, Solon Barocas, Hal Daum\u00e9 III, and Hanna Wallach. 2020. Language (Technology) is Power: A Critical Survey of\" Bias\" in NLP. arXiv preprint arXiv:2005.14050 (2020)."},{"key":"e_1_3_2_1_5_1","volume-title":"Morphosyntactic tagging with a meta-bilstm model over context sensitive token encodings. arXiv preprint arXiv:1805.08237","author":"Bohnet Bernd","year":"2018","unstructured":"Bernd Bohnet , Ryan McDonald , Goncalo Simoes , Daniel Andor , Emily Pitler , and Joshua Maynez . 2018. Morphosyntactic tagging with a meta-bilstm model over context sensitive token encodings. arXiv preprint arXiv:1805.08237 ( 2018 ). Bernd Bohnet, Ryan McDonald, Goncalo Simoes, Daniel Andor, Emily Pitler, and Joshua Maynez. 2018. Morphosyntactic tagging with a meta-bilstm model over context sensitive token encodings. arXiv preprint arXiv:1805.08237 (2018)."},{"key":"e_1_3_2_1_6_1","unstructured":"Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357.  Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in neural information processing systems. 4349--4357."},{"key":"e_1_3_2_1_7_1","unstructured":"Michael Borenstein Larry Hedges and Hannah Rothstein. [n.d.]. Meta-analysis: Fixed effect vs. random effects. ( [n. d.]).  Michael Borenstein Larry Hedges and Hannah Rothstein. [n.d.]. Meta-analysis: Fixed effect vs. random effects. ( [n. d.])."},{"key":"e_1_3_2_1_8_1","unstructured":"Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell etal 2020. Language Models are Few-Shot Learners. arXiv preprint arXiv:2005.14165 (2020).  Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language Models are Few-Shot Learners. arXiv preprint arXiv:2005.14165 (2020)."},{"key":"e_1_3_2_1_9_1","volume-title":"Conference on fairness, accountability and transparency. 77--91","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru . 2018 . Gender shades: Intersectional accuracy disparities in commercial gender classification . In Conference on fairness, accountability and transparency. 77--91 . Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77--91."},{"key":"e_1_3_2_1_10_1","unstructured":"Angel Alexander Cabrera Minsuk Kahng Fred Hohman Jamie Morgenstern and Duen Horng Chau. [n.d.]. DISCOVERY OF INTERSECTIONAL BIAS IN MACHINE LEARNING USING AUTOMATIC SUBGROUP GENERATION. ( [n. d.]).  Angel Alexander Cabrera Minsuk Kahng Fred Hohman Jamie Morgenstern and Duen Horng Chau. [n.d.]. DISCOVERY OF INTERSECTIONAL BIAS IN MACHINE LEARNING USING AUTOMATIC SUBGROUP GENERATION. ( [n. d.])."},{"key":"e_1_3_2_1_11_1","volume-title":"Science","volume":"356","author":"Caliskan Aylin","year":"2017","unstructured":"Aylin Caliskan , Joanna J Bryson , and Arvind Narayanan . 2017 . Semantics derived automatically from language corpora contain human-like biases . Science , Vol. 356 , 6334 (2017), 183--186. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, Vol. 356, 6334 (2017), 183--186."},{"key":"e_1_3_2_1_12_1","volume-title":"AI now 2017 report","author":"Campolo Alex","year":"2017","unstructured":"Alex Campolo , Madelyn Sanfilippo , Meredith Whittaker , and Kate Crawford . 2017. AI now 2017 report . AI Now Institute at New York University ( 2017 ). Alex Campolo, Madelyn Sanfilippo, Meredith Whittaker, and Kate Crawford. 2017. AI now 2017 report. AI Now Institute at New York University (2017)."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797620963619"},{"key":"e_1_3_2_1_14_1","volume-title":"One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005","author":"Chelba Ciprian","year":"2013","unstructured":"Ciprian Chelba , Tomas Mikolov , Mike Schuster , Qi Ge , Thorsten Brants , Phillipp Koehn , and Tony Robinson . 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 ( 2013 ). Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005 (2013)."},{"key":"e_1_3_2_1_15_1","unstructured":"Kimberle Crenshaw. 1989. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine feminist theory and antiracist politics. u. Chi. Legal f. (1989) 139.  Kimberle Crenshaw. 1989. Demarginalizing the intersection of race and sex: A black feminist critique of antidiscrimination doctrine feminist theory and antiracist politics. u. Chi. Legal f. (1989) 139."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287572"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cct.2006.04.004"},{"key":"e_1_3_2_1_18_1","volume-title":"Meta-analysis in clinical trials. Controlled clinical trials","author":"DerSimonian Rebecca","year":"1986","unstructured":"Rebecca DerSimonian and Nan Laird . 1986. Meta-analysis in clinical trials. Controlled clinical trials , Vol. 7 , 3 ( 1986 ), 177--188. Rebecca DerSimonian and Nan Laird. 1986. Meta-analysis in clinical trials. Controlled clinical trials, Vol. 7, 3 (1986), 177--188."},{"key":"e_1_3_2_1_19_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_3_2_1_20_1","volume-title":"Understanding back-translation at scale. arXiv preprint arXiv:1808.09381","author":"Edunov Sergey","year":"2018","unstructured":"Sergey Edunov , Myle Ott , Michael Auli , and David Grangier . 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381 ( 2018 ). Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381 (2018)."},{"key":"e_1_3_2_1_21_1","volume-title":"How contextual are contextualized word representations? comparing the geometry of BERT, ELMo, and GPT-2 embeddings. arXiv preprint arXiv:1909.00512","author":"Ethayarajh Kawin","year":"2019","unstructured":"Kawin Ethayarajh . 2019. How contextual are contextualized word representations? comparing the geometry of BERT, ELMo, and GPT-2 embeddings. arXiv preprint arXiv:1909.00512 ( 2019 ). Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of BERT, ELMo, and GPT-2 embeddings. arXiv preprint arXiv:1909.00512 (2019)."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1166"},{"key":"e_1_3_2_1_23_1","volume-title":"A synopsis of linguistic theory","author":"Firth John R","year":"1930","unstructured":"John R Firth . 1957. A synopsis of linguistic theory , 1930 --1955. Studies in linguistic analysis (1957). John R Firth. 1957. A synopsis of linguistic theory, 1930--1955. Studies in linguistic analysis (1957)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09548-1"},{"key":"e_1_3_2_1_25_1","volume-title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling. arXiv preprint arXiv:2101.00027","author":"Gao Leo","year":"2020","unstructured":"Leo Gao , Stella Biderman , Sid Black , Laurence Golding , Travis Hoppe , Charles Foster , Jason Phang , Horace He , Anish Thite , Noa Nabeshima , 2020 . The Pile: An 800GB Dataset of Diverse Text for Language Modeling. arXiv preprint arXiv:2101.00027 (2020). Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020. The Pile: An 800GB Dataset of Diverse Text for Language Modeling. arXiv preprint arXiv:2101.00027 (2020)."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1720347115"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1177\/0361684312464203"},{"key":"e_1_3_2_1_28_1","volume-title":"Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862","author":"Gonen Hila","year":"2019","unstructured":"Hila Gonen and Yoav Goldberg . 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862 ( 2019 ). Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862 (2019)."},{"key":"e_1_3_2_1_29_1","volume-title":"Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review","author":"Greenwald Anthony G","year":"1995","unstructured":"Anthony G Greenwald and Mahzarin R Banaji . 1995. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review , Vol. 102 , 1 ( 1995 ), 4. Anthony G Greenwald and Mahzarin R Banaji. 1995. Implicit social cognition: attitudes, self-esteem, and stereotypes. Psychological review, Vol. 102, 1 (1995), 4."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.74.6.1464"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.85.2.197"},{"key":"e_1_3_2_1_32_1","volume-title":"When multiplication doesn't equal quick addition: Examining intersectionality as a research paradigm. Perspectives on politics","author":"Hancock Ange-Marie","year":"2007","unstructured":"Ange-Marie Hancock . 2007. When multiplication doesn't equal quick addition: Examining intersectionality as a research paradigm. Perspectives on politics , Vol. 5 , 1 ( 2007 ), 63--79. Ange-Marie Hancock. 2007. When multiplication doesn't equal quick addition: Examining intersectionality as a research paradigm. Perspectives on politics, Vol. 5, 1 (2007), 63--79."},{"key":"e_1_3_2_1_33_1","volume-title":"The meaning of difference: Gender theory, postmodernism, and psychology. American psychologist","author":"Hare-Mustin Rachel T","year":"1988","unstructured":"Rachel T Hare-Mustin and Jeanne Marecek . 1988. The meaning of difference: Gender theory, postmodernism, and psychology. American psychologist , Vol. 43 , 6 ( 1988 ), 455. Rachel T Hare-Mustin and Jeanne Marecek. 1988. The meaning of difference: Gender theory, postmodernism, and psychology. American psychologist, Vol. 43, 6 (1988), 455."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1080\/00437956.1954.11659520"},{"key":"e_1_3_2_1_35_1","volume-title":"35th annual meeting of the association for computational linguistics and 8th conference of the european","author":"Hatzivassiloglou Vasileios","unstructured":"Vasileios Hatzivassiloglou and Kathleen McKeown . 1997. Predicting the semantic orientation of adjectives . In 35th annual meeting of the association for computational linguistics and 8th conference of the european chapter of the association for computational linguistics. 174--181. Vasileios Hatzivassiloglou and Kathleen McKeown. 1997. Predicting the semantic orientation of adjectives. In 35th annual meeting of the association for computational linguistics and 8th conference of the european chapter of the association for computational linguistics. 174--181."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.93.2.388"},{"key":"e_1_3_2_1_37_1","volume-title":"Statistical methods for meta-analysis","author":"Hedges Larry V","unstructured":"Larry V Hedges and Ingram Olkin . 2014. Statistical methods for meta-analysis . Academic press . Larry V Hedges and Ingram Olkin. 2014. Statistical methods for meta-analysis .Academic press."},{"key":"e_1_3_2_1_38_1","volume-title":"Fixed-and random-effects models in meta-analysis. Psychological methods","author":"Hedges Larry V","year":"1998","unstructured":"Larry V Hedges and Jack L Vevea . 1998. Fixed-and random-effects models in meta-analysis. Psychological methods , Vol. 3 , 4 ( 1998 ), 486. Larry V Hedges and Jack L Vevea. 1998. Fixed-and random-effects models in meta-analysis. Psychological methods, Vol. 3, 4 (1998), 486."},{"key":"e_1_3_2_1_39_1","volume-title":"Long short-term memory. Neural computation","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and J\u00fcrgen Schmidhuber . 1997. Long short-term memory. Neural computation , Vol. 9 , 8 ( 1997 ), 1735--1780. Sepp Hochreiter and J\u00fcrgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11199-008-9405-7"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.487"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1471-6402.1989.tb01011.x"},{"key":"e_1_3_2_1_43_1","volume-title":"Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing .","author":"Kurita Keita","year":"2019","unstructured":"Keita Kurita , Nidhi Vyas , Ayush Pareek , Alan W Black , and Yulia Tsvetkov . 2019 . Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing . Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Quantifying Social Biases in Contextual Word Representations. In 1st ACL Workshop on Gender Bias for Natural Language Processing ."},{"key":"e_1_3_2_1_44_1","first-page":"6","article-title":"CIA psychological warfare operations. Case Studies in Chile, Jamaica and Nicaragua","volume":"14","author":"Landis Fred","year":"1982","unstructured":"Fred Landis , Richard Lewontin , Luc Desnoyers , Donna Mergler , and Anthony Weston . 1982 . CIA psychological warfare operations. Case Studies in Chile, Jamaica and Nicaragua . Science for the People 14 (1982), 6 -- 11 . Fred Landis, Richard Lewontin, Luc Desnoyers, Donna Mergler, and Anthony Weston. 1982. CIA psychological warfare operations. Case Studies in Chile, Jamaica and Nicaragua. Science for the People 14 (1982), 6--11.","journal-title":"Science for the People"},{"key":"e_1_3_2_1_45_1","volume-title":"On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561","author":"May Chandler","year":"2019","unstructured":"Chandler May , Alex Wang , Shikha Bordia , Samuel R Bowman , and Rachel Rudinger . 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 ( 2019 ). Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561 (2019)."},{"key":"e_1_3_2_1_46_1","unstructured":"Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.  Tomas Mikolov Ilya Sutskever Kai Chen Greg S Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119."},{"key":"e_1_3_2_1_47_1","volume-title":"StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456","author":"Nadeem Moin","year":"2020","unstructured":"Moin Nadeem , Anna Bethke , and Siva Reddy . 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 ( 2020 ). Moin Nadeem, Anna Bethke, and Siva Reddy. 2020. StereoSet: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 (2020)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1037\/1089-2699.6.1.101"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1525\/aa.1964.66.3.02a00880"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/00277738.2016.1118858"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_1_52_1","volume-title":"Deep contextualized word representations. arXiv preprint arXiv:1802.05365","author":"Peters Matthew E","year":"2018","unstructured":"Matthew E Peters , Mark Neumann , Mohit Iyyer , Matt Gardner , Christopher Clark , Kenton Lee , and Luke Zettlemoyer . 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 ( 2018 ). Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365 (2018)."},{"key":"e_1_3_2_1_53_1","volume-title":"Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf","author":"Radford Alec","year":"2018","unstructured":"Alec Radford , Karthik Narasimhan , Tim Salimans , and Ilya Sutskever . 2018. Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf ( 2018 ). Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https:\/\/s3-us-west-2. amazonaws. com\/openai-assets\/researchcovers\/languageunsupervised\/language understanding paper. pdf (2018)."},{"key":"e_1_3_2_1_54_1","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019 . Language models are unsupervised multitask learners . OpenAI Blog , Vol. 1 , 8 (2019), 9 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, Vol. 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_1_55_1","volume-title":"Challenges for mitigating bias in algorithmic hiring","author":"Raghavan M","year":"2019","unstructured":"M Raghavan and S Barocas . [n.d.]. Challenges for mitigating bias in algorithmic hiring . 2019 . URL https:\/\/www. brookings. edu\/research\/challenges-for-mitigating-bias-in-algorithmic-hiring ( [n.,d.]). M Raghavan and S Barocas. [n.d.]. Challenges for mitigating bias in algorithmic hiring. 2019. URL https:\/\/www. brookings. edu\/research\/challenges-for-mitigating-bias-in-algorithmic-hiring ( [n.,d.])."},{"key":"e_1_3_2_1_56_1","volume-title":"Cohen's d, and r. Law and human behavior","author":"Rice Marnie E","year":"2005","unstructured":"Marnie E Rice and Grant T Harris . 2005. Comparing effect sizes in follow-up studies: ROC Area , Cohen's d, and r. Law and human behavior , Vol. 29 , 5 ( 2005 ), 615--620. Marnie E Rice and Grant T Harris. 2005. Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r. Law and human behavior, Vol. 29, 5 (2005), 615--620."},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.3115\/1119355.1119369"},{"key":"e_1_3_2_1_58_1","volume-title":"Stevens' handbook of experimental psychology","author":"Rosenthal Robert","year":"2002","unstructured":"Robert Rosenthal and M Robin DiMatteo . 2002. Meta\u00adanalysis. Stevens' handbook of experimental psychology ( 2002 ). Robert Rosenthal and M Robin DiMatteo. 2002. Meta\u00adanalysis. Stevens' handbook of experimental psychology (2002)."},{"key":"e_1_3_2_1_59_1","volume-title":"Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243","author":"Strubell Emma","year":"2019","unstructured":"Emma Strubell , Ananya Ganesh , and Andrew McCallum . 2019. Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 ( 2019 ). Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and policy considerations for deep learning in NLP. arXiv preprint arXiv:1906.02243 (2019)."},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314270"},{"key":"e_1_3_2_1_61_1","unstructured":"Yi Chern Tan and L Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems. 13209--13220.  Yi Chern Tan and L Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems. 13209--13220."},{"key":"e_1_3_2_1_62_1","volume-title":"Samuel R Bowman, Dipanjan Das, et al.","author":"Tenney Ian","year":"2019","unstructured":"Ian Tenney , Patrick Xia , Berlin Chen , Alex Wang , Adam Poliak , R Thomas McCoy , Najoung Kim , Benjamin Van Durme , Samuel R Bowman, Dipanjan Das, et al. 2019 . What do you learn from context? probing for sentence structure in contextualized word representations. arXiv preprint arXiv:1905.06316 (2019). Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. 2019. What do you learn from context? probing for sentence structure in contextualized word representations. arXiv preprint arXiv:1905.06316 (2019)."},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"crossref","unstructured":"Veronica G Thomas and Shari E Miles. 1995. Psychology of Black women: Past present and future. (1995).  Veronica G Thomas and Shari E Miles. 1995. Psychology of Black women: Past present and future. (1995).","DOI":"10.1037\/10501-013"},{"key":"e_1_3_2_1_64_1","volume-title":"ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades. arXiv preprint arXiv:2006.03950","author":"Toney Autumn","year":"2020","unstructured":"Autumn Toney and Aylin Caliskan . 2020. ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades. arXiv preprint arXiv:2006.03950 ( 2020 ). Autumn Toney and Aylin Caliskan. 2020. ValNorm: A New Word Embedding Intrinsic Evaluation Method Reveals Valence Biases are Consistent Across Languages and Over Decades. arXiv preprint arXiv:2006.03950 (2020)."},{"key":"e_1_3_2_1_65_1","volume-title":"Pro-Russian Biases in Anti-Chinese Tweets about the Novel Coronavirus. arXiv preprint arXiv:2004.08726","author":"Toney Autumn","year":"2020","unstructured":"Autumn Toney , Akshat Pandey , Wei Guo , David Broniatowski , and Aylin Caliskan . 2020. Pro-Russian Biases in Anti-Chinese Tweets about the Novel Coronavirus. arXiv preprint arXiv:2004.08726 ( 2020 ). Autumn Toney, Akshat Pandey, Wei Guo, David Broniatowski, and Aylin Caliskan. 2020. Pro-Russian Biases in Anti-Chinese Tweets about the Novel Coronavirus. arXiv preprint arXiv:2004.08726 (2020)."},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944013"},{"key":"e_1_3_2_1_67_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_3_2_1_68_1","volume-title":"Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC","author":"Voigt Rob","year":"2018","unstructured":"Rob Voigt , David Jurgens , Vinodkumar Prabhakaran , Dan Jurafsky , and Yulia Tsvetkov . 2018 . RtGender: A corpus for studying differential responses to gender . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). Rob Voigt, David Jurgens, Vinodkumar Prabhakaran, Dan Jurafsky, and Yulia Tsvetkov. 2018. RtGender: A corpus for studying differential responses to gender. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)."},{"key":"e_1_3_2_1_69_1","volume-title":"Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5754--5764.","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le . 2019 . Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5754--5764. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5754--5764."},{"key":"e_1_3_2_1_70_1","volume-title":"Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310","author":"Zhao Jieyu","year":"2019","unstructured":"Jieyu Zhao , Tianlu Wang , Mark Yatskar , Ryan Cotterell , Vicente Ordonez , and Kai-Wei Chang . 2019. Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310 ( 2019 ). Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. arXiv preprint arXiv:1904.03310 (2019)."},{"key":"e_1_3_2_1_71_1","volume-title":"Learning gender-neutral word embeddings. arXiv preprint arXiv:1809.01496","author":"Zhao Jieyu","year":"2018","unstructured":"Jieyu Zhao , Yichao Zhou , Zeyu Li , Wei Wang , and Kai-Wei Chang . 2018. Learning gender-neutral word embeddings. arXiv preprint arXiv:1809.01496 ( 2018 ). Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai-Wei Chang. 2018. Learning gender-neutral word embeddings. arXiv preprint arXiv:1809.01496 (2018)."},{"key":"e_1_3_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.11"}],"event":{"name":"AIES '21: AAAI\/ACM Conference on AI, Ethics, and Society","location":"Virtual Event USA","acronym":"AIES '21","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","AAAI"]},"container-title":["Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462536","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461702.3462536","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:49:06Z","timestamp":1750193346000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462536"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":72,"alternative-id":["10.1145\/3461702.3462536","10.1145\/3461702"],"URL":"https:\/\/doi.org\/10.1145\/3461702.3462536","relation":{},"subject":[],"published":{"date-parts":[[2021,7,21]]},"assertion":[{"value":"2021-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}