{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T11:12:08Z","timestamp":1768907528029,"version":"3.49.0"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T00:00:00Z","timestamp":1733788800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T00:00:00Z","timestamp":1733788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-2106666"],"award-info":[{"award-number":["IIS-2106666"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Columbia World Projects","award":["Columbia World Projects"],"award-info":[{"award-number":["Columbia World Projects"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI Ethics"],"published-print":{"date-parts":[[2025,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>The growing accessibility of large language models (LLMs) has raised many questions about the reliability of probabilistically generated natural language responses. While researchers have documented how bias in the training data leads to biased and ethically problematic output, little attention has been paid to the problems which arise from the nature of the varieties of language on which these models are trained. In particular, certain kinds of expressive and performative language use are more common among African American social media users than they occur in the naturalistic speech of African Americans, a discrepancy which models may fail to take into account when they are training on easily-scraped data as being representative of African American speech. Because LLM training data is generally proprietary, in this work we simulate the training data using a collected dataset consisting of 274 posts from Twitter, Reddit, and Hip-Hop lyrics and analyze how LLMs interpreted their meaning. We highlight the difficulties LLMs, including GPT-3 and GPT-4, have in understanding performative AAL and examine how camouflaging and performativity are addressed (or not) by LLMs and demonstrate the harmful implications of misinterpreting online performance.<\/jats:p>","DOI":"10.1007\/s43681-024-00623-2","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T13:06:33Z","timestamp":1733835993000},"page":"29-37","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Unmasking camouflage: exploring the challenges of large language models in deciphering African American language &amp; online performativity"],"prefix":"10.1007","volume":"5","author":[{"given":"Shana","family":"Kleiner","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jessica A.","family":"Grieser","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shug","family":"Miller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Shepard","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Javier","family":"Garcia-Perez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nick","family":"Deas","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Desmond U.","family":"Patton","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elsbeth","family":"Turcan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kathleen","family":"McKeown","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,12,10]]},"reference":[{"issue":"2","key":"623_CR1","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1017\/S004740450001037X","volume":"13","author":"A Bell","year":"1984","unstructured":"Bell, A.: Language style as audience design. Lang. Soc. 13(2), 145\u2013204 (1984)","journal-title":"Lang. Soc."},{"key":"623_CR2","doi-asserted-by":"publisher","unstructured":"Blodgett, S.L., O\u2019Connor, B.: Racial disparity in natural language processing: a case study of social media African American English. Cornell University\u00a0(2017). arXiv https:\/\/doi.org\/10.48550\/arxiv.1707.00061","DOI":"10.48550\/arxiv.1707.00061"},{"key":"623_CR3","doi-asserted-by":"publisher","DOI":"10.48550\/arxiv.1608.08868","author":"SL Blodgett","year":"2016","unstructured":"Blodgett, S.L., Green, L., O\u2019Connor, B.: Demographic dialectal variation in social media: a case study of African American English. Cornell University. (2016). arXiv https:\/\/doi.org\/10.48550\/arxiv.1608.08868","journal-title":"arXiv (Cornell University)"},{"issue":"4","key":"623_CR4","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1177\/0741088314547188","volume":"31","author":"J Cunningham","year":"2014","unstructured":"Cunningham, J.: Features of digital African American language in a social network site. Writ. Communication. 31(4), 404\u2013433 (2014). https:\/\/doi.org\/10.1177\/0741088314547188","journal-title":"Writ. Communication"},{"issue":"5","key":"623_CR5","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1177\/1461444812462849","volume":"15","author":"J Daniels","year":"2013","unstructured":"Daniels, J.: Race and racism in internet studies: a review and critique. new. Media Soc. 15(5), 695\u2013719 (2013)","journal-title":"new. Media Soc."},{"key":"623_CR6","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w19-3504","author":"T Davidson","year":"2019","unstructured":"Davidson, T., Bhattacharya, D., Weber, I.: Racial bias in hate speech and abusive language detection datasets. Proc. Third Workshop Abusive Lang. Online. (2019). https:\/\/doi.org\/10.18653\/v1\/w19-3504","journal-title":"Proc. Third Workshop Abusive Lang. Online"},{"key":"623_CR7","doi-asserted-by":"crossref","unstructured":"Deas, N., Grieser, J., Kleiner, S., Patton, D., Turcan, E., McKeown, K.: Evaluation of African American language bias in natural language generation (2023). arXiv https:\/\/arxiv.org\/abs\/2305.14291","DOI":"10.18653\/v1\/2023.emnlp-main.421"},{"issue":"3","key":"623_CR8","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1177\/1527476413480247","volume":"15","author":"S Florini","year":"2014","unstructured":"Florini, S.: Tweets, tweeps, and signifyin\u2019: communication and cultural performance on Black Twitter. Telev. New. Media. 15(3), 223\u2013237 (2014). https:\/\/doi.org\/10.1177\/1527476413480247","journal-title":"Telev. New. Media"},{"key":"623_CR9","unstructured":"Goffman, E.: The presentation of self in everyday life. 1959. Garden City, 259 (2002)"},{"issue":"4","key":"623_CR10","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1177\/2332649216639067","volume":"2","author":"R Graham","year":"2016","unstructured":"Graham, R., Smith, S.: The content of our #Characters: Black Twitter as counterpublic. Sociol. Race Ethn. 2(4), 433\u2013449 (2016). https:\/\/doi.org\/10.1177\/2332649216639067","journal-title":"Sociol. Race Ethn."},{"key":"623_CR11","unstructured":"Green, J.L.: Digital blackface: the repackaging of the Black masculine image. Master\u2019s Thesis, Miami University (2006)"},{"issue":"1","key":"623_CR12","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1215\/00031283-7322011","volume":"94","author":"JA Grieser","year":"2019","unstructured":"Grieser, J.A.: Investigating topic-based style shifting in the classic sociolinguistic interview. Am. Speech: Q. Linguistic Usage. 94(1), 54\u201371 (2019)","journal-title":"Am. Speech: Q. Linguistic Usage"},{"key":"623_CR13","volume-title":"The Black Side of the River: Race, Language, and Belonging in Washington","author":"JA Grieser","year":"2022","unstructured":"Grieser, J.A.: The Black Side of the River: Race, Language, and Belonging in Washington. Georgetown University Press, Washington (2022)"},{"key":"623_CR14","unstructured":"Grieser, J.A., Shepard, J., Deas, N.: (Under Review) Exploring the deficiencies of large language models in use of AAL: an interdisciplinary study. Under review at Linguistic Vanguard"},{"key":"623_CR15","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1109\/TASLP.2023.3331139","volume":"32","author":"E Hamel","year":"2024","unstructured":"Hamel, E., Kani, N.: Factors that influence automatic recognition of African American vernacular english in machine-learning models. IEEE\/ACM Trans. Audio Speech Lang. Process. 32, 509\u2013516 (2024). https:\/\/doi.org\/10.1109\/TASLP.2023.3331139","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"623_CR16","volume-title":"Discussing Blackness on Reddit? Photograph your Forearm First","author":"A Harmon","year":"2019","unstructured":"Harmon, A.: Discussing Blackness on Reddit? Photograph your Forearm First. New York Times (2019). (nytimes.com)"},{"key":"623_CR17","unstructured":"https:\/\/www.kaporcenter.org\/wp-content\/uploads\/2023\/03\/KC22001_black-tech-report.final_.pdf"},{"issue":"2","key":"623_CR18","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1111\/josl.12366","volume":"24","author":"C Ilbury","year":"2019","unstructured":"Ilbury, C.: Sassy Queens: stylistic orthographic variation in Twitter and the enregisterment of AAVE. J. Sociolinguistics (Print). 24(2), 245\u2013264 (2019). https:\/\/doi.org\/10.1111\/josl.12366","journal-title":"J. Sociolinguistics (Print)"},{"issue":"4","key":"623_CR19","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1215\/00031283-3442117","volume":"90","author":"T Jones","year":"2015","unstructured":"Jones, T.: Toward a description of African American vernacular english dialect regions using Black Twitter. Am. Speech. 90(4), 403\u2013440 (2015). https:\/\/doi.org\/10.1215\/00031283-3442117","journal-title":"Am. Speech"},{"key":"623_CR20","unstructured":"Kapor Center: State of Tech Diversity: The Black Tech Ecosystem (2023)"},{"key":"623_CR21","volume-title":"The Corpus of Regional African American Language. Version 2023.06. Eugene, Ore","author":"T Kendall","year":"2023","unstructured":"Kendall, T., Farrington, C.: The Corpus of Regional African American Language. Version 2023.06. Eugene, Ore. The Online Resources for African American Language Project (2023)"},{"issue":"3","key":"623_CR22","first-page":"178","volume":"152","author":"S King","year":"2023","unstructured":"King, S., Rickford, J.R.: Language on trial. D\u00e6dalus. 152(3), 178\u2013193 (2023)","journal-title":"D\u00e6dalus"},{"issue":"4","key":"623_CR23","doi-asserted-by":"publisher","first-page":"205630512211443","DOI":"10.1177\/20563051221144317","volume":"8","author":"S Klassen","year":"2022","unstructured":"Klassen, S., Fiesler, C.: This isn\u2019t your data, friend\u2019: Black Twitter as a case study on research ethics for public data. Social Media + Soc. 8(4), 205630512211443 (2022). https:\/\/doi.org\/10.1177\/20563051221144317","journal-title":"Social Media + Soc."},{"key":"623_CR24","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780199381265.001.0001","volume-title":"The Digital Street","author":"J Lane","year":"2018","unstructured":"Lane, J.: The Digital Street. Oxford University Press (2018)"},{"key":"623_CR25","doi-asserted-by":"crossref","unstructured":"Lockett, A.: What is Black Twitter? A rhetorical criticism of race, dis\/information, and social media. Race, Rhetoric, and Research Methods, 165\u2013213 (2021)","DOI":"10.37514\/PER-B.2021.1206"},{"key":"623_CR26","volume-title":"Because Internet","author":"G McCulloch","year":"2019","unstructured":"McCulloch, G.: Because Internet. Riverhead Books, New York (2019)"},{"key":"623_CR27","unstructured":"Mitchell-Kernan, C.: Signifying, loud-talking, and marking. In Dagel Caponi, Gena, ed. Signfyin(g), Sanctifyin\u2019, & Slam Dunking: A Reader in African American Expressive Culture. University of Massachusetts Press, Amherst (1999)"},{"key":"623_CR28","first-page":"18","volume":"6","author":"Z Papacharissi","year":"2012","unstructured":"Papacharissi, Z.: Without you, I\u2019m nothing: performances of the self on Twitter. Int. J. Communication. 6, 18 (2012)","journal-title":"Int. J. Communication"},{"key":"623_CR29","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1016\/j.chb.2016.05.052","volume":"65","author":"DU Patton","year":"2016","unstructured":"Patton, D.U., Eschmann, R.D., Elsaesser, C., Bocanegra, E.: Sticks, stones and Facebook accounts: what violence outreach workers know about social media and urban-based gang violence in Chicago. Comput. Hum. Behav. 65, 591\u2013600 (2016). https:\/\/doi.org\/10.1016\/j.chb.2016.05.052","journal-title":"Comput. Hum. Behav."},{"key":"623_CR30","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1057\/s41599-019-0330-x","volume":"5","author":"DU Patton","year":"2019","unstructured":"Patton, D.U., Frey, W.R., Gaskell, M.: Guns on social media: complex interpretations of gun images posted by Chicago Youth. Palgrave Commun. 5, 119 (2019). https:\/\/doi.org\/10.1057\/s41599-019-0330-x","journal-title":"Palgrave Commun."},{"key":"623_CR31","unstructured":"Patton, D., Darkwa, E., Anguiano, K.: Recommendations to end virtual stop and frisk policing on social media. Tech Policy Press. Accessed (2021).\u00a0https:\/\/www.techpolicy.press\/recommendations-to-end-virtual-stop-and-frisk-policing-on-social-media\/"},{"key":"623_CR32","doi-asserted-by":"crossref","unstructured":"Pavalanathan, U., Fitzpatrick, J., Kiesling, S.F., Eisenstein, J.: A multidimensional lexicon for interpersonal stancetaking. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1: Long Papers, pp. 884\u2013895 (2017)","DOI":"10.18653\/v1\/P17-1082"},{"issue":"2","key":"623_CR33","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1080\/10350330.2013.859355","volume":"24","author":"JL Rightler-McDaniels","year":"2013","unstructured":"Rightler-McDaniels, J.L., Hendrickson, E.M.: Hoes and hashtags: constructions of gender and race in trending topics. Social Semiot. 24(2), 175\u2013190 (2013). https:\/\/doi.org\/10.1080\/10350330.2013.859355","journal-title":"Social Semiot."},{"issue":"10","key":"623_CR34","doi-asserted-by":"publisher","first-page":"1643","DOI":"10.1177\/1461444814531364","volume":"17","author":"R Schwartz","year":"2015","unstructured":"Schwartz, R., Halegoua, G.R.: The spatial self: location-based identity performance on social media. New. Media Soc. 17(10), 1643\u20131660 (2015). https:\/\/doi.org\/10.1177\/1461444814531364","journal-title":"New. Media Soc."},{"issue":"4","key":"623_CR35","doi-asserted-by":"publisher","first-page":"850","DOI":"10.2307\/413960","volume":"58","author":"AK Spears","year":"1982","unstructured":"Spears, A.K.: The Black english semi-auxiliary come. Language. 58(4), 850\u2013872 (1982). https:\/\/doi.org\/10.2307\/413960","journal-title":"Language"},{"key":"623_CR36","doi-asserted-by":"publisher","DOI":"10.1017\/9781139021531","volume-title":"Middle-class African American English","author":"TL Weldon","year":"2021","unstructured":"Weldon, T.L.: Middle-class African American English. Cambridge University Press (2021)"}],"container-title":["AI and Ethics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-024-00623-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s43681-024-00623-2\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s43681-024-00623-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,24]],"date-time":"2025-03-24T06:39:22Z","timestamp":1742798362000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s43681-024-00623-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,10]]},"references-count":36,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["623"],"URL":"https:\/\/doi.org\/10.1007\/s43681-024-00623-2","relation":{},"ISSN":["2730-5953","2730-5961"],"issn-type":[{"value":"2730-5953","type":"print"},{"value":"2730-5961","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,10]]},"assertion":[{"value":"3 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 November 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 December 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors whose names are listed immediately below certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}