{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T10:47:36Z","timestamp":1790246856145,"version":"4.1.0"},"reference-count":160,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,2]]},"abstract":"<jats:p>Behind the scenes of maintaining the safety of technology products from harmful and illegal digital content lies unrecognized human labor. The recent rise in the use of generative AI technologies and the accelerating demands to meet responsible AI (RAI) aims necessitates an increased focus on the labor behind such efforts in the age of AI. This study investigates the nature and challenges of content work that supports RAI efforts, or \"RAI content work,\" that spans content moderation, data labeling, and red teaming -- through the lived experiences of content workers. We conduct a formative survey and semi-structured interview studies to develop a conceptualization of RAI content work and a subsequent framework of recommendations for providing holistic support for content workers. We validate our recommendations through a series of workshops with content workers and derive considerations for and examples of implementing such recommendations. We discuss how our framework may guide future innovation to support the well-being and professional development of the RAI content workforce.<\/jats:p>","DOI":"10.1145\/3710931","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T11:36:19Z","timestamp":1747740979000},"page":"1-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-6407-6981","authenticated-orcid":false,"given":"Alice Qian","family":"Zhang","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, PA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1285-6909","authenticated-orcid":false,"given":"Judith","family":"Amores","sequence":"additional","affiliation":[{"name":"Microsoft Research, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5364-3718","authenticated-orcid":false,"given":"Hong","family":"Shen","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0881-401X","authenticated-orcid":false,"given":"Mary","family":"Czerwinski","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9972-6829","authenticated-orcid":false,"given":"Mary L.","family":"Gray","sequence":"additional","affiliation":[{"name":"Microsoft Research, Cambridge, Massachusetts, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7646-5563","authenticated-orcid":false,"given":"Jina","family":"Suh","sequence":"additional","affiliation":[{"name":"Microsoft Research, Redmond, Washington, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2010.939578"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11948-004-0011-y"},{"key":"e_1_2_1_3_1","volume-title":"Harvard Journal of Law and Technology","volume":"2","author":"Arsht Andrew","year":"2018","unstructured":"Andrew Arsht and Daniel Etcovitch. 2018. The human cost of online content moderation. Harvard Journal of Law and Technology, Vol. 2 (2018)."},{"key":"e_1_2_1_4_1","volume-title":"Diagnostic and statistical manual of mental disorders: DSM-5","author":"American Psychiatric Association","unstructured":"American Psychiatric Association. 2013. Diagnostic and statistical manual of mental disorders: DSM-5. Vol. 5. American psychiatric association Washington, DC."},{"key":"e_1_2_1_5_1","unstructured":"Avasant. 2024. https:\/\/avasant.com\/report\/impact-of-generative-ai-on-content-moderation\/"},{"key":"e_1_2_1_6_1","unstructured":"Azure. 2024. PyRIT: Python Risk Intelligence Toolkit. https:\/\/github.com\/Azure\/PyRIT Accessed: 2024-07--15."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.3402\/qhw.v8i0.19823"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11896-016-9223-8"},{"key":"e_1_2_1_9_1","unstructured":"Miles Brundage Shahar Avin Jasmine Wang Haydn Belfield Gretchen Krueger Gillian Hadfield Heidy Khlaaf Jingying Yang Helen Toner Ruth Fong Tegan Maharaj Pang Wei Koh Sara Hooker Jade Leung Andrew Trask Emma Bluemke Jonathan Lebensold Cullen O'Keefe Mark Koren Th\u00e9o Ryffel J. B. Rubinovitz Tamay Besiroglu Federica Carugati Jack Clark Peter Eckersley Sarah de Haas Maritza Johnson Ben Laurie Alex Ingerman Igor Krawczuk Amanda Askell Rosario Cammarota Andrew Lohn David Krueger Charlotte Stix Peter Henderson Logan Graham Carina Prunkl Bianca Martin Elizabeth Seger Noa Zilberman Se\u00e1n \u00d3 h\u00c9igeartaigh Frens Kroeger Girish Sastry Rebecca Kagan Adrian Weller Brian Tse Elizabeth Barnes Allan Dafoe Paul Scharre Ariel Herbert-Voss Martijn Rasser Shagun Sodhani Carrick Flynn Thomas Krendl Gilbert Lisa Dyer Saif Khan Yoshua Bengio and Markus Anderljung. 2020. Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims. arxiv: 2004.07213 [cs] http:\/\/arxiv.org\/abs\/2004.07213"},{"key":"e_1_2_1_10_1","volume-title":"Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, and Saleema Amershi.","author":"Buccinca Zana","year":"2023","unstructured":"Zana Buccinca, Chau Minh Pham, Maurice Jakesch, Marco Tulio Ribeiro, Alexandra Olteanu, and Saleema Amershi. 2023. AHA!: Facilitating AI Impact Assessment by Generating Examples of Harms. ArXiv, Vol. abs\/2306.03280 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259088554"},{"key":"e_1_2_1_11_1","unstructured":"Katie Canales. 2021. Mark Zuckerberg said content moderation requires ''nuances'' that consider the intent behind a post but also highlighted Facebook's reliance on ai to do that job. https:\/\/www.businessinsider.com\/zuckerberg-nuances-content-moderation-ai-misinformation-hearing-2021--3?r=US&IR=T"},{"key":"e_1_2_1_12_1","first-page":"440","article-title":"The hidden listeners: regulating the line from telephone operators to content moderators","volume":"13","author":"Carmi Elinor","year":"2019","unstructured":"Elinor Carmi. 2019. The hidden listeners: regulating the line from telephone operators to content moderators. International Journal of Communication, Vol. 13 (2019), 440--458.","journal-title":"International Journal of Communication"},{"key":"e_1_2_1_13_1","volume-title":"Explore","author":"Casper Stephen","year":"2023","unstructured":"Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadfield-Menell. 2023. Explore, Establish, Exploit: Red Teaming Language Models from Scratch. ArXiv, Vol. abs\/2306.09442 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259187620"},{"key":"e_1_2_1_14_1","unstructured":"Sven Cattell. 2023. Generative Red Team Recap. https:\/\/aivillage.org\/defcon%2031\/generative-recap\/"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025985"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819963"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1097\/JOM.0000000000001330"},{"key":"e_1_2_1_18_1","unstructured":"Kathy Charmaz. 2006. Constructing grounded theory: A practical guide through qualitative analysis. sage."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610103"},{"key":"e_1_2_1_20_1","volume-title":"Validity in qualitative research revisited. Qualitative research","author":"Cho Jeasik","year":"2006","unstructured":"Jeasik Cho and Allen Trent. 2006. Validity in qualitative research revisited. Qualitative research, Vol. 6, 3 (2006), 319--340."},{"key":"e_1_2_1_21_1","unstructured":"Rumman Chowdhury. 2023. Redefining Red Teaming. https:\/\/www.hackthefuture.com\/news\/redefining-red-teaming"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1300\/J490v21n02_07"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555168"},{"key":"e_1_2_1_24_1","volume-title":"Predictors of amygdala activation during the processing of emotional stimuli: a meta-analysis of 385 PET and fMRI studies. Brain research reviews","author":"Costafreda Sergi G","year":"2008","unstructured":"Sergi G Costafreda, Michael J Brammer, Anthony S David, and Cynthia HY Fu. 2008. Predictors of amygdala activation during the processing of emotional stimuli: a meta-analysis of 385 PET and fMRI studies. Brain research reviews, Vol. 58, 1 (2008), 57--70."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444814543163"},{"key":"e_1_2_1_26_1","volume-title":"Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson Education","author":"Creswell John W","unstructured":"John W Creswell. 2012. Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Pearson Education, Inc."},{"key":"e_1_2_1_27_1","volume-title":"Determining validity in qualitative inquiry. Theory into practice","author":"Creswell John W","year":"2000","unstructured":"John W Creswell and Dana L Miller. 2000. Determining validity in qualitative inquiry. Theory into practice, Vol. 39, 3 (2000), 124--130."},{"key":"e_1_2_1_28_1","volume-title":"Cynthia Feltner, Kimberly A Brownley, Kristine Rae Olmsted, Amy Greenblatt, et al.","author":"Cusack Karen","year":"2016","unstructured":"Karen Cusack, Daniel E Jonas, Catherine A Forneris, Candi Wines, Jeffrey Sonis, Jennifer Cook Middleton, Cynthia Feltner, Kimberly A Brownley, Kristine Rae Olmsted, Amy Greenblatt, et al. 2016. Psychological treatments for adults with posttraumatic stress disorder: A systematic review and meta-analysis. Clinical psychology review, Vol. 43 (2016), 128--141."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v8i1.7461"},{"key":"e_1_2_1_30_1","first-page":"400","article-title":"Reframing clinician distress: moral injury not burnout","volume":"36","author":"Dean Wendy","year":"2019","unstructured":"Wendy Dean, Simon Talbot, and Austin Dean. 2019. Reframing clinician distress: moral injury not burnout. Federal Practitioner, Vol. 36, 9 (2019), 400.","journal-title":"Federal Practitioner"},{"key":"e_1_2_1_31_1","volume-title":"2014 international conference on computer vision theory and applications (VISAPP)","volume":"2","author":"Deniz Oscar","year":"2014","unstructured":"Oscar Deniz, Ismael Serrano, Gloria Bueno, and Tae-Kyun Kim. 2014. Fast violence detection in video. In 2014 international conference on computer vision theory and applications (VISAPP), Vol. 2. IEEE, 478--485."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300372"},{"key":"e_1_2_1_33_1","unstructured":"Evelyn Douek. 2021. More content moderation is not always better. https:\/\/www.wired.com\/story\/more-content-moderation-not-always-better\/"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.2105\/ajph.2005.076943"},{"key":"e_1_2_1_35_1","unstructured":"Elizabeth Dwoskin. 2019. Inside facebook the second-class workers who do the hardest job are waging a quiet battle. https:\/\/www.washingtonpost.com\/technology\/2019\/05\/08\/inside-facebook-second-class-workers-who-do-hardest-job-are-waging-quiet-battle\/"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/1077558709335173"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1037\/1076-8998.6.2.101"},{"key":"e_1_2_1_38_1","unstructured":"Daniel Fabian. 2023. Google's AI Red Team: The ethical hackers making ai safer. https:\/\/blog.google\/technology\/safety-security\/googles-ai-red-team-the-ethical-hackers-making-ai-safer\/"},{"key":"e_1_2_1_39_1","unstructured":"Fact.MR. 2024. Content Moderation Solution Market. https:\/\/www.factmr.com\/report\/4522\/content-moderation-solutions-market"},{"key":"e_1_2_1_40_1","volume-title":"Red-Teaming for Generative AI: Silver Bullet or Security Theater? arXiv preprint arXiv:2401.15897","author":"Feffer Michael","year":"2024","unstructured":"Michael Feffer, Anusha Sinha, Zachary C Lipton, and Hoda Heidari. 2024. Red-Teaming for Generative AI: Silver Bullet or Security Theater? arXiv preprint arXiv:2401.15897 (2024)."},{"key":"e_1_2_1_41_1","volume-title":"Witnessing images of extreme violence: a psychological study of journalists in the newsroom. JRSM open","author":"Feinstein Anthony","year":"2014","unstructured":"Anthony Feinstein, Blair Audet, and Elizabeth Waknine. 2014. Witnessing images of extreme violence: a psychological study of journalists in the newsroom. JRSM open, Vol. 5, 8 (2014), 2054270414533323."},{"key":"e_1_2_1_42_1","unstructured":"Sorelle Friedler Ranjit Singh Borhane Blili-Hamelin Jacob Metcalf and Brian Chen. 2023a. Ai Red-teaming is not a one-stop solution to ai harms. https:\/\/datasociety.net\/wp-content\/uploads\/2023\/10\/Recommendations-for-Using-Red-Teaming-for-AI-Accountability-PolicyBrief.pdf"},{"key":"e_1_2_1_43_1","unstructured":"Sorelle Friedler Ranjit Singh Borhane Blili-Hamelin Jacob Metcalf and Brian J Chen. 2023b. AI Red-Teaming Is Not a One-Stop Solution to AI Harms."},{"key":"e_1_2_1_44_1","first-page":"418","article-title":"Functional atlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies","volume":"34","author":"Fusar-Poli Paolo","year":"2009","unstructured":"Paolo Fusar-Poli, Anna Placentino, Francesco Carletti, Paola Landi, Paul Allen, Simon Surguladze, Francesco Benedetti, Marta Abbamonte, Roberto Gasparotti, Francesco Barale, et al. 2009. Functional atlas of emotional faces processing: a voxel-based meta-analysis of 105 functional magnetic resonance imaging studies. Journal of psychiatry and neuroscience, Vol. 34, 6 (2009), 418--432.","journal-title":"Journal of psychiatry and neuroscience"},{"key":"e_1_2_1_45_1","unstructured":"Hack The Future. 2023. https:\/\/www.airedteam.org\/news\/ai-village-at-def-con-announces-largest-ever-public-generative-ai-red-team"},{"key":"e_1_2_1_46_1","unstructured":"Deep Ganguli Liane Lovitt Jackson Kernion Amanda Askell Yuntao Bai Saurav Kadavath Ben Mann Ethan Perez Nicholas Schiefer Kamal Ndousse Andy Jones Sam Bowman Anna Chen Tom Conerly Nova DasSarma Dawn Drain Nelson Elhage Sheer El-Showk Stanislav Fort Zac Hatfield-Dodds Tom Henighan Danny Hernandez Tristan Hume Josh Jacobson Scott Johnston Shauna Kravec Catherine Olsson Sam Ringer Eli Tran-Johnson Dario Amodei Tom Brown Nicholas Joseph Sam McCandlish Chris Olah Jared Kaplan and Jack Clark. 2022. Red Teaming Language Models to Reduce Harms: Methods Scaling Behaviors and Lessons Learned. arxiv: 2209.07858 [cs] http:\/\/arxiv.org\/abs\/2209.07858"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2016.1153700"},{"key":"e_1_2_1_48_1","first-page":"205630512311861","article-title":"What Teams Do: Exploring Volunteer Content Moderation Team Labor on Facebook","volume":"9","author":"Gibson Anna D","year":"2023","unstructured":"Anna D Gibson. 2023. What Teams Do: Exploring Volunteer Content Moderation Team Labor on Facebook. Social Media Society, Vol. 9, 3 (2023), 20563051231186109.","journal-title":"Social Media Society"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Kirsten Gollatz Felix Beer and Christian Katzenbach. 2018. The turn to artificial intelligence in governing communication online. https:\/\/nbn-resolving.org\/urn:nbn:de:0168-ssoar-59528--6.","DOI":"10.31235\/osf.io\/vwpcz"},{"key":"e_1_2_1_50_1","volume-title":"Algorithmic content moderation: Technical and political challenges in the automation of platform governance","author":"Gorwa Robert","year":"2020","unstructured":"Robert Gorwa, Reuben Binns, and Christian Katzenbach. 2020. Algorithmic content moderation: Technical and political challenges in the automation of platform governance. Big Data & Society, Vol. 7 (2020). https:\/\/api.semanticscholar.org\/CorpusID:213703822"},{"key":"e_1_2_1_51_1","volume-title":"Ghost work: How to stop Silicon Valley from building a new global underclass","author":"Gray Mary L","unstructured":"Mary L Gray and Siddharth Suri. 2019. Ghost work: How to stop Silicon Valley from building a new global underclass. Eamon Dolan Books."},{"key":"e_1_2_1_52_1","volume-title":"Managing mental health challenges faced by healthcare workers during covid-19 pandemic. bmj","author":"Greenberg Neil","year":"2020","unstructured":"Neil Greenberg, Mary Docherty, Sam Gnanapragasam, and Simon Wessely. 2020. Managing mental health challenges faced by healthcare workers during covid-19 pandemic. bmj, Vol. 368 (2020)."},{"key":"e_1_2_1_53_1","volume-title":"Work--family spillover and daily reports of work and family stress in the adult labor force. Family relations","author":"Grzywacz Joseph G","year":"2002","unstructured":"Joseph G Grzywacz, David M Almeida, and Daniel A McDonald. 2002. Work--family spillover and daily reports of work and family stress in the adult labor force. Family relations, Vol. 51, 1 (2002), 28--36."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1111\/1472-9296.00053"},{"key":"e_1_2_1_55_1","volume-title":"A scoping study of moral injury: Identifying directions for social work research. Children and youth services review","author":"Haight Wendy","year":"2016","unstructured":"Wendy Haight, Erin Sugrue, Molly Calhoun, and James Black. 2016. A scoping study of moral injury: Identifying directions for social work research. Children and youth services review, Vol. 70 (2016), 190--200."},{"key":"e_1_2_1_56_1","volume-title":"Hao Ma, Umut Ozertem, Patrick Pantel, Marzieh Saeidi, Fabrizio Silvestri, and Ves Stoyanov.","author":"Halevy Alon","year":"2020","unstructured":"Alon Halevy, Cristian Canton Ferrer, Hao Ma, Umut Ozertem, Patrick Pantel, Marzieh Saeidi, Fabrizio Silvestri, and Ves Stoyanov. 2020. Preserving integrity in online social networks. arXiv:2009.10311 (2020)."},{"key":"e_1_2_1_57_1","volume-title":"Proceedings of the International Conference on Industrial Networks and Intelligent Systems. Springer, 164--173","author":"Hammer Hugo Lewi","year":"2016","unstructured":"Hugo Lewi Hammer. 2016. Automatic detection of hateful comments in online discussion. In Proceedings of the International Conference on Industrial Networks and Intelligent Systems. Springer, 164--173."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3579609"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1037\/ocp0000069"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369457.3369491"},{"key":"e_1_2_1_61_1","volume-title":"Dana Rose Garfin, and Roxane Cohen Silver","author":"Holman E Alison","year":"2014","unstructured":"E Alison Holman, Dana Rose Garfin, and Roxane Cohen Silver. 2014. Media's role in broadcasting acute stress following the Boston Marathon bombings., Vol. 111, 1 (2014), 93--98. Publisher: National Acad Sciences."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0004153"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0013706"},{"key":"e_1_2_1_64_1","unstructured":"The White House. 2023. FACT SHEET: Biden-Harris Administration Secures Voluntary Commitments from Leading Artificial Intelligence Companies to Manage the Risks Posed by AI. https:\/\/www.whitehouse.gov\/briefing-room\/statements-releases\/2023\/07\/21\/fact-sheet-biden-harris-administration-secures-voluntary-commitments-from-leading-artificial-intelligence-companies-to-manage-the-risks-posed-by-ai\/"},{"key":"e_1_2_1_65_1","unstructured":"Human Intelligence SeedAI and DEFCON AI Village. 2023. Generative AI Red Teaming Challenge: Transparency Report. Technical Report. Humane Intelligence. 1--41 pages. Retrieved July 13 2024 from https:\/\/drive.google.com\/file\/d\/1JqpbIP6DNomkb32umLoiEPombK2-0Rc-\/view."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338243"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1177\/14614448231217993"},{"key":"e_1_2_1_68_1","volume-title":"Quanze Chen, Nikhila Natarajan, Ruotong Wang, and Amy Zhang.","author":"Jhaver Shagun","year":"2023","unstructured":"Shagun Jhaver, Alice Qian Zhang, Quanze Chen, Nikhila Natarajan, Ruotong Wang, and Amy Zhang. 2023. Personalizing Content Moderation on Social Media: User Perspectives on Moderation Choices, Interface Design, and Labor. arXiv preprint arXiv:2305.10374 (2023)."},{"key":"e_1_2_1_69_1","volume-title":"Cyberspace Crime","author":"Jordan Tim","unstructured":"Tim Jordan and Paul Taylor. 2017. A sociology of hackers. In Cyberspace Crime. Routledge, 163--186."},{"key":"e_1_2_1_70_1","unstructured":"Sabine Junginger. 2013. Design and Innovation in the Public Sector: Matters of Design in Policy-Making and Policy Implementation. https:\/\/api.semanticscholar.org\/CorpusID:154113223"},{"key":"e_1_2_1_71_1","volume-title":"A Just and Comprehensive Strategy for Using NLP to Address Online Abuse. arXiv:1906.01738","author":"Jurgens David","year":"2019","unstructured":"David Jurgens, Eshwar Chandrasekharan, and Libby Hemphill. 2019. A Just and Comprehensive Strategy for Using NLP to Address Online Abuse. arXiv:1906.01738 (2019)."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5270"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610207"},{"key":"e_1_2_1_74_1","volume-title":"AI assurance processes. Available at SSRN 3685087","author":"Kazim Emre","year":"2020","unstructured":"Emre Kazim and Adriano Koshiyama. 2020. AI assurance processes. Available at SSRN 3685087 (2020)."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1353\/jod.2021.0043"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1177\/0887302X0602400101"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.5117\/9789053564448"},{"key":"e_1_2_1_78_1","unstructured":"Ram Shankar Siva Kumar. 2023. Microsoft AI Red Team Building Future of Safer Ai. https:\/\/www.microsoft.com\/en-us\/security\/blog\/2023\/08\/07\/microsoft-ai-red-team-building-future-of-safer-ai\/"},{"key":"e_1_2_1_79_1","unstructured":"Jan Leike and Ilya Sutskever. 2023. https:\/\/openai.com\/blog\/introducing-superalignment"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v16i1.19317"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2009.05.021"},{"key":"e_1_2_1_82_1","doi-asserted-by":"crossref","unstructured":"Yvonna S Lincoln and Egon G Guba. 1985. Naturalistic inquiry. sage.","DOI":"10.1016\/0147-1767(85)90062-8"},{"key":"e_1_2_1_83_1","volume-title":"Moral injury and moral repair in war veterans: A preliminary model and intervention strategy. Clinical psychology review","author":"Litz Brett T","year":"2009","unstructured":"Brett T Litz, Nathan Stein, Eileen Delaney, Leslie Lebowitz, William P Nash, Caroline Silva, and Shira Maguen. 2009. Moral injury and moral repair in war veterans: A preliminary model and intervention strategy. Clinical psychology review, Vol. 29, 8 (2009), 695--706."},{"key":"e_1_2_1_84_1","volume-title":"Red teaming: past and present. School of Advanced Military Studies","author":"Longbine David F","year":"2008","unstructured":"David F Longbine. 2008. Red teaming: past and present. School of Advanced Military Studies, Army Command and General Staff College (2008)."},{"key":"e_1_2_1_85_1","volume-title":"Hate speech detection: Challenges and solutions. PloS one","author":"MacAvaney Sean","year":"2019","unstructured":"Sean MacAvaney, Hao-Ren Yao, Eugene Yang, Katina Russell, Nazli Goharian, and Ophir Frieder. 2019. Hate speech detection: Challenges and solutions. PloS one, Vol. 14, 8 (2019)."},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376445"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb028117"},{"key":"e_1_2_1_88_1","unstructured":"Dale A Masi. 2020. The history of employee assistance programs in the United States."},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1177\/2056305119836778"},{"key":"e_1_2_1_90_1","volume-title":"David S Goldbloom, Mona Gupta, Jonathan J Hunter, et al.","author":"Maunder Robert G","year":"2006","unstructured":"Robert G Maunder, William J Lancee, Kenneth E Balderson, Jocelyn P Bennett, Bjug Borgundvaag, Susan Evans, Christopher MB Fernandes, David S Goldbloom, Mona Gupta, Jonathan J Hunter, et al. 2006. Long-term psychological and occupational effects of providing hospital healthcare during SARS outbreak. Emerging infectious diseases, Vol. 12, 12 (2006), 1924."},{"key":"e_1_2_1_91_1","volume-title":"Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249","author":"Mazeika Mantas","year":"2024","unstructured":"Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. arXiv preprint arXiv:2402.04249 (2024)."},{"key":"e_1_2_1_92_1","unstructured":"Sharan B Merriam et al. 2002. Introduction to qualitative research. Qualitative research in practice: Examples for discussion and analysis Vol. 1 1 (2002) 1--17."},{"key":"e_1_2_1_93_1","unstructured":"Michel. 2018. Ex-Content Moderator Sues Facebook Saying Violent Images Caused Her PTSD - e-traces. https:\/\/api.semanticscholar.org\/CorpusID:150309904"},{"key":"e_1_2_1_94_1","unstructured":"Microsoft. 2023. Introduction to red teaming large language models (llms). https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/openai\/concepts\/red-teaming"},{"key":"e_1_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0033291710002291.Barriers"},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24637"},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1177\/1461444818773059"},{"key":"e_1_2_1_98_1","unstructured":"Casey Newton. [n. d.]. The trauma floor. https:\/\/www.theverge.com\/2019\/2\/25\/18229714\/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona Publication Title: The Verge."},{"key":"e_1_2_1_99_1","unstructured":"Casey Newton. 2020a. What tech companies should do about their content moderators' PTSD. https:\/\/www.theverge.com\/interface\/2020\/1\/28\/21082642\/content-moderator-ptsd-facebook-youtube-accenture-solutions"},{"key":"e_1_2_1_100_1","unstructured":"Casey Newton. 2020b. YouTube moderators are being forced to sign a statement acknowledging the job can give them PTSD. https:\/\/www.theverge.com\/2020\/1\/24\/21075830\/youtube-moderators-ptsd-accenture-statement-lawsuits-mental-health"},{"key":"e_1_2_1_101_1","volume-title":"CTI4AI: Threat Intelligence Generation and Sharing after Red Teaming AI Models. arXiv preprint arXiv:2208.07476","author":"Nguyen Chuyen","year":"2022","unstructured":"Chuyen Nguyen, Caleb Morgan, and Sudip Mittal. 2022. CTI4AI: Threat Intelligence Generation and Sharing after Red Teaming AI Models. arXiv preprint arXiv:2208.07476 (2022)."},{"key":"e_1_2_1_102_1","unstructured":"The American Institute of Stress. 2021. Workplace Stress. https:\/\/www.stress.org\/workplace-stress"},{"key":"e_1_2_1_103_1","unstructured":"OpenAI. 2023a. https:\/\/openai.com\/blog\/red-teaming-network"},{"key":"e_1_2_1_104_1","unstructured":"OpenAI. 2023b. Frontier Model Forum: What is Red Teaming? https:\/\/www.frontiermodelforum.org\/uploads\/2023\/10\/FMF-AI-Red-Teaming.pdf"},{"key":"e_1_2_1_105_1","unstructured":"OpenAI. 2023c. GPT-4 System Card. https:\/\/cdn.openai.com\/papers\/gpt-4-system-card.pdf"},{"key":"e_1_2_1_106_1","volume-title":"Meet the hackers who are trying to make AI go rogue. The Washington Post. https:\/\/www.washingtonpost.com\/technology\/2023\/08\/08\/ai-red-team-defcon\/","author":"Oremus Will","year":"2023","unstructured":"Will Oremus. 2023. Meet the hackers who are trying to make AI go rogue. The Washington Post. https:\/\/www.washingtonpost.com\/technology\/2023\/08\/08\/ai-red-team-defcon\/"},{"key":"e_1_2_1_107_1","unstructured":"PAI. 2021. Responsible sourcing of data enrichment services. https:\/\/partnershiponai.org\/wp-content\/uploads\/2021\/08\/PAI-Responsible-Sourcing-of-Data-Enrichment-Services.pdf"},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858389"},{"key":"e_1_2_1_109_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517672"},{"key":"e_1_2_1_110_1","volume-title":"Jessica Quaye, Charvi Rastogi, Max Bartolo, Oana Inel, Juan Ciro, Rafael Mosquera, Addison Howard, William J. Cukierski, D. Sculley, Vijay Janapa Reddi, and Lora Aroyo.","author":"Parrish Alicia","year":"2023","unstructured":"Alicia Parrish, Hannah Rose Kirk, Jessica Quaye, Charvi Rastogi, Max Bartolo, Oana Inel, Juan Ciro, Rafael Mosquera, Addison Howard, William J. Cukierski, D. Sculley, Vijay Janapa Reddi, and Lora Aroyo. 2023. Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models. ArXiv, Vol. abs\/2305.14384 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258865543"},{"key":"e_1_2_1_111_1","doi-asserted-by":"crossref","unstructured":"Ethan Perez Saffron Huang Francis Song Trevor Cai Roman Ring John Aslanides Amelia Glaese Nat McAleese and Geoffrey Irving. 2022. Red Teaming Language Models with Language Models. arxiv: 2202.03286 [cs] http:\/\/arxiv.org\/abs\/2202.03286","DOI":"10.18653\/v1\/2022.emnlp-main.225"},{"key":"e_1_2_1_112_1","unstructured":"Billy Perrigo. 2023. OpenAI used Kenyan workers on less than $2 per hour: Exclusive. https:\/\/time.com\/6247678\/openai-chatgpt-kenya-workers\/"},{"key":"e_1_2_1_113_1","doi-asserted-by":"publisher","DOI":"10.1177\/01634437221122226"},{"key":"e_1_2_1_114_1","volume-title":"Aart: Ai-assisted red-teaming with diverse data generation for new llm-powered applications. arXiv preprint arXiv:2311.08592","author":"Radharapu Bhaktipriya","year":"2023","unstructured":"Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, and Preethi Lahoti. 2023a. Aart: Ai-assisted red-teaming with diverse data generation for new llm-powered applications. arXiv preprint arXiv:2311.08592 (2023)."},{"key":"e_1_2_1_115_1","doi-asserted-by":"crossref","unstructured":"Bhaktipriya Radharapu Kevin Robinson Lora Aroyo and Preethi Lahoti. 2023b. AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications. In Conference on Empirical Methods in Natural Language Processing. https:\/\/api.semanticscholar.org\/CorpusID:265213125","DOI":"10.18653\/v1\/2023.emnlp-industry.37"},{"key":"e_1_2_1_116_1","volume-title":"Evolution of Rules in Reddit Communities. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing. 278--282","author":"Reddy Harita","year":"2023","unstructured":"Harita Reddy and Eshwar Chandrasekharan. 2023. Evolution of Rules in Reddit Communities. In Companion Publication of the 2023 Conference on Computer Supported Cooperative Work and Social Computing. 278--282."},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.4278\/0890-1171-10.4.318"},{"key":"e_1_2_1_118_1","volume-title":"A survey on visual adult image recognition. Multimedia tools and applications","author":"Ries Christian X","year":"2014","unstructured":"Christian X Ries and Rainer Lienhart. 2014. A survey on visual adult image recognition. Multimedia tools and applications, Vol. 69, 3 (2014), 661--688."},{"key":"e_1_2_1_119_1","unstructured":"Sarah T Roberts. 2016. Commercial content moderation: Digital laborers' dirty work. (2016)."},{"key":"e_1_2_1_120_1","doi-asserted-by":"crossref","unstructured":"Sarah T Roberts. 2017. Content moderation.","DOI":"10.1007\/978-3-319-32001-4_44-1"},{"key":"e_1_2_1_121_1","doi-asserted-by":"publisher","DOI":"10.1057\/9780230306738"},{"key":"e_1_2_1_122_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998246"},{"key":"e_1_2_1_123_1","volume-title":"Re-humanizing the platform: Content moderators and the logic of care","author":"Ruckenstein Minna","year":"2020","unstructured":"Minna Ruckenstein and Linda Lisa Maria Turunen. 2020. Re-humanizing the platform: Content moderators and the logic of care., Vol. 22, 6 (2020), 1026--1042. Publisher: Sage Publications Sage UK: London, England."},{"key":"e_1_2_1_124_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLEI.2014.6965181"},{"key":"e_1_2_1_125_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1101"},{"key":"e_1_2_1_126_1","doi-asserted-by":"publisher","unstructured":"Angela M. Sch\u00f6pke-Gonzalez Shubham Atreja Han Na Shin Najmin Ahmed and Libby Hemphill. [n. d.]. Why do volunteer content moderators quit? Burnout conflict and harmful behaviors. ( [n. d.]) 146144482211385. https:\/\/doi.org\/10.1177\/14614448221138529","DOI":"10.1177\/14614448221138529"},{"key":"e_1_2_1_127_1","doi-asserted-by":"publisher","DOI":"10.54501\/jots.v1i2.34"},{"key":"e_1_2_1_128_1","volume-title":"Moral injury. Psychoanalytic psychology","author":"Shay Jonathan","year":"2014","unstructured":"Jonathan Shay. 2014. Moral injury. Psychoanalytic psychology, Vol. 31, 2 (2014), 182."},{"key":"e_1_2_1_129_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610194"},{"key":"e_1_2_1_130_1","volume-title":"Employers' Role in Employee Health: Why They Do What They Do. Journal of Occupational & Environmental Medicine","author":"Singer Sara J.","year":"2020","unstructured":"Sara J. Singer, Stacie Vilendrer, Grace Joseph, Jason Kim, and Jeffrey Pfeffer. 2020. Employers' Role in Employee Health: Why They Do What They Do. Journal of Occupational & Environmental Medicine (2020). https:\/\/api.semanticscholar.org\/CorpusID:225410719"},{"key":"e_1_2_1_131_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-016-0350-0"},{"key":"e_1_2_1_132_1","unstructured":"Ranjit Singh Borhane Blili-Hamelin and Jacob Metcalf. 2023. Can we red team our way to AI accountability? https:\/\/techpolicy.press\/can-we-red-team-our-way-to-ai-accountability\/"},{"key":"e_1_2_1_133_1","unstructured":"Daisy Soderberg-Rivkin. 2023. Five myths about online content moderation from a former content moderator. https:\/\/www.rstreet.org\/commentary\/five-myths-about-online-content-moderation-from-a-former-content-moderator\/"},{"key":"e_1_2_1_134_1","unstructured":"Irene Solaiman Zeerak Talat William Agnew Lama Ahmad Dylan Baker Su Lin Blodgett Hal Daum\u00e9 III Jesse Dodge Ellie Evans Sara Hooker Yacine Jernite Alexandra Sasha Luccioni Alberto Lusoli Margaret Mitchell Jessica Newman Marie-Therese Png Andrew Strait and Apostol Vassilev. [n. d.]. Evaluating the Social Impact of Generative AI Systems in Systems and Society. arxiv: 2306.05949 [cs] http:\/\/arxiv.org\/abs\/2306.05949"},{"key":"e_1_2_1_135_1","volume-title":"Well-being at work: a cross-national analysis of the levels and determinants of job satisfaction. The journal of socio-economics","author":"Sousa-Poza Alfonso","year":"2000","unstructured":"Alfonso Sousa-Poza and Andres A Sousa-Poza. 2000. Well-being at work: a cross-national analysis of the levels and determinants of job satisfaction. The journal of socio-economics, Vol. 29, 6 (2000), 517--538."},{"key":"e_1_2_1_136_1","first-page":"16","article-title":"AI and essential labor: representing the invisible work of integration. XRDS: Crossroads","volume":"28","author":"Spektor Franchesca","year":"2022","unstructured":"Franchesca Spektor, Estefania Rodriguez, Samantha Shorey, and Sarah Fox. 2022. AI and essential labor: representing the invisible work of integration. XRDS: Crossroads, The ACM Magazine for Students, Vol. 28, 2 (2022), 16--19.","journal-title":"The ACM Magazine for Students"},{"key":"e_1_2_1_137_1","volume-title":"How color enhances visual memory for natural scenes. Psychological science","author":"Spence Ian","year":"2006","unstructured":"Ian Spence, Patrick Wong, Maria Rusan, and Naghmeh Rastegar. 2006. How color enhances visual memory for natural scenes. Psychological science, Vol. 17, 1 (2006), 1--6."},{"key":"e_1_2_1_138_1","doi-asserted-by":"publisher","DOI":"10.54501\/jots.v1i5.91"},{"key":"e_1_2_1_139_1","unstructured":"Thomas Stackpole. 2022. Content moderation is terrible by design. https:\/\/hbr.org\/2022\/11\/content-moderation-is-terrible-by-design#: :text=Their%20work%20is%20largely%20invisible or%20video%20at%20a%20time."},{"key":"e_1_2_1_140_1","unstructured":"PAI Staff. 2021. Responsible Sourcing of Data Enrichment Services. https:\/\/partnershiponai.org\/paper\/responsible-sourcing-considerations\/"},{"key":"e_1_2_1_141_1","volume-title":"Proceedings of Seventh International Congress on Information and Communication Technology: ICICT","volume":"4","author":"Steiger Miriah","year":"2022","unstructured":"Miriah Steiger, Timir J Bharucha, Wilfredo Torralba, Marlyn Savio, Priyanka Manchanda, and Rachel Lutz-Guevara. 2022. Effects of a Novel Resiliency Training Program for Social Media Content Moderators. In Proceedings of Seventh International Congress on Information and Communication Technology: ICICT 2022, London, Volume 4. Springer, 283--298."},{"key":"e_1_2_1_142_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445092"},{"key":"e_1_2_1_143_1","volume-title":"Establishing and maintaining healthy environments. Toward a social ecology of health promotion. The American psychologist","author":"Stokols Daniel","year":"1992","unstructured":"Daniel Stokols. 1992. Establishing and maintaining healthy environments. Toward a social ecology of health promotion. The American psychologist, Vol. 47 1 (1992), 6--22. https:\/\/api.semanticscholar.org\/CorpusID:12152556"},{"key":"e_1_2_1_144_1","first-page":"18","article-title":"What do we mean when we talk about transparency? Toward meaningful transparency in commercial content moderation","volume":"13","author":"Suzor Nicolas P","year":"2019","unstructured":"Nicolas P Suzor, Sarah Myers West, Andrew Quodling, and Jillian York. 2019. What do we mean when we talk about transparency? Toward meaningful transparency in commercial content moderation. International Journal of Communication, Vol. 13 (2019), 18.","journal-title":"International Journal of Communication"},{"key":"e_1_2_1_145_1","volume-title":"Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al.","author":"Thoppilan Romal","year":"2022","unstructured":"Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022)."},{"key":"e_1_2_1_146_1","volume-title":"Indirect exposure to the trauma of others: The experiences of 9--1--1 telecommunicators. Ph.,D. Dissertation","author":"Troxell Roberta Mary","unstructured":"Roberta Mary Troxell. 2008. Indirect exposure to the trauma of others: The experiences of 9--1--1 telecommunicators. Ph.,D. Dissertation. University of Illinois at Chicago Doctoral dissertation."},{"key":"e_1_2_1_147_1","volume-title":"Employee involvement and participation in digital transformation: a combined analysis of literature and practitioners' expertise. Journal of Organizational Change Management","author":"Ullrich Andr\u00e9","year":"2023","unstructured":"Andr\u00e9 Ullrich, Malte Rei\u00dfig, Silke Niehoff, and Grischa Beier. 2023. Employee involvement and participation in digital transformation: a combined analysis of literature and practitioners' expertise. Journal of Organizational Change Management (2023). https:\/\/api.semanticscholar.org\/CorpusID:258996239"},{"key":"e_1_2_1_148_1","doi-asserted-by":"publisher","DOI":"10.1109\/AVSS.2012.16"},{"key":"e_1_2_1_149_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.54.6.1063"},{"key":"e_1_2_1_150_1","doi-asserted-by":"publisher","DOI":"10.4088\/JCP.12r08225"},{"key":"e_1_2_1_151_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300390"},{"key":"e_1_2_1_152_1","volume-title":"Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00","volume":"2","author":"Wood Bradley J","year":"2000","unstructured":"Bradley J Wood and Ruth A Duggan. 2000. Red teaming of advanced information assurance concepts. In Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00, Vol. 2. IEEE, 112--118."},{"key":"e_1_2_1_153_1","doi-asserted-by":"publisher","DOI":"10.1006\/jrpe.1997.2162"},{"key":"e_1_2_1_154_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052591"},{"key":"e_1_2_1_155_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.235"},{"key":"e_1_2_1_156_1","volume-title":"Clinical Decision-Making Processes. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems","author":"Yang Qian","year":"2019","unstructured":"Qian Yang, Aaron Steinfeld, and John Zimmerman. 2019. Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019). https:\/\/api.semanticscholar.org\/CorpusID:127989976"},{"key":"e_1_2_1_157_1","volume-title":"Case study research: Design and methods","author":"Yin Robert K","unstructured":"Robert K Yin. 2009. Case study research: Design and methods. Vol. 5. sage."},{"key":"e_1_2_1_158_1","volume-title":"Red Team: How to succeed by thinking like the enemy","author":"Zenko Micah","year":"2015","unstructured":"Micah Zenko. 2015. Red Team: How to succeed by thinking like the enemy. Basic Books."},{"key":"e_1_2_1_159_1","unstructured":"Micah Zenko and Richard Haass. 2015. \"red team: How to succeed by thinking like the enemy\". https:\/\/www.cfr.org\/event\/red-team-how-succeed-thinking-enemy"},{"key":"e_1_2_1_160_1","doi-asserted-by":"publisher","DOI":"10.1145\/3584931.3607013"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710931","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3710931","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T09:21:47Z","timestamp":1755768107000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710931"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":160,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,5,2]]}},"alternative-id":["10.1145\/3710931"],"URL":"https:\/\/doi.org\/10.1145\/3710931","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"2025-05-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}