{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T21:13:52Z","timestamp":1774559632308,"version":"3.50.1"},"reference-count":103,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"This research received funding from Good in Tech research network, under the aegis of the Fondation du Risque in partnership with Institut Mines-T\u00e9l\u00e9com and Sciences Po."}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Big Data &amp; Society"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p> This article examines algorithmic content moderation, using the moderation of violent extremist content as a specific case. In recent years, algorithms have increasingly been mobilized to perform essential moderation functions for online social media platforms such as Facebook, YouTube, and Twitter, including limiting the proliferation of extremist speech. Drawing on Katherine Hayles\u2019 concept of \u201ccognitive assemblages\u201d and the Critical Security Studies literature, we show how algorithmic regulation operates within larger assemblages of humans and non-humans to influence the surveillance and regulation of information flows. We argue that the dynamics of algorithmic regulation are more liquid, cobbled together and distributed than it appears. It is characterized by a set of shifting human and machine entities, which mix traditional surveillance methods with more sophisticated tools, and whose linkages and interactions are transient. The processes that enable the consolidation of knowledge about risky profiles and contents are, therefore, collective and distributed among humans and machines. This allows us to argue that the cognitive assemblages involved in content moderation become a cobbled space of preemptive calculation. <\/jats:p>","DOI":"10.1177\/20539517221143361","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T09:03:27Z","timestamp":1670835807000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":9,"title":["Cognitive assemblages: The entangled nature of algorithmic content moderation"],"prefix":"10.1177","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5659-1124","authenticated-orcid":false,"given":"Valentine","family":"Crosset","sequence":"first","affiliation":[{"name":"M\u00e9dialab Sciences Po, Paris, France"}]},{"given":"Beno\u00eet","family":"Dupont","sequence":"additional","affiliation":[{"name":"International Centre for Comparative Criminology, Universit\u00e9 de Montr\u00e9al, Montreal, Canada"}]}],"member":"179","published-online":{"date-parts":[[2022,12,12]]},"reference":[{"key":"bibr1-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/j.1475-4762.2011.01005.x"},{"key":"bibr2-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1016\/j.polgeo.2006.02.001"},{"key":"bibr3-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8330.2008.00655.x"},{"key":"bibr4-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1215\/9780822377269"},{"key":"bibr5-20539517221143361","volume-title":"Cloud Ethics: Algorithms and the Attributes of Ourselves and Others","author":"Amoore L","year":"2020"},{"key":"bibr6-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010616680753"},{"key":"bibr7-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0162243915606523"},{"issue":"2","key":"bibr8-20539517221143361","first-page":"352","volume":"4","author":"Ananny M","year":"2020","journal-title":"Georgetown Law Technology Review"},{"key":"bibr9-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0309132510362600"},{"key":"bibr10-20539517221143361","doi-asserted-by":"publisher","DOI":"10.24908\/ss.v2i4.3359"},{"key":"bibr11-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715609066"},{"key":"bibr12-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1368431016667623"},{"key":"bibr13-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1017\/eis.2017.14"},{"key":"bibr14-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1354066107074290"},{"key":"bibr15-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1206331211430015"},{"key":"bibr16-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1461444809336551"},{"key":"bibr17-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/jcms.13306"},{"key":"bibr18-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/rego.12338"},{"key":"bibr19-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010614545200"},{"key":"bibr20-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715622512"},{"key":"bibr21-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1007\/s10699-016-9489-4"},{"key":"bibr22-20539517221143361","volume-title":"\u00c0 quoi r\u00eavent les algorithmes : Nos vies \u00e0 l\u2019heure des big data","author":"Cardon D","year":"2015"},{"key":"bibr23-20539517221143361","volume-title":"We are data: Algorithms and the Making of our Digital Selves","author":"Cheney-Lippold J","year":"2018"},{"key":"bibr24-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0162243915589635"},{"key":"bibr25-20539517221143361","doi-asserted-by":"publisher","DOI":"10.5749\/minnesota\/9780816675890.001.0001"},{"key":"bibr26-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8330.2012.01039.x"},{"key":"bibr27-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0263775815599309"},{"key":"bibr28-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010614543585"},{"key":"bibr29-20539517221143361","unstructured":"De Smedt T, De Pauw G, Van Ostaeyen P (2018) Automatic detection of online jihadist hate speech. arXiv preprint arXiv:1803.04596."},{"key":"bibr30-20539517221143361","doi-asserted-by":"publisher","DOI":"10.3366\/edinburgh\/9780748620920.003.0013"},{"key":"bibr31-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1016\/j.geoforum.2006.08.004"},{"key":"bibr32-20539517221143361","unstructured":"Facebook (2017a, June 15) Hard questions: How we counter terrorism. Facebook Newsroom. https:\/\/about.fb.com\/news\/2017\/06\/how-we-counter-terrorism\/."},{"key":"bibr33-20539517221143361","unstructured":"Facebook (2017b, November 28) Are we winning the war on terrorism online? Facebook Newsroom. https:\/\/about.fb.com\/news\/2017\/11\/hard-questions-are-we-winning-the-war-on-terrorism-online\/."},{"key":"bibr34-20539517221143361","unstructured":"Facebook (2018a, July 31) Removing bad actors on facebook. Facebook Newsroom. https:\/\/newsroom.fb.com\/news\/2018\/07\/removing-bad-actors-on-facebook\/."},{"key":"bibr35-20539517221143361","unstructured":"Facebook (2018b, September 17) Combating hate and extremism. Facebook Newsroom. https:\/\/about.fb.com\/news\/2019\/09\/combating-hate-and-extremism\/."},{"key":"bibr36-20539517221143361","unstructured":"Facebook (2021) Facebook Transparency report. https:\/\/transparency.facebook.com\/community-standards-enforcement#terrorist-propaganda [accessed 8 September 2021]."},{"key":"bibr37-20539517221143361","unstructured":"Flick M, Paresh D (2019, April 23) Facebook\u2019s flood of languages leaves it struggling to monitor content. Reuters. https:\/\/www.reuters.com\/article\/us-facebook-languages-insight-idUSKCN1RZ0DW."},{"key":"bibr38-20539517221143361","volume-title":"Security, Territory, Population. Lectures at the College de France 1977\u20131978","author":"Foucault M","year":"1978"},{"key":"bibr39-20539517221143361","volume-title":"The Culture of Control: Crime and Social Order in Contemporary Society","author":"Garland D","year":"2012"},{"key":"bibr40-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1527476416680453"},{"key":"bibr41-20539517221143361","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/9042.003.0013"},{"key":"bibr42-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1515\/9781400880553-004"},{"key":"bibr43-20539517221143361","volume-title":"Custodians of the Internet: Platforms, Content Moderation, and the Information Society and Culture Hidden Decisions that Shape Social Media","author":"Gillespie T","year":"2018"},{"key":"bibr44-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/2053951720943234"},{"key":"bibr45-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/2053951719897945"},{"key":"bibr46-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1024258916687250"},{"key":"bibr47-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1080\/00071310020015280"},{"key":"bibr48-20539517221143361","doi-asserted-by":"publisher","DOI":"10.7208\/chicago\/9780226447919.001.0001"},{"key":"bibr49-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010616671642"},{"key":"bibr50-20539517221143361","unstructured":"Highman S, Nakashima E (2015, July 16) Why the Islamic State Leaves Tech Companies Torn Between Free Speech and Security. Washington Post. https:\/\/www.washingtonpost.com\/world\/national-security\/islamic-states-embrace-of-social-media-puts-tech-companies-in-a-bind\/2015\/07\/15\/0e5624c4-169c-11e5-89f3-61410da94eb1_story.html."},{"key":"bibr51-20539517221143361","volume-title":"Google and the Culture of Search","author":"Hillis K","year":"2013"},{"key":"bibr52-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010614544312"},{"issue":"2","key":"bibr53-20539517221143361","first-page":"177","volume":"2","author":"Introna L","year":"2004","journal-title":"Surveillance & Society"},{"key":"bibr54-20539517221143361","unstructured":"Kirschenbaum MG (2003) Virtuality and VRML: Software studies after Manovich. Electronic Book Review, pp. 1\u201320. https:\/\/electronicbookreview.com\/essay\/virtuality-and-vrml-software-studies-after-manovich\/"},{"key":"bibr55-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2016.1154087"},{"key":"bibr56-20539517221143361","first-page":"1598","volume":"131","author":"Klonick K","year":"2017","journal-title":"Harv. L. Rev"},{"key":"bibr57-20539517221143361","unstructured":"Klonick K (2019, April 25) Inside the team at Facebook that dealt with the Christchurch Shooting. The New Yorker. https:\/\/www.newyorker.com\/news\/news-desk\/inside-the-team-at-facebook-that-dealt-with-the-christchurch-shooting."},{"key":"bibr58-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1080\/03085147.2016.1220665"},{"key":"bibr59-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0263276407075956"},{"key":"bibr60-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780199256044.001.0001"},{"key":"bibr61-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0263276419829541"},{"key":"bibr62-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2013.126"},{"key":"bibr63-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1353\/tae.2007.0066"},{"key":"bibr64-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1215\/9780822375197"},{"key":"bibr65-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/1367549415577389"},{"key":"bibr66-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/2053951716679679"},{"key":"bibr67-20539517221143361","volume":"2","author":"Miyazaki S","year":"2012","journal-title":"Computational Culture"},{"issue":"1","key":"bibr68-20539517221143361","volume":"2","author":"Munn L","year":"2017","journal-title":"Studies in Control Societies"},{"key":"bibr69-20539517221143361","volume-title":"Ferocious Logics: Unmaking the Algorithm","author":"Munn L","year":"2018"},{"key":"bibr70-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2016.1156141"},{"key":"bibr71-20539517221143361","doi-asserted-by":"publisher","DOI":"10.18574\/nyu\/9781479833641.001.0001"},{"key":"bibr72-20539517221143361","volume-title":"Weapons of Math Destruction. How Big Data Increases Inequality and Threatens Democracy","author":"O\u2019Neil C","year":"2016"},{"key":"bibr73-20539517221143361","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7595.001.0001"},{"key":"bibr74-20539517221143361","first-page":"125","volume-title":"Augmented Intelligence Trauma","author":"Parisi L","year":"2015"},{"key":"bibr75-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0263276418818889"},{"key":"bibr76-20539517221143361","volume-title":"Black Box Society. The Secret Algorithms that Control the Economy and Information","author":"Pasquale F","year":"2015"},{"key":"bibr77-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/20539517211046181"},{"key":"bibr78-20539517221143361","doi-asserted-by":"publisher","DOI":"10.2307\/j.ctvhrcz0v"},{"key":"bibr79-20539517221143361","doi-asserted-by":"publisher","DOI":"10.2307\/591464"},{"key":"bibr80-20539517221143361","first-page":"143","volume-title":"Privacy, Due Process and the Computational Turn. Philosophers of Law Meet Philosophers of Technology","author":"Rouvroy A","year":"2013"},{"key":"bibr81-20539517221143361","doi-asserted-by":"crossref","unstructured":"Schmidt A, Wiegand M (2017) A survey on hate speech detection using natural language processing. In Proceedings of the fifth international workshop on natural language processing for social media, pp. 1\u201310. https:\/\/aclanthology.org\/W17-1101\/","DOI":"10.18653\/v1\/W17-1101"},{"key":"bibr82-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1093\/joc\/jqy029"},{"key":"bibr83-20539517221143361","unstructured":"Scott M, Kayli L (2020, October 21) What happened when humans stopped managing social media content. Politico, pp. 1\u201316. https:\/\/www.politico.eu\/article\/facebook-content-moderation-automation\/"},{"key":"bibr84-20539517221143361","first-page":"1","volume":"8","author":"Seaver N","year":"2013","journal-title":"Media in Transition"},{"key":"bibr85-20539517221143361","unstructured":"Twitter (2016, February 5) Combating violent extremism. Twitter Blog. https:\/\/blog.twitter.com\/en_us\/a\/2016\/combating-violent-extremism.html."},{"key":"bibr86-20539517221143361","unstructured":"Twitter (2017, June 26) Global internet forum to counter terrorism. Twitter Blog. https:\/\/blog.twitter.com\/en_us\/topics\/company\/2017\/Global-Internet-Forum-to-Counter-Terrorism.html."},{"key":"bibr87-20539517221143361","unstructured":"Twitter (2019, March) Terrorism and violent extremism policy. Help Center. https:\/\/help.twitter.com\/en\/rules-and-policies\/violent-groups."},{"key":"bibr88-20539517221143361","unstructured":"Twitter (2021, July 14) An update to the twitter transparency center. Twitter Blog. https:\/\/blog.twitter.com\/en_us\/topics\/company\/2021\/an-update-to-the-twitter-transparency-center."},{"key":"bibr89-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/rego.12437"},{"key":"bibr90-20539517221143361","doi-asserted-by":"publisher","DOI":"10.4159\/harvard.9780674065086"},{"key":"bibr91-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0967010616657947"},{"key":"bibr92-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1111\/rego.12158"},{"key":"bibr93-20539517221143361","volume-title":"Silicon Values: The Future of Free Speech Under Surveillance Capitalism","author":"York JC","year":"2021"},{"key":"bibr94-20539517221143361","unstructured":"YouTube (2008, May 19) Dialogue with Sen. Lieberman on terrorism video. Official Blog. https:\/\/blog.youtube\/news-and-events\/dialogue-with-sen-lieberman-on\/"},{"key":"bibr95-20539517221143361","unstructured":"YouTube (2017a, August 1) An update on our commitment to fight terror content online. Official Blog. https:\/\/youtube.googleblog.com\/2017\/08\/an-update-on-our-commitment-to-fight.html."},{"key":"bibr96-20539517221143361","unstructured":"YouTube (2017b, December 4) Expanding our work against abuse of our platform. Official Blog. https:\/\/youtube.googleblog.com\/2017\/12\/expanding-our-work-against-abuse-of-our.html."},{"key":"bibr97-20539517221143361","unstructured":"YouTube (2018, April 23) More information, faster removals, more people\u2014an update on what we\u2019re doing to enforce YouTube\u2019s Community Guidelines. Official Blog. https:\/\/youtube.googleblog.com\/2018\/04\/more-information-faster-removals-more.html."},{"key":"bibr98-20539517221143361","unstructured":"YouTube (2021) Application of the YouTube community rules. Information transparency. https:\/\/transparencyreport.google.com\/youtube-policy\/removals?hl=fr. [accessed May 15, 2021]"},{"key":"bibr99-20539517221143361","unstructured":"YouTube (n.d.) YouTube trusted flagger program. YouTube Support. https:\/\/support.google.com\/youtube\/answer\/7554338?hl=fr [accessed 15 May 2019]."},{"key":"bibr100-20539517221143361","doi-asserted-by":"publisher","DOI":"10.1177\/0162243915608948"},{"key":"bibr101-20539517221143361","volume-title":"The Age of Surveillance Capitalism: The Fight for the Future at the New Frontier of Power","author":"Zuboff S","year":"2019"},{"key":"bibr102-20539517221143361","unstructured":"Zuckerberg M (2017, February 16) Building global community. Facebook Publication. https:\/\/www.facebook.com\/notes\/mark-zuckerberg\/building-global-community\/10154544292806634\/."},{"key":"bibr103-20539517221143361","unstructured":"Zuckerberg M (2018, April 10) Mark Zuckerberg testifies before Congress. https:\/\/www.youtube.com\/watch?v=mZaec_mlq9M."}],"container-title":["Big Data &amp; Society"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517221143361","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/20539517221143361","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/20539517221143361","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T03:34:25Z","timestamp":1740972865000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/20539517221143361"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":103,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1177\/20539517221143361"],"URL":"https:\/\/doi.org\/10.1177\/20539517221143361","relation":{},"ISSN":["2053-9517","2053-9517"],"issn-type":[{"value":"2053-9517","type":"print"},{"value":"2053-9517","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"article-number":"20539517221143361"}}