{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T22:42:07Z","timestamp":1782168127534,"version":"3.54.5"},"reference-count":60,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,10,14]],"date-time":"2019-10-14T00:00:00Z","timestamp":1571011200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,10,14]],"date-time":"2019-10-14T00:00:00Z","timestamp":1571011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ethics Inf Technol"],"published-print":{"date-parts":[[2020,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper we explore quarantining as a more ethical method for delimiting the spread of Hate Speech via online social media platforms. Currently, companies like Facebook, Twitter, and Google generally respond<jats:italic>reactively<\/jats:italic>to such material: offensive messages that have already been posted are reviewed by human moderators if complaints from users are received. The offensive posts are only<jats:italic>subsequently<\/jats:italic>removed if the complaints are upheld; therefore, they still cause the recipients psychological harm. In addition, this approach has frequently been criticised for delimiting freedom of expression, since it requires the service providers to elaborate and implement censorship regimes. In the last few years, an emerging generation of automatic Hate Speech detection systems has started to offer new strategies for dealing with this particular kind of offensive online material. Anticipating the future efficacy of such systems, the present article advocates an approach to online Hate Speech detection that is analogous to the quarantining of malicious computer software. If a given post is automatically classified as being harmful in a reliable manner, then it can be temporarily quarantined, and the direct recipients can receive an alert, which protects them from the harmful content in the first instance. The quarantining framework is an example of more ethical online safety technology that can be extended to the handling of Hate Speech. Crucially, it provides flexible options for obtaining a more justifiable balance between freedom of expression and appropriate censorship.<\/jats:p>","DOI":"10.1007\/s10676-019-09516-z","type":"journal-article","created":{"date-parts":[[2019,10,14]],"date-time":"2019-10-14T18:39:26Z","timestamp":1571078366000},"page":"69-80","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":77,"title":["Quarantining online hate speech: technical and ethical perspectives"],"prefix":"10.1007","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5069-6152","authenticated-orcid":false,"given":"Stefanie","family":"Ullmann","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus","family":"Tomalin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,10,14]]},"reference":[{"key":"9516_CR1","unstructured":"Allan, R. (2017, June 27). Hard questions: Who should decide what is hate speech in an online global community? Facebook Newsroom. Retrieved January 28, 2019 from https:\/\/newsroom.fb.com\/news\/2017\/06\/hard-questions-hate-speech\/ ."},{"key":"9516_CR2","unstructured":"Aristotle. (1984[1782]). Nicomachean ethics 5.2.1129b23. In Aristotle, [4th century BCE], vol. 2."},{"issue":"2","key":"9516_CR3","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1007\/s12115-017-0114-0","volume":"54","author":"I Awan","year":"2017","unstructured":"Awan, I. (2017). Cyber-extremism: Isis and the power of social media. Society,54(2), 138\u2013149. https:\/\/doi.org\/10.1007\/s12115-017-0114-0 .","journal-title":"Society"},{"key":"9516_CR4","unstructured":"Bates, L. (2013, May 29). The day the Everyday Sexism Project won - and Facebook changed its image. The Independent. Retrieved April 12, 2019 from https:\/\/www.independent.co.uk\/voices\/comment\/the-day-the-everyday-sexism-project-won-and-facebook-changed-its-image-8636661.html ."},{"key":"9516_CR5","unstructured":"BBC. (2019). What is the Parental Guidance Lock? Retrieved April 12, 2019 from https:\/\/www.bbc.co.uk\/iplayer\/help\/how-to-guides\/parental-guidance\/parental_guidance_info ."},{"key":"9516_CR41","unstructured":"BBC News. (2019, March 28). Facebook to ban white nationalism and separatism. Retrieved April 12, 2019 from https:\/\/www.bbc.co.uk\/news\/world-us-canada-47728471 ."},{"key":"9516_CR6","unstructured":"Benesch, S. (2019). The Dangerous Speech Project. Retrieved April 12, 2019 from https:\/\/dangerousspeech.org\/ ."},{"key":"9516_CR7","first-page":"118","volume-title":"Four essays on liberty","author":"I Berlin","year":"1969","unstructured":"Berlin, I. (1969). Two concepts of liberty. In I. Berlin (Ed.), Four essays on liberty (pp. 118\u2013172). Oxford: Oxford University Press."},{"key":"9516_CR8","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1093\/jaarel\/lfq020","volume":"78","author":"BM Britt","year":"2010","unstructured":"Britt, B. M. (2010). Curses left and right: Hate speech in the biblical tradition. Journal of the American Academy of Religion,78, 633\u2013661.","journal-title":"Journal of the American Academy of Religion"},{"key":"9516_CR9","doi-asserted-by":"publisher","DOI":"10.4324\/9781315714899","volume-title":"Hate speech law: A philosophical examination","author":"A Brown","year":"2015","unstructured":"Brown, A. (2015). Hate speech law: A philosophical examination. Abingdon: Routledge."},{"key":"9516_CR10","volume-title":"Excitable speech: A politics of the performative","author":"J Butler","year":"1997","unstructured":"Butler, J. (1997). Excitable speech: A politics of the performative. Abingdon: Routledge."},{"key":"9516_CR11","volume-title":"Improving cyberbullying detection with user context. Proceedings of the 35th European Conference on IR Research: Advances in Information Retrieval","author":"M Dadvar","year":"2013","unstructured":"Dadvar, M., Trieschnigg, D., Ordelman, R., & de Jong, F. (2013). Improving cyberbullying detection with user context. In Proceedings of the 35th European Conference on IR Research: Advances in Information Retrieval. Moscow: European Conference on IR Research."},{"issue":"2","key":"9516_CR12","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1109\/MIC.2019.2893137","volume":"23","author":"F Daniel","year":"2019","unstructured":"Daniel, F., Cappiello, C., & Benatallah, B. (2019). Bots acting like humans: Understanding and preventing harm. IEEE Internet Computing,23(2), 40\u201349.","journal-title":"IEEE Internet Computing"},{"key":"9516_CR13","unstructured":"Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017). Automated hate speech detection and the problem of offensive language. In Proceedings of the Eleventh International AAAI Conference on Web and Social Media (ICWSM), . (pp. 512\u2013515). Atlanta: ICWSM."},{"issue":"1","key":"9516_CR14","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1080\/03064220500532412","volume":"35","author":"R Dworkin","year":"2006","unstructured":"Dworkin, R. (2006). A new map of censorship. Index On Censorship,35(1), 131\u2013133.","journal-title":"Index On Censorship"},{"key":"9516_CR15","volume-title":"Extreme speech and democracy","author":"R Dworkin","year":"2009","unstructured":"Dworkin, R. (2009). Foreword. In I. Hare & J. Weinstein (Eds.), Extreme speech and democracy. Oxford: Oxford University Press."},{"key":"9516_CR17","unstructured":"European Commission. (2019, March 18). Countering illegal hate speech online #NoPlace4Hate. European Commission Justice and Consumers Newsroom. Retrieved April 12, 2019 from https:\/\/dangerousspeech.org ."},{"key":"9516_CR18","unstructured":"Facebook. (2019). Community standards Part III: Objectionable content. Retrieved April 12, 2019 from https:\/\/www.facebook.com\/communitystandards\/objectionable_content ."},{"key":"9516_CR19","unstructured":"Facebook. (2019). Community standards: Hate speech. Retrieved February 18, 2019 from https:\/\/www.facebook.com\/communitystandards\/objectionable_content ."},{"key":"9516_CR20","unstructured":"Facebook. (2019). Zero friction future. Retrieved April 12, 2019 from https:\/\/www.facebook.com\/business\/m\/zero-friction-future ."},{"key":"9516_CR21","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199641321.001.0001","volume-title":"The ethics of information","author":"L Floridi","year":"2013","unstructured":"Floridi, L. (2013). The ethics of information. Oxford: Oxford University Press."},{"issue":"4","key":"9516_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3232676","volume":"51","author":"P Fortuna","year":"2018","unstructured":"Fortuna, P., & Nunes, S. (2018). A survey on automatic detection of hate speech in Text. ACM Computing Surveys,51(4), 1\u201330.","journal-title":"ACM Computing Surveys"},{"key":"9516_CR23","unstructured":"Gadde, V., Harvey, D. (2018). Creating new policies together. Twitter Blog, 25 September 2018. Retrieved February 18, 2019 from https:\/\/blog.twitter.com\/en_us\/topics\/company\/2018\/Creating-new-policies-together.html ."},{"issue":"4","key":"9516_CR24","doi-asserted-by":"publisher","first-page":"769","DOI":"10.2307\/2586111","volume":"93","author":"WA Galston","year":"1999","unstructured":"Galston, W. A. (1999). Value pluralism and liberal political theory. The American Political Science Review,93(4), 769\u2013778.","journal-title":"The American Political Science Review"},{"key":"9516_CR25","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-3013","author":"B Gamb\u00e4ck","year":"2017","unstructured":"Gamb\u00e4ck, B., & Sikdar, U. K. (2017). Using convolutional neural networks to classify hate-speech. In Proceedings of the First Workshop on Abusive Language Online. https:\/\/doi.org\/10.18653\/v1\/W17-3013 .","journal-title":"Proceedings of the First Workshop on Abusive Language Online"},{"key":"9516_CR26","unstructured":"Gao, L., Kuppersmith, A., & Huang, R. (2017). Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1)"},{"issue":"3","key":"9516_CR27","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1080\/13504630.2015.1128810","volume":"22","author":"K Gelber","year":"2016","unstructured":"Gelber, K., & McNamara, L. J. (2016). Evidencing the harms of hate speech. Social Identities,22(3), 324\u2013341.","journal-title":"Social Identities"},{"key":"9516_CR28","unstructured":"Google. (2019). YouTube help: Hate speech policy. Retrieved April 12, 2019 from https:\/\/support.google.com\/youtube\/answer\/2801939?hl=en-GB ."},{"key":"9516_CR29","unstructured":"HM Government. (2019). Online Harms White Paper, April 2019. Retrieved April 12, 2019 from https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/793360\/Online_Harms_White_Paper.pdf ."},{"key":"9516_CR30","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780198759027.001.0001","volume-title":"Hate speech and democratic citizenship","author":"E Heinze","year":"2016","unstructured":"Heinze, E. (2016). Hate speech and democratic citizenship. Oxford: Oxford University Press."},{"key":"9516_CR31","doi-asserted-by":"crossref","unstructured":"Hosseinmardi, H., Mattson, S. A., Rafiq, R. I., Han, R., Lv, Q., & Mishra, S. (2015). Detection of cyberbullying incidents on the Instagram social network. Computing Research Repository (CoRR) https:\/\/arxiv.org\/abs\/1503.03909 .","DOI":"10.1007\/978-3-319-27433-1_4"},{"issue":"1","key":"9516_CR32","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1177\/0081246317728165","volume":"48","author":"M Judge","year":"2018","unstructured":"Judge, M., & Nel, J. A. (2018). Psychology and hate speech: A critical and restorative encounter. South African Journal of Psychology,48(1), 15\u201320.","journal-title":"South African Journal of Psychology"},{"key":"9516_CR33","unstructured":"Kjierland S, Baumgartner P (2018) Anti-spam and anti-malware protection [EOP]. Microsoft Docs, Retrieved April 15, 2019 from https:\/\/docs.microsoft.com\/en-us\/office365\/servicedescriptions\/exchange-online-protection-service-description\/anti-spam-and-anti-malware-protection-eop ."},{"key":"9516_CR34","volume-title":"Multimodality: A social semiotic approach to contemporary communication","author":"G Kress","year":"2010","unstructured":"Kress, G. (2010). Multimodality: A social semiotic approach to contemporary communication. London: Routledge."},{"issue":"3","key":"9516_CR35","first-page":"431","volume":"39","author":"CR Lawrence","year":"1990","unstructured":"Lawrence, C. R. (1990). If he hollers, let him go: Regulating hate speech on campus. Controversies in Constitutional Law,39(3), 431\u2013483.","journal-title":"Controversies in Constitutional Law"},{"key":"9516_CR36","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199236282.001.0001","volume-title":"Speech and harm: Controversies over free speech","author":"I Maitra","year":"2012","unstructured":"Maitra, I., & McGowan, M. K. (2012). Speech and harm: Controversies over free speech. New York: Oxford University Press."},{"key":"9516_CR37","doi-asserted-by":"publisher","unstructured":"Mathew, B., Tharad, H., Rajgaria, S., Singhania, P., Maity, S. K., Goyal, P., & Mukherje, A. (2018). Thou shalt not hate: Countering Online Hate Speech. ICWSM 2019. https:\/\/doi.org\/10.13140\/RG.2.2.31128.85765 .","DOI":"10.13140\/RG.2.2.31128.85765"},{"key":"9516_CR38","unstructured":"Matsakis, L. (2018). Twitter releases new policy on \u2018dehumanizing speech\u2019. Wired, 25 September 2018. Retrieved February 18, 2019 from https:\/\/www.wired.com\/story\/twitter-dehumanizing-speech-policy\/ ."},{"key":"9516_CR39","first-page":"17","volume-title":"Words that wound: Critical race theory, assaultive speech, and the first amendment","author":"MJ Matsuda","year":"1993","unstructured":"Matsuda, M. J. (1993). Public response to racist speech: Considering the victim\u2019s story. In M. J. Matsuda, C. R. Lawrence III, R. Delgado, & K. Williams (Eds.), Words that wound: Critical race theory, assaultive speech, and the first amendment (pp. 17\u201352). New York: Routledge."},{"key":"9516_CR40","volume-title":"Defense and detection strategies against internet worms","author":"J Nazario","year":"2004","unstructured":"Nazario, J. (2004). Defense and detection strategies against internet worms. Boston: Artech House."},{"key":"9516_CR42","unstructured":"Newton C (2019) The trauma floor: The secret lives of Facebook moderators in America. The Verge, 25 February 2019, Retrieved April 01, 2019 from https:\/\/www.theverge.com\/2019\/2\/25\/18229714\/cognizant-facebook-content-moderator-interviews-trauma-working-conditions-arizona ."},{"key":"9516_CR43","doi-asserted-by":"crossref","unstructured":"Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., & Chang, Y. (2016). Abusive language detection in online user content. In Proceedings of the 25th International Conference on World Wide Web, (pp. 145\u2013153).","DOI":"10.1145\/2872427.2883062"},{"key":"9516_CR44","doi-asserted-by":"publisher","DOI":"10.1145\/32010643201081","author":"R Ottoni","year":"2018","unstructured":"Ottoni, R., Bernardina, P., Cunha, E., Meira, W., Jr., Magno, G., & Almeida, V. (2018). Analyzing right-wing YouTube channels: Hate. Violence and Discrimination. https:\/\/doi.org\/10.1145\/32010643201081 .","journal-title":"Violence and Discrimination"},{"key":"9516_CR45","doi-asserted-by":"crossref","unstructured":"Qian, J., El Sherief, M., Belding-Royer, E. M., & Wang, W. Y. (2018). Hierarchical CVAE for fine-grained hate speech classification.In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, (pp. 3550\u20133559).","DOI":"10.18653\/v1\/D18-1391"},{"key":"9516_CR46","unstructured":"Reddit. (2019). Account and community restrictions: Quarantining subreddits. Retrieved August 12, 2019 from https:\/\/www.reddithelp.com\/en\/categories\/rules-reporting\/account-and-community-restrictions\/quarantined-subreddits ."},{"key":"9516_CR47","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-32001-4","volume-title":"Encyclopedia of big data","author":"ST Roberts","year":"2017","unstructured":"Roberts, S. T. (2017). Content moderation. In L. Schintler & C. McNeely (Eds.), Encyclopedia of big data. Cham: Springer. https:\/\/doi.org\/10.1007\/978-3-319-32001-4 ."},{"key":"9516_CR48","doi-asserted-by":"crossref","unstructured":"Schmidt, A., Wiegand, M. (2017). A survey on hate speech detection using natural language processing\u2019. In Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media (pp. 1\u201310).","DOI":"10.18653\/v1\/W17-1101"},{"key":"9516_CR49","doi-asserted-by":"crossref","unstructured":"Schmidt, A., Wiegand, M. (2017). A survey on hate speech using natural language processing. In Proceedings of the Fifth International workshop on natural language processing for social media (pp. 1\u201310).","DOI":"10.18653\/v1\/W17-1101"},{"key":"9516_CR50","unstructured":"Simon, S., Bowman, E. (2019). Propaganda, hate speech, violence: The working lives of Facebook\u2019s content moderators. NPR, 02 March 2019, Retrieved April 01, 2019 from https:\/\/www.npr.org\/2019\/03\/02\/699663284\/the-working-lives-of-facebooks-content-moderators ."},{"key":"9516_CR51","first-page":"267","volume":"28","author":"MG Sindoni","year":"2018","unstructured":"Sindoni, M. G. (2018). Direct hate speech versus indirect fear speech: A multimodal critical discourse analysis of the Sun\u2019s editorial \u201c1 in 5 Brit Muslims\u2019 sympathy for jihadis\u201d. Lingue Linguaggi,28, 267\u2013292.","journal-title":"Lingue Linguaggi"},{"key":"9516_CR52","first-page":"17","volume-title":"The Oxford handbook of philosophy of criminal law","author":"LW Sumner","year":"2011","unstructured":"Sumner, L. W. (2011). Criminalizing expression: Hate speech and obscenity. In J. Deigh & D. Dolinko (Eds.), The Oxford handbook of philosophy of criminal law (pp. 17\u201333). Oxford: Oxford University Press."},{"key":"9516_CR53","unstructured":"Twitter. (2019). Hateful conduct policy. Retrieved April 12, 2019 from https:\/\/help.twitter.com\/en\/rules-and-policies\/hateful-conduct-policy ."},{"key":"9516_CR54","unstructured":"Union Magazine. (2015). Tommy Robinson interview for UNION magazine Edition#2. Retrieved August 18, 2019 from https:\/\/help.twitter.com\/en\/rules-and-policies\/hateful-conduct-policyhttps:\/\/www.youtube.com\/watch?v=5dRiBRM-BD8 ."},{"key":"9516_CR55","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1163\/9789004382022_007","volume-title":"Handbook of conspiracy theory and contemporary religion","author":"JE Uscinski","year":"2018","unstructured":"Uscinski, J. E., DeWitt, D., & Atkinson, M. D. (2018). A web of conspiracy? internet and conspiracy theory. In A. Dyrendal, D. G. Robertson, & E. Asprem (Eds.), Handbook of conspiracy theory and contemporary religion (pp. 106\u2013130). Leiden: Brill."},{"key":"9516_CR56","doi-asserted-by":"publisher","DOI":"10.4159\/harvard.9780674065086","volume-title":"The harm in hate speech","author":"J Waldron","year":"2012","unstructured":"Waldron, J. (2012). The harm in hate speech. Cambridge: Harvard University Press."},{"key":"9516_CR57","first-page":"697","volume":"32","author":"J Waldron","year":"2017","unstructured":"Waldron, J. (2017). The conditions of legitimacy: A response to James Weinstein. Constitutional Commentary,32, 697\u2013714.","journal-title":"Constitutional Commentary"},{"key":"9516_CR58","unstructured":"Warner, W., Hirschberg, J. (2012). Detecting hate speech on the World Wide Web. In Proceedings of the 2012 Workshop on Language in Social Media (LSM 2012), (pp. 19\u201326)."},{"key":"9516_CR59","first-page":"527","volume":"32","author":"J Weinstein","year":"2017","unstructured":"Weinstein, J. (2017). Hate speech bans, democracy, and political legitimacy. Constitutional Commentary,32, 527\u2013583.","journal-title":"Constitutional Commentary"},{"key":"9516_CR60","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199548781.003.0001","author":"J Weinstein","year":"2009","unstructured":"Weinstein, J., & Hare, I. (2009). General introduction: Free speech, democracy, and the suppression of extreme speech. Past and Present. https:\/\/doi.org\/10.1093\/acprof:oso\/9780199548781.003.0001 .","journal-title":"Past and Present"},{"key":"9516_CR61","unstructured":"Zhong, H., Li, H., Squicciarini, A. C., Rajtmajer, S. M., Griffin, C., Miller, D. J., & Caragea, C. (2016). Content-driven detection of cyberbullying on the Instagram social network. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, (pp. 3952\u20133958)."}],"container-title":["Ethics and Information Technology"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-019-09516-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10676-019-09516-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-019-09516-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,21]],"date-time":"2023-09-21T17:22:59Z","timestamp":1695316979000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10676-019-09516-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,14]]},"references-count":60,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,3]]}},"alternative-id":["9516"],"URL":"https:\/\/doi.org\/10.1007\/s10676-019-09516-z","relation":{},"ISSN":["1388-1957","1572-8439"],"issn-type":[{"value":"1388-1957","type":"print"},{"value":"1572-8439","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,14]]},"assertion":[{"value":"14 October 2019","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}