{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T17:58:57Z","timestamp":1783533537855,"version":"3.55.0"},"reference-count":74,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2019,7,26]],"date-time":"2019-07-26T00:00:00Z","timestamp":1564099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000269","name":"Economic and Social Research Council Research","doi-asserted-by":"crossref","award":["ES\/P010695\/1"],"award-info":[{"award-number":["ES\/P010695\/1"]}],"id":[{"id":"10.13039\/501100000269","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"<jats:p>Offensive or antagonistic language targeted at individuals and social groups based on their personal characteristics (also known as cyber hate speech or cyberhate) has been frequently posted and widely circulated via the World Wide Web. This can be considered as a key risk factor for individual and societal tension surrounding regional instability. Automated Web-based cyberhate detection is important for observing and understanding community and regional societal tension\u2014especially in online social networks where posts can be rapidly and widely viewed and disseminated. While previous work has involved using lexicons, bags-of-words, or probabilistic language parsing approaches, they often suffer from a similar issue, which is that cyberhate can be subtle and indirect\u2014thus, depending on the occurrence of individual words or phrases, can lead to a significant number of false negatives, providing inaccurate representation of the trends in cyberhate. This problem motivated us to challenge thinking around the representation of subtle language use, such as references to perceived threats from \u201cthe other\u201d including immigration or job prosperity in a hateful context. We propose a novel \u201cothering\u201d feature set that utilizes language use around the concept of \u201cothering\u201d and intergroup threat theory to identify these subtleties, and we implement a wide range of classification methods using embedding learning to compute semantic distances between parts of speech considered to be part of an \u201cothering\u201d narrative. To validate our approach, we conducted two sets of experiments. The first involved comparing the results of our novel method with state-of-the-art baseline models from the literature. Our approach outperformed all existing methods. The second tested the best performing models from the first phase on unseen datasets for different types of cyberhate, namely religion, disability, race, and sexual orientation. The results showed F-measure scores for classifying hateful instances obtained through applying our model of 0.81, 0.71, 0.89, and 0.72, respectively, demonstrating the ability of the \u201cothering\u201d narrative to be an important part of model generalization.<\/jats:p>","DOI":"10.1145\/3324997","type":"journal-article","created":{"date-parts":[[2019,7,26]],"date-time":"2019-07-26T13:17:18Z","timestamp":1564147038000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":49,"title":["\u201cThe Enemy Among Us\u201d"],"prefix":"10.1145","volume":"13","author":[{"given":"Wafa","family":"Alorainy","sequence":"first","affiliation":[{"name":"Cardiff University, UK; Shaqra University, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pete","family":"Burnap","sequence":"additional","affiliation":[{"name":"Cardiff University, Wales, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Han","family":"Liu","sequence":"additional","affiliation":[{"name":"Cardiff University, Wales, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew L.","family":"Williams","sequence":"additional","affiliation":[{"name":"Cardiff University, Wales, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,26]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","volume-title":"Data Mining","author":"Aggarwal Charu C.","DOI":"10.1007\/978-3-319-14142-8"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2487986"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0171649"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3041021.3054223"},{"key":"e_1_2_1_5_1","unstructured":"Mario Guajardo-C\u00e9spedes 8 Margaret Mitchell Ben Packer Yoni Halpern. 2018. Text Embedding Models Contain Bias. Here\u2019s Why That Matters. Retrieved from https:\/\/developers.googleblog.com\/2018\/04\/text-embedding-models-contain-bias.html. Mario Guajardo-C\u00e9spedes 8 Margaret Mitchell Ben Packer Yoni Halpern. 2018. Text Embedding Models Contain Bias. Here\u2019s Why That Matters. Retrieved from https:\/\/developers.googleblog.com\/2018\/04\/text-embedding-models-contain-bias.html."},{"key":"e_1_2_1_6_1","volume-title":"International Conference of the German Society for Computational Linguistics and Language Technology. Springer, 171--179","author":"Benikova Darina","year":"2017"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871741"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_1_9_1","volume-title":"Internet, Policy 8 Politics","author":"Burnap Peter"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Pete Burnap and Matthew L. Williams. 2015. Cyber hate speech on Twitter: An application of machine classification and statistical modeling for policy and decision making. Policy 8 Internet 7 2 (2015) 223--242. Pete Burnap and Matthew L. Williams. 2015. Cyber hate speech on Twitter: An application of machine classification and statistical modeling for policy and decision making. Policy 8 Internet 7 2 (2015) 223--242.","DOI":"10.1002\/poi3.85"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1140\/epjds\/s13688-016-0072-6"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/SocialCom-PASSAT.2012.55"},{"key":"e_1_2_1_13_1","unstructured":"Junyoung Chung Caglar Gulcehre KyungHyun Cho and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014). Junyoung Chung Caglar Gulcehre KyungHyun Cho and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-3001"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1080\/03637751.2012.739704"},{"key":"e_1_2_1_16_1","unstructured":"Thomas Davidson Dana Warmsley Michael Macy and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. arXiv preprint arXiv:1703.04009 (2017). Thomas Davidson Dana Warmsley Michael Macy and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. arXiv preprint arXiv:1703.04009 (2017)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Marie-Catherine De Marneffe and Christopher D. Manning. 2008. Stanford Typed Dependencies Manual. Technical report Stanford University. Marie-Catherine De Marneffe and Christopher D. Manning. 2008. Stanford Typed Dependencies Manual. Technical report Stanford University.","DOI":"10.3115\/1608858.1608859"},{"key":"e_1_2_1_18_1","unstructured":"Fabio Del Vigna12 Andrea Cimino23 Felice Dell\u2019Orletta Marinella Petrocchi and Maurizio Tesconi. 2017. Hate me hate me not: Hate speech detection on Facebook. (2017). Fabio Del Vigna12 Andrea Cimino23 Felice Dell\u2019Orletta Marinella Petrocchi and Maurizio Tesconi. 2017. Hate me hate me not: Hate speech detection on Facebook. (2017)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2742760"},{"key":"e_1_2_1_20_1","unstructured":"Iginio Gagliardone Danit Gal Thiago Alves and Gabriela Martinez. 2015. Countering Online Hate Speech. UNESCO Publishing. Iginio Gagliardone Danit Gal Thiago Alves and Gabriela Martinez. 2015. Countering Online Hate Speech. UNESCO Publishing."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-3013"},{"key":"e_1_2_1_22_1","unstructured":"Lei Gao Alexis Kuppersmith and Ruihong Huang. 2017. Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. arXiv preprint arXiv:1710.07394 Lei Gao Alexis Kuppersmith and Ruihong Huang. 2017. Recognizing explicit and implicit hate speech using a weakly supervised two-path bootstrapping approach. arXiv preprint arXiv:1710.07394"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/EUSIPCO.2015.7362668"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2013.05.057"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14257\/ijmue.2015.10.4.21"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009074"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2006.18.7.1527"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1014052.1014073"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2005.50"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1162"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-2902"},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Christopher S. Josey. 2010. Hate speech and identity: An analysis of neo racism and the indexing of identity. Discourse 8 Society 21 1 (2010) 27--39. Christopher S. Josey. 2010. Hate speech and identity: An analysis of neo racism and the indexing of identity. Discourse 8 Society 21 1 (2010) 27--39.","DOI":"10.1177\/0957926509345071"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2014.04.020"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Yoon Kim Yi-I Chiu Kentaro Hanaki Darshan Hegde and Slav Petrov. 2014. Temporal analysis of language through neural language models. arXiv preprint arXiv:1405.3515. Yoon Kim Yi-I Chiu Kentaro Hanaki Darshan Hegde and Slav Petrov. 2014. Temporal analysis of language through neural language models. arXiv preprint arXiv:1405.3515.","DOI":"10.3115\/v1\/W14-2517"},{"key":"e_1_2_1_35_1","unstructured":"Sebastian K\u00f6ffer Dennis M. Riehle Steffen H\u00f6henberger and J\u00f6rg Becker. 2018. Discussing the value of automatic hate speech detection in online debates. Multikonferenz Wirtschaftsinformatik (MKWI 2018): Data Driven X-Turning Data in Value Leuphana Germany (2018). Sebastian K\u00f6ffer Dennis M. Riehle Steffen H\u00f6henberger and J\u00f6rg Becker. 2018. Discussing the value of automatic hate speech detection in online debates. Multikonferenz Wirtschaftsinformatik (MKWI 2018): Data Driven X-Turning Data in Value Leuphana Germany (2018)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-1175"},{"key":"e_1_2_1_37_1","first-page":"1188","article-title":"Distributed representations of sentences and documents","volume":"14","author":"Le Quoc V.","year":"2014","journal-title":"ICML"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15326926CLP0602_2"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-2050"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2008.2007853"},{"key":"e_1_2_1_41_1","doi-asserted-by":"crossref","unstructured":"Shervin Malmasi and Marcos Zampieri. 2017. Detecting hate speech in social media. arXiv preprint arXiv:1712.06427 Shervin Malmasi and Marcos Zampieri. 2017. Detecting hate speech in social media. arXiv preprint arXiv:1712.06427","DOI":"10.26615\/978-954-452-049-6_062"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/08900520903320936"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-3638"},{"key":"e_1_2_1_44_1","unstructured":"Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013). Tomas Mikolov Kai Chen Greg Corrado and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)."},{"key":"e_1_2_1_45_1","unstructured":"Tomas Mikolov Ilya Sutskever Kai Chen Greg S. Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119. Tomas Mikolov Ilya Sutskever Kai Chen Greg S. Corrado and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111--3119."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11109-016-9373-5"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883062"},{"key":"e_1_2_1_48_1","first-page":"1320","article-title":"Twitter as a corpus for sentiment analysis and opinion mining","volume":"10","author":"Pak Alexander","year":"2010","journal-title":"LREc"},{"key":"e_1_2_1_49_1","unstructured":"Aasish Pappu and Amanda Stent. 2015. Location-Based Recommendations Using Nearest Neighbors in a Locality Sensitive Hashing (LSH) Index. US Patent App. 14 948 213 (2015). Aasish Pappu and Amanda Stent. 2015. Location-Based Recommendations Using Nearest Neighbors in a Locality Sensitive Hashing (LSH) Index. US Patent App. 14 948 213 (2015)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/13600830902814984"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-018-1242-y"},{"key":"e_1_2_1_52_1","unstructured":"Haji Mohammad Saleem Kelly P. Dillon Susan Benesch and Derek sRuths. 2017. A web of hate: Tackling hateful speech in online social spaces. arXiv preprint arXiv:1709.10159 (2017). Haji Mohammad Saleem Kelly P. Dillon Susan Benesch and Derek sRuths. 2017. A web of hate: Tackling hateful speech in online social spaces. arXiv preprint arXiv:1709.10159 (2017)."},{"key":"e_1_2_1_53_1","volume-title":"Proceedings of the 20th Nordic Conference of Computational Linguistics, NODALIDA 2015, May 11--13","author":"Sien\u010dnik Scharolta Katharina","year":"2015"},{"key":"e_1_2_1_54_1","unstructured":"Leandro Ara\u00fajo Silva Mainack Mondal Denzil Correa Fabr\u00edcio Benevenuto and Ingmar Weber. 2016. Analyzing the targets of hate in online social media. In ICWSM. 687--690. Leandro Ara\u00fajo Silva Mainack Mondal Denzil Correa Fabr\u00edcio Benevenuto and Ingmar Weber. 2016. Analyzing the targets of hate in online social media. In ICWSM. 687--690."},{"key":"e_1_2_1_55_1","doi-asserted-by":"crossref","unstructured":"Walter G. Stephan and Cookie White Stephan. 2017. Intergroup threat theory. The International Encyclopedia of Intercultural Communication (2017) 1--12. Walter G. Stephan and Cookie White Stephan. 2017. Intergroup threat theory. The International Encyclopedia of Intercultural Communication (2017) 1--12.","DOI":"10.1002\/9781118783665.ieicc0162"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0147-1767(99)00012-7"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-012-1251-y"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.21416"},{"key":"e_1_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Teun A. Van Dijk. 1993. Elite Discourse and Racism. Vol. 6. Sage. Teun A. Van Dijk. 1993. Elite Discourse and Racism. Vol. 6. Sage.","DOI":"10.4135\/9781483326184"},{"key":"e_1_2_1_60_1","doi-asserted-by":"crossref","unstructured":"Zeerak Waseem Thomas Davidson Dana Warmsley and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. arXiv preprint arXiv:1705.09899 (2017). Zeerak Waseem Thomas Davidson Dana Warmsley and Ingmar Weber. 2017. Understanding abuse: A typology of abusive language detection subtasks. arXiv preprint arXiv:1705.09899 (2017).","DOI":"10.18653\/v1\/W17-3012"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-2013"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2806394"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1099554.1099714"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1093\/bjc\/azv059"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1093\/bjc\/azu043"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.3115\/1220575.1220619"},{"key":"e_1_2_1_67_1","unstructured":"Ruth Wodak. 2009. Discursive Construction of National Identity. Edinburgh University Press. Ruth Wodak. 2009. Discursive Construction of National Identity. Edinburgh University Press."},{"key":"e_1_2_1_68_1","unstructured":"Ruth Wodak and Norman Fairclough. 1997. Critical discourse analysis. Discourse as Social Interaction T. A. van Dijk (Ed.). Sage 258--284. Ruth Wodak and Norman Fairclough. 1997. Critical discourse analysis. Discourse as Social Interaction T. A. van Dijk (Ed.). Sage 258--284."},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.anthro.28.1.175"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052591"},{"key":"e_1_2_1_71_1","unstructured":"Ziqi Zhang and Lei Luo. 2018. Hate speech detection: A solved problem? The challenging case of long tail on Twitter. arXiv preprint arXiv:1803.03662 (2018). Ziqi Zhang and Lei Luo. 2018. Hate speech detection: A solved problem? The challenging case of long tail on Twitter. arXiv preprint arXiv:1803.03662 (2018)."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93417-4_48"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-93417-4_48"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/IALP.2014.6973490"}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3324997","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3324997","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:47:24Z","timestamp":1750193244000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3324997"}},"subtitle":["Detecting Cyber Hate Speech with Threats-based Othering Language Embeddings"],"short-title":[],"issued":{"date-parts":[[2019,7,26]]},"references-count":74,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3324997"],"URL":"https:\/\/doi.org\/10.1145\/3324997","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"value":"1559-1131","type":"print"},{"value":"1559-114X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,26]]},"assertion":[{"value":"2018-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}