{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:34:23Z","timestamp":1760236463935,"version":"build-2065373602"},"reference-count":27,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2021,11,25]],"date-time":"2021-11-25T00:00:00Z","timestamp":1637798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology","doi-asserted-by":"publisher","award":["MOST 110-2221-E-007 -085 -MY3"],"award-info":[{"award-number":["MOST 110-2221-E-007 -085 -MY3"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The permanent transition to online activity has brought with it a surge in hate speech discourse. This has prompted increased calls for automatic detection methods, most of which currently rely on a dictionary of hate speech words, and supervised classification. This approach often falls short when dealing with newer words and phrases produced by online extremist communities. These code words are used with the aim of evading automatic detection by systems. Code words are frequently used and have benign meanings in regular discourse, for instance, \u201cskypes, googles, bing, yahoos\u201d are all examples of words that have a hidden hate speech meaning. Such overlap presents a challenge to the traditional keyword approach of collecting data that is specific to hate speech. In this work, we first introduced a word embedding model that learns the hidden hate speech meaning of words. With this insight on code words, we developed a classifier that leverages linguistic patterns to reduce the impact of individual words. The proposed method was evaluated across three different datasets to test its generalizability. The empirical results show that the linguistic patterns approach outperforms the baselines and enables further analysis on hate speech expressions.<\/jats:p>","DOI":"10.3390\/s21237859","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T01:45:02Z","timestamp":1638323102000},"page":"7859","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Linguistic Patterns for Code Word Resilient Hate Speech Identification"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8089-2349","authenticated-orcid":false,"given":"Fernando H.","family":"Calder\u00f3n","sequence":"first","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"},{"name":"Social Networks and Human-Centered Computing, Taiwan International Graduate Program, Institute of Information Sciences, Academia Sinica, 128, Academia Road, Sec. 2, Nankang, Taipei 115, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Namrita","family":"Balani","sequence":"additional","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jherez","family":"Taylor","sequence":"additional","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Melvyn","family":"Peignon","sequence":"additional","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yen-Hao","family":"Huang","sequence":"additional","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8087-9670","authenticated-orcid":false,"given":"Yi-Shin","family":"Chen","sequence":"additional","affiliation":[{"name":"Institute of Information Systems and Applications, National Tsing Hua University, East District, Guang Fu Rd. Sec. 2, No. 101, Hsinchu City 300, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,25]]},"reference":[{"key":"ref_1","unstructured":"Bloomberg (2016). Disney Dropped Twitter Pursuit Partly Over Image, Bloomberg."},{"key":"ref_2","unstructured":"Forbes (2017). Europe Fine Companies for Hate Speech, Forbes."},{"key":"ref_3","unstructured":"United Nations General Assembly Resolution 2200A [XX1] (2021, November 20). International Covenant on Civil and Political Rights. Available online: https:\/\/www.ohchr.org\/en\/professionalinterest\/pages\/ccpr.aspx."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1477","DOI":"10.3233\/IDA-140267","article-title":"Multilingual emotion classifier using unsupervised pattern extraction from microblog data","volume":"20","author":"Argueta","year":"2016","journal-title":"Intell. Data Anal."},{"key":"ref_5","first-page":"238","article-title":"An Effective Approach for Cyberbullying Detection","volume":"3","author":"Pang","year":"2013","journal-title":"Commun. Inf. Sci. Manag. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Nahar, V., Unankard, S., Li, X., and Pang, C. (2012, January 11\u201313). Sentiment analysis for effective detection of cyber bullying. Proceedings of the Asia-Pacific Web Conference, Kunming, China.","DOI":"10.1007\/978-3-642-29253-8_75"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1002\/poi3.85","article-title":"Cyber Hate Speech on Twitter: An Application of Machine Classification and Statistical Modeling for Policy and Decision Making","volume":"7","author":"Burnap","year":"2015","journal-title":"Policy Internet"},{"key":"ref_8","unstructured":"Silva, L., Mondal, M., Correa, D., Benevenuto, F., and Weber, I. (2016). Analyzing the Targets of Hate in Online Social Media. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nobata, C., Tetreault, J., Thomas, A., Mehdad, Y., and Chang, Y. (2016, January 11\u201315). Abusive Language Detection in Online User Content. Proceedings of the 25th International Conference on World Wide Web, Montreal, QC, Canada.","DOI":"10.1145\/2872427.2883062"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Waseem, Z., and Hovy, D. (2016, January 12\u201317). Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. Proceedings of the NAACL Student Research Workshop, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-2013"},{"key":"ref_11","first-page":"1","article-title":"Detection of harassment on web 2.0","volume":"2","author":"Yin","year":"2009","journal-title":"Proc. Content Anal. Web"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Schmidt, A., and Wiegand, M. (2017, January 11). A Survey on Hate Speech Detection using Natural Language Processing. Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media, Boston, MA, USA.","DOI":"10.18653\/v1\/W17-1101"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"O\u2019Callaghan, D., Greene, D., Conway, M., Carthy, J., and Cunningham, P. (2013). An analysis of interactions within and between extreme right communities in social media. Ubiquitous Social Media Analysis, Springer.","DOI":"10.1007\/978-3-642-45392-2_5"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1140\/epjds\/s13688-016-0072-6","article-title":"Us and them: Identifying cyber hate on Twitter across multiple protected characteristics","volume":"5","author":"Burnap","year":"2016","journal-title":"EPJ Data Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Waseem, Z. (2016, January 5). Are You a Racist or Am I Seeing Things? Annotator Influence on Hate Speech Detection on Twitter. Proceedings of the 2016 EMNLP Workshop on Natural Language Processing and Computational Social Science, Austin, TX, USA.","DOI":"10.18653\/v1\/W16-5618"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"ElSherief, M., Kulkarni, V., Nguyen, D., Wang, W.Y., and Belding, E. (2018). Hate Lingo: A Target-based Linguistic Analysis of Hate Speech in Social Media. arXiv.","DOI":"10.1609\/icwsm.v12i1.15041"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., and Bhamidipati, N. (2015, January 18\u201322). Hate Speech Detection with Comment Embeddings. Proceedings of the 24th International Conference on World Wide Web, Florence, Italy.","DOI":"10.1145\/2740908.2742760"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mehdad, Y., and Tetreault, J. (2016, January 13\u201315). Do Characters Abuse More Than Words?. Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Los Angeles, CA, USA.","DOI":"10.18653\/v1\/W16-3638"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_20","unstructured":"Omer, L., and Yoav, G. (2014, January 22\u201327). Dependency-Based Word Embeddings. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Short Papers), Baltimore, MD, USA."},{"key":"ref_21","unstructured":"Magu, R., Joshi, K., and Luo, J. (2017, January 15\u201318). Detecting the Hate Code on Social Media. Proceedings of the Eleventh International AAAI Conference on Web and Social Media (ICWSM 2017), Montreal, QC, Canada."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2303","DOI":"10.1142\/S0218127407018403","article-title":"Centrality estimation in large networks","volume":"17","author":"Brandes","year":"2007","journal-title":"Int. J. Bifurc. Chaos"},{"key":"ref_23","unstructured":"Page, L., Brin, S., Motwani, R., and Winograd, T. (1999). The PageRank Citation Ranking: Bringing Order to the Web, Stanford InfoLab. Technical Report 1999-66."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Downing, A., and Locke, P. (2006). English Grammar: A University Course, Routledge.","DOI":"10.4324\/9780203087640"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Halliday, M.A.K., Matthiessen, C.M., Halliday, M., and Matthiessen, C. (2014). An Introduction to Functional Grammar, Routledge.","DOI":"10.4324\/9780203783771"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Davidson, T., Warmsley, D., Macy, M., and Weber, I. (2017, January 15\u201318). Automated hate speech detection and the problem of offensive language. Proceedings of the International AAAI Conference on Web and Social Media, Montr\u00e9al, QC, Canada.","DOI":"10.1609\/icwsm.v11i1.14955"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Saravia, E., Liu, H.C.T., Huang, Y.H., Wu, J., and Chen, Y.S. (November, January 31). Carer: Contextualized affect representations for emotion recognition. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1404"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7859\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:35:56Z","timestamp":1760168156000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/23\/7859"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,25]]},"references-count":27,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2021,12]]}},"alternative-id":["s21237859"],"URL":"https:\/\/doi.org\/10.3390\/s21237859","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,11,25]]}}}