{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T11:39:14Z","timestamp":1784547554096,"version":"3.55.0"},"reference-count":103,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,2,13]],"date-time":"2021-02-13T00:00:00Z","timestamp":1613174400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,13]],"date-time":"2021-02-13T00:00:00Z","timestamp":1613174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001858","name":"VINNOVA","doi-asserted-by":"publisher","award":["2019-02996"],"award-info":[{"award-number":["2019-02996"]}],"id":[{"id":"10.13039\/501100001858","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Lulea University of Technology"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"published-print":{"date-parts":[[2021,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The detection of hate speech in social media is a crucial task. The uncontrolled spread of hate has the potential to gravely damage our society, and severely harm marginalized people or groups. A major arena for spreading hate speech online is social media. This significantly contributes to the difficulty of automatic detection, as social media posts include paralinguistic signals (e.g. emoticons, and hashtags), and their linguistic content contains plenty of poorly written text. Another difficulty is presented by the context-dependent nature of the task, and the lack of consensus on what constitutes as hate speech, which makes the task difficult even for humans. This makes the task of creating large labeled corpora difficult, and resource consuming. The problem posed by ungrammatical text has been largely mitigated by the recent emergence of deep neural network (DNN) architectures that have the capacity to efficiently learn various features. For this reason, we proposed a deep natural language processing (NLP) model\u2014combining convolutional and recurrent layers\u2014for the automatic detection of hate speech in social media data. We have applied our model on the HASOC2019 corpus, and attained a macro F1 score of 0.63 in hate speech detection on the test set of HASOC. The capacity of DNNs for efficient learning, however, also means an increased risk of overfitting. Particularly, with limited training data available (as was the case for HASOC). For this reason, we investigated different methods for expanding resources used. We have explored various opportunities, such as leveraging unlabeled data, similarly labeled corpora, as well as the use of novel models. Our results showed that by doing so, it was possible to significantly increase the classification score attained.<\/jats:p>","DOI":"10.1007\/s42979-021-00457-3","type":"journal-article","created":{"date-parts":[[2021,2,14]],"date-time":"2021-02-14T08:37:40Z","timestamp":1613291860000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":102,"title":["Challenges of Hate Speech Detection in Social Media"],"prefix":"10.1007","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0546-116X","authenticated-orcid":false,"given":"Gy\u00f6rgy","family":"Kov\u00e1cs","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro","family":"Alonso","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rajkumar","family":"Saini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,13]]},"reference":[{"key":"457_CR1","doi-asserted-by":"publisher","first-page":"203","DOI":"10.20901\/pm.55.4.08","volume":"55","author":"N Alkiviadou","year":"2018","unstructured":"Alkiviadou N. The legal regulation of hate speech: the international and European frameworks. Politi\u010dka misao. 2018;55:203\u201329. https:\/\/doi.org\/10.20901\/pm.55.4.08.","journal-title":"Politi\u010dka misao"},{"issue":"1","key":"457_CR2","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1080\/13600834.2018.1494417","volume":"28","author":"N Alkiviadou","year":"2019","unstructured":"Alkiviadou N. Hate speech on social media networks: towards a regulatory framework? Inf Commun Technol Law. 2019;28(1):19\u201335. https:\/\/doi.org\/10.1080\/13600834.2018.1494417.","journal-title":"Inf Commun Technol Law"},{"key":"457_CR3","unstructured":"Alonso P, Saini R, Kov\u00e1cs G. Hate speech detection using transformer ensembles on the hasoc dataset. In: Speech and Computer: 22nd International Conference, SPECOM 2020, St. Petersburg, Russia, October 7\u20139, 2020, Proceedings, vol. 12335, p.\u00a013. Springer Nature; 2020."},{"key":"457_CR4","doi-asserted-by":"publisher","unstructured":"Arango A, P\u00e9rez J, Poblete B. Hate speech detection is not as easy as you may think: A closer look at model validation. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR\u201919, p. 45\u201354. Association for Computing Machinery, New York, NY, USA; 2019. https:\/\/doi.org\/10.1145\/3331184.3331262.","DOI":"10.1145\/3331184.3331262"},{"key":"457_CR5","unstructured":"Assembly UNG. Annual report of the united nations high commissioner for human rights, report of the united nations high commissioner for human rights on the expert workshops on the prohibition of incitement to national, racial or religious hatred; 2013. https:\/\/www.ohchr.org\/Documents\/Issues\/Opinion\/SeminarRabat\/Rabat_draft_outcome.pdf."},{"key":"457_CR6","doi-asserted-by":"crossref","unstructured":"Badjatiya P, Gupta S, Gupta M, Varma V. Deep learning for hate speech detection in tweets. In: Proceedings of the 26th International Conference on World Wide Web Companion, WWW \u201917 Companion, pp. 759\u2013760; 2017.","DOI":"10.1145\/3041021.3054223"},{"key":"457_CR7","unstructured":"Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Y.\u00a0Bengio, Y.\u00a0LeCun (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings 2015. arXiv:1409.0473."},{"key":"457_CR8","unstructured":"Bahlmann C, Haasdonk B, Burkhardt H. Online handwriting recognition with support vector machines-a kernel approach. In: Proceedings Eighth International Workshop on Frontiers in Handwriting Recognition, pp. 49\u201354. IEEE; 2002."},{"key":"457_CR9","doi-asserted-by":"publisher","unstructured":"Barendt, E.: What is the harm of hate speech? Ethic theory. Moral Pract. 22, 2019. https:\/\/doi.org\/10.1007\/s10677-019-10002-0.","DOI":"10.1007\/s10677-019-10002-0"},{"key":"457_CR10","doi-asserted-by":"publisher","unstructured":"Basile V, Bosco C, Fersini E, Nozza D, Patti V, Rangel\u00a0Pardo FM, Rosso P, Sanguinetti M. SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 54\u201363. Association for Computational Linguistics, Minneapolis, Minnesota, USA; 2019. https:\/\/doi.org\/10.18653\/v1\/S19-2007. https:\/\/www.aclweb.org\/anthology\/S19-2007.","DOI":"10.18653\/v1\/S19-2007"},{"key":"457_CR11","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.pragma.2014.02.009","volume":"66","author":"C Bianchi","year":"2014","unstructured":"Bianchi C. Slurs and appropriation: an echoic account. J Pragmat. 2014;66:35\u201344. https:\/\/doi.org\/10.1016\/j.pragma.2014.02.009.","journal-title":"J Pragmat"},{"key":"457_CR12","doi-asserted-by":"crossref","unstructured":"Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. arXiv:1607.04606; 2016.","DOI":"10.1162\/tacl_a_00051"},{"issue":"3","key":"457_CR13","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1177\/1468796817709846","volume":"18","author":"A Brown","year":"2018","unstructured":"Brown A. What is so special about online (as compared to offline) hate speech? Ethnicities. 2018;18(3):297\u2013326. https:\/\/doi.org\/10.1177\/1468796817709846.","journal-title":"Ethnicities"},{"issue":"2","key":"457_CR14","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1002\/poi3.85","volume":"7","author":"P Burnap","year":"2015","unstructured":"Burnap P, Williams ML. Cyber hate speech on twitter: an application of machine classification and statistical modeling for policy and decision making. Policy Internet. 2015;7(2):223\u201342. https:\/\/doi.org\/10.1002\/poi3.85.","journal-title":"Policy Internet"},{"key":"457_CR15","first-page":"853","volume-title":"Data mining for imbalanced datasets: an overview","author":"N Chawla","year":"2005","unstructured":"Chawla N. Data mining for imbalanced datasets: an overview, vol. 5. New York: Springer; 2005. p. 853\u201367."},{"key":"457_CR16","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/S0003-2670(01)95359-0","volume":"136","author":"D Coomans","year":"1982","unstructured":"Coomans D, Massart DL. Alternative k-nearest neighbour rules in supervised pattern recognition: part 1. k-nearest neighbour classification by using alternative voting rules. Anal Chim Acta. 1982;136:15\u201327.","journal-title":"Anal Chim Acta"},{"key":"457_CR17","unstructured":"Davidson T. Hate speech and offensive language. https:\/\/github.com\/t-davidson\/hate-speech-and-offensive-language; 2019. Accessed on 25 Mar 2020."},{"key":"457_CR18","doi-asserted-by":"publisher","unstructured":"Davidson T, Bhattacharya D, Weber I. Racial bias in hate speech and abusive language detection datasets. In: Proceedings of the Third Workshop on Abusive Language Online, pp. 25\u201335. Association for Computational Linguistics, Florence, Italy; 2019. https:\/\/doi.org\/10.18653\/v1\/W19-3504. https:\/\/www.aclweb.org\/anthology\/W19-3504.","DOI":"10.18653\/v1\/W19-3504"},{"key":"457_CR19","doi-asserted-by":"crossref","unstructured":"Davidson T, Warmsley D, Macy M, Weber I. Automated hate speech detection and the problem of offensive language. In: Proceedings of the 11th International AAAI Conference on Web and Social Media, ICWSM \u201917, pp. 512\u2013515; 2017.","DOI":"10.1609\/icwsm.v11i1.14955"},{"key":"457_CR20","unstructured":"Del\u00a0Vigna F, Cimino A, Dell\u2019Orletta F, Petrocchi M, Tesconi M. Hate me, hate me not: Hate speech detection on facebook. In: ITASEC; 2017."},{"key":"457_CR21","unstructured":"Devlin J, Chang M, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR arXiv:abs\/1810.04805; 2018."},{"key":"457_CR22","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171\u20134186. Association for Computational Linguistics, Minneapolis, Minnesota; 2019. https:\/\/doi.org\/10.18653\/v1\/N19-1423. https:\/\/www.aclweb.org\/anthology\/N19-1423.","DOI":"10.18653\/v1\/N19-1423"},{"key":"457_CR23","doi-asserted-by":"publisher","unstructured":"Djuric N, Zhou J, Morris R, Grbovic M, Radosavljevic V, Bhamidipati N. Hate speech detection with comment embeddings. In: Proceedings of the 24th International Conference on World Wide Web, WWW \u201915 Companion, p. 29\u201330. Association for Computing Machinery, New York, NY, USA; 2015. https:\/\/doi.org\/10.1145\/2740908.2742760.","DOI":"10.1145\/2740908.2742760"},{"key":"457_CR24","unstructured":"Do HTT, Huynh HD, Nguyen KV, Nguyen NLT, Nguyen AGT. Hate speech detection on vietnamese social media text using the bidirectional-lstm model; 2019. arXiv:1911.03648."},{"issue":"1","key":"457_CR25","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1080\/03064220500532412","volume":"35","author":"R Dworkin","year":"2006","unstructured":"Dworkin R. A new map of censorship. Index Censorship. 2006;35(1):130\u20133. https:\/\/doi.org\/10.1080\/03064220500532412.","journal-title":"Index Censorship"},{"key":"457_CR26","doi-asserted-by":"publisher","unstructured":"Gamb\u00e4ck B, Sikdar UK. Using convolutional neural networks to classify hate-speech. In: Proceedings of the First Workshop on Abusive Language Online, pp. 85\u201390. Association for Computational Linguistics, Vancouver, BC, Canada; 2017. https:\/\/doi.org\/10.18653\/v1\/W17-3013. https:\/\/www.aclweb.org\/anthology\/W17-3013.","DOI":"10.18653\/v1\/W17-3013"},{"key":"457_CR27","unstructured":"Gasc\u00f3 G, Rocha M, Sanchis-Trilles G, Andr\u00e9s-Ferrer J, Casacuberta F. Does more data always yield better translations? In: Daelemans W, Lapata M, M\u00e0rquez L (eds.) EACL 2012, 13th Conference of the European Chapter of the Association for Computational Linguistics, Avignon, France, April 23-27, 2012, pp. 152\u2013161. The Association for Computer Linguistics; 2012. https:\/\/www.aclweb.org\/anthology\/E12-1016\/."},{"key":"457_CR28","unstructured":"Gelashvili T. Hate Speech on Social Media: Implications of private regulation and governance gaps. Master\u2019s thesis, Lund University, Sweden; 2018."},{"key":"457_CR29","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1080\/13504630.2015.1128810","volume":"22","author":"K Gelber","year":"2016","unstructured":"Gelber K, McNamara L. Evidencing the harms of hate speech. Soc Identities. 2016;22:324\u201341.","journal-title":"Soc Identities"},{"key":"457_CR30","unstructured":"Grave E, Bojanowski P, Gupta P, Joulin A, Mikolov T. Learning word vectors for 157 languages. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018); 2018."},{"key":"457_CR31","doi-asserted-by":"publisher","unstructured":"Greevy E, Smeaton AF. Classifying racist texts using a support vector machine. In: Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR \u201904, p. 468\u2013469. Association for Computing Machinery, New York, NY, USA; 2004. https:\/\/doi.org\/10.1145\/1008992.1009074.","DOI":"10.1145\/1008992.1009074"},{"key":"457_CR32","doi-asserted-by":"publisher","unstructured":"Gr\u00f6ndahl T, Pajola L, Juuti M, Conti M, Asokan N. All you need is \u201clove\u201d: Evading hate speech detection. In: Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security, AISec \u201918, p. 2\u201312. Association for Computing Machinery, New York, NY, USA; 2018. https:\/\/doi.org\/10.1145\/3270101.3270103.","DOI":"10.1145\/3270101.3270103"},{"key":"457_CR33","unstructured":"Hern A. Revealed: catastrophic effects of working as a facebook moderator. The Guardian; 2019. https:\/\/www.theguardian.com\/technology\/2019\/sep\/17\/revealed-catastrophic-effects-working-facebook-moderator. Accessed on 26 Apr 2020."},{"key":"457_CR34","doi-asserted-by":"publisher","unstructured":"Heyman S. Hate speech, public discourse, and the first amendment. In: Hare I, Weinstein J (eds.) Extreme Speech and Democracy. Oxford Scholarship Online; 2009. https:\/\/doi.org\/10.1093\/acprof:oso\/9780199548781.003.0010.","DOI":"10.1093\/acprof:oso\/9780199548781.003.0010"},{"key":"457_CR35","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s11098-011-9749-7","volume":"159","author":"C Hom","year":"2012","unstructured":"Hom C. A puyyle about pejoratives. Philos. Stud. 2012;159:383\u2013405. https:\/\/doi.org\/10.1007\/s11098-011-9749-7.","journal-title":"Philos. Stud."},{"key":"457_CR36","unstructured":"Huynh TV, Nguyen VD, Nguyen KV, Nguyen NLT, Nguyen AGT. Hate speech detection on vietnamese social media text using the bi-gru-lstm-cnn model. arXiv:1911.03644; 2019."},{"key":"457_CR37","unstructured":"Immpermium: Detecting insults in social commentary. https:\/\/kaggle.com\/c\/detecting-insults-in-social-commentary. Accessed on 27 Apr 2020."},{"key":"457_CR38","unstructured":"Isasi AC, Juanatey A. Hate speech in social media: a state-of-the-art review; 2017."},{"key":"457_CR39","doi-asserted-by":"crossref","unstructured":"Jiao X, Yin Y, Shang L, Jiang X, Chen X, Li L, Wang F, Liu Q. Tiny{bert}: Distilling {bert} for natural language understanding; 2020. https:\/\/openreview.net\/forum?id=rJx0Q6EFPB.","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"457_CR40","doi-asserted-by":"crossref","unstructured":"Joulin A, Grave E, Bojanowski P, Mikolov T. Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp. 427\u2013431. Association for Computational Linguistics, Valencia, Spain; 2017. https:\/\/www.aclweb.org\/anthology\/E17-2068.","DOI":"10.18653\/v1\/E17-2068"},{"key":"457_CR41","unstructured":"Kim JY, Ortiz C, Nam S, Santiago S, Datta V. Intersectional bias in hate speech and abusive language datasets; 2020."},{"key":"457_CR42","doi-asserted-by":"publisher","unstructured":"Kirch W. (ed.) Pearson\u2019s Correlation Coefficient, pp. 1090\u20131091. Springer Netherlands, Dordrecht; 2008. https:\/\/doi.org\/10.1007\/978-1-4020-5614-7_2569.","DOI":"10.1007\/978-1-4020-5614-7_2569"},{"key":"457_CR43","unstructured":"Kumar\u00a0R. Ojha\u00a0AK., M.S., M., Z.: Benchmarking aggression identification in social media. In: Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 1\u201311; 2018."},{"key":"457_CR44","doi-asserted-by":"crossref","unstructured":"Kwok I, Wang Y. Locate the hate: Detecting tweets against blacks. In: Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, AAAI\u201913, p. 1621\u20131622. AAAI Press; 2013.","DOI":"10.1609\/aaai.v27i1.8539"},{"key":"457_CR45","unstructured":"Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R. Albert: A lite bert for self-supervised learning of language representations. In: International Conference on Learning Representations; 2020. https:\/\/openreview.net\/forum?id=H1eA7AEtvS."},{"issue":"3","key":"457_CR46","first-page":"18","volume":"2","author":"A Liaw","year":"2002","unstructured":"Liaw A, Wiener M, et al. Classification and regression by randomforest. R NEWS. 2002;2(3):18\u201322.","journal-title":"R NEWS"},{"key":"457_CR47","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT pretraining approach. arXiv:1907.11692; 2019."},{"issue":"8","key":"457_CR48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0221152","volume":"14","author":"S MacAvaney","year":"2019","unstructured":"MacAvaney S, Yao HR, Yang E, Russell K, Goharian N, Frieder O. Hate speech detection: Challenges and solutions. PLoS ONE. 2019;14(8):1\u201316. https:\/\/doi.org\/10.1371\/journal.pone.0221152.","journal-title":"PLoS ONE"},{"key":"457_CR49","unstructured":"Mandl T, Modha S, Mandlia C, Patel D, Patel A, Dave M. HASOC - hate speech and offensive content identification in indo-european languages. https:\/\/hasoc2019.github.io\/call_for_participation.html. Accessed 20 Sept 2019."},{"key":"457_CR50","doi-asserted-by":"crossref","unstructured":"Mandl T, Modha S, Patel D, Dave M, Mandlia C, Patel A. Overview of the HASOC track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages). In: Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation; 2019.","DOI":"10.1145\/3368567.3368584"},{"key":"457_CR51","doi-asserted-by":"crossref","unstructured":"Matsuda MJ. Public response to racist spech: Considering the victim\u2019s story. In: R.D. M.\u00a0J.\u00a0Matsuda C. R. Lawrence\u00a0III, K.\u00a0Williams (eds.) Words that wound: Critical race theory, assaultive speech, and the first amendment, pp. 17\u201352. Routledge, New York; 1993.","DOI":"10.4324\/9780429502941-2"},{"key":"457_CR52","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"M McHugh","year":"2012","unstructured":"McHugh M. Interrater reliability: the kappa statistic. Biochemia medica : \u010dasopis Hrvatskoga dru\u0161tva medicinskih biokemi\u010dara \/ HDMB. 2012;22:276\u201382. https:\/\/doi.org\/10.11613\/BM.2012.031.","journal-title":"Biochemia medica : \u010dasopis Hrvatskoga dru\u0161tva medicinskih biokemi\u010dara \/ HDMB"},{"key":"457_CR53","volume-title":"Discriminant analysis and statistical pattern recognition","author":"GJ McLachlan","year":"2004","unstructured":"McLachlan GJ. Discriminant analysis and statistical pattern recognition, vol. 544. Amsterdam: Wiley; 2004."},{"key":"457_CR54","doi-asserted-by":"publisher","unstructured":"Mehdad Y, Tetreault J. Do characters abuse more than words? In: Proceedings of the SIGDIAL2016 conference, pp. 299\u2013303; 2016. https:\/\/doi.org\/10.18653\/v1\/W16-3638.","DOI":"10.18653\/v1\/W16-3638"},{"key":"457_CR55","unstructured":"Mikolov T, Grave E, Bojanowski P, Puhrsch C, Joulin A. Advances in pre-training distributed word representations. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018); 2018."},{"key":"457_CR56","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado G, Dean J. Distributed representations of words and phrases and their compositionality. In: Proc. NIPS, pp. 3111\u20133119; 2013."},{"key":"457_CR57","doi-asserted-by":"publisher","unstructured":"Mondal M, Silva LA, Benevenuto F. A measurement study of hate speech in social media. In: Proceedings of the 28th ACM Conference on Hypertext and Social Media, HT \u201917, p. 85\u201394. Association for Computing Machinery, New York, NY, USA; 2017. https:\/\/doi.org\/10.1145\/3078714.3078723.","DOI":"10.1145\/3078714.3078723"},{"key":"457_CR58","doi-asserted-by":"publisher","unstructured":"M\u00fcller K, Schwarz C. Fanning the flames of hate: Social media and hate crime. SSRN Electronic Journal; 2017. https:\/\/doi.org\/10.2139\/ssrn.3082972.","DOI":"10.2139\/ssrn.3082972"},{"key":"457_CR59","unstructured":"Nina-Alcocer V. Vito at HASOC 2019: Detecting hate speech and offensive content through ensembles. In: Mehta P, Rosso P, Majumder P, Mitra M (eds.) Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, CEUR Workshop Proceedings, vol. 2517, pp. 214\u2013220. CEUR-WS.org; 2019. http:\/\/ceur-ws.org\/Vol-2517\/T3-5.pdf."},{"key":"457_CR60","doi-asserted-by":"publisher","unstructured":"Nobata C, Tetreault J, Thomas A, Mehdad Y, Chang Y. Abusive language detection in online user content. In: Proceedings of the 25th International Conference on World Wide Web, WWW \u201916, p. 145\u2013153. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE; 2016. https:\/\/doi.org\/10.1145\/2872427.2883062.","DOI":"10.1145\/2872427.2883062"},{"key":"457_CR61","doi-asserted-by":"publisher","unstructured":"Nourbakhsh, A., Vermeer, F., Wiltvank, G., van\u00a0der Goot, R.: sthruggle at SemEval-2019 task 5: An ensemble approach to hate speech detection. In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 484\u2013488. Association for Computational Linguistics, Minneapolis, Minnesota, USA (2019). https:\/\/doi.org\/10.18653\/v1\/S19-2086. https:\/\/www.aclweb.org\/anthology\/S19-2086.","DOI":"10.18653\/v1\/S19-2086"},{"issue":"1","key":"457_CR62","first-page":"129","volume":"24","author":"A Ohieku","year":"2020","unstructured":"Ohieku A, Sabo S. Journalism practice in an era of unguided utterances: framing of hate speech in selected Nigerian newspapers. Univ Nigeria Interdiscip J Commun Stud. 2020;24(1):129\u201340.","journal-title":"Univ Nigeria Interdiscip J Commun Stud"},{"issue":"1","key":"457_CR63","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1093\/clp\/cuy012","volume":"71","author":"C O\u2019Regan","year":"2018","unstructured":"O\u2019Regan C. Hate Speech Online: an (Intractable) Contemporary Challenge? Current Legal Problems. 2018;71(1):403\u201329. https:\/\/doi.org\/10.1093\/clp\/cuy012.","journal-title":"Current Legal Problems"},{"key":"457_CR64","unstructured":"Otter DW, Medina JR, Kalita JK. A survey of the usages of deep learning for natural language processing. IEEE Transactions on Neural Networks and Learning Systems pp. 1\u201321; 2020."},{"key":"457_CR65","doi-asserted-by":"crossref","unstructured":"Park J, Fung P. One-step and two-step classification for abusive language detection on twitter. In: ALW1: 1st Workshop on Abusive Language Online; 2017.","DOI":"10.18653\/v1\/W17-3006"},{"key":"457_CR66","unstructured":"Pedro\u00a0Alonso Rajkumar\u00a0Saini GK. TheNorth at HASOC 2019 Hate Speech Detection in Social Media Data. In: Proceedings of the 11th annual meeting of the Forum for Information Retrieval Evaluation; 2019."},{"key":"457_CR67","doi-asserted-by":"crossref","unstructured":"Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. In: Proc. EMNLP, pp. 1532\u20131543; 2014. http:\/\/www.aclweb.org\/anthology\/D14-1162.","DOI":"10.3115\/v1\/D14-1162"},{"issue":"21","key":"457_CR68","doi-asserted-by":"publisher","first-page":"4654","DOI":"10.3390\/s19214654","volume":"19","author":"JC Pereira-Kohatsu","year":"2019","unstructured":"Pereira-Kohatsu JC, S\u00e1nchez LQ, Liberatore F, Camacho-Collados M. Detecting and monitoring hate speech in twitter. Sensors. 2019;19(21):4654. https:\/\/doi.org\/10.3390\/s19214654.","journal-title":"Sensors"},{"key":"457_CR69","doi-asserted-by":"publisher","first-page":"2879","DOI":"10.1007\/s11098-017-0986-2","volume":"175","author":"M Popa-Wyatt","year":"2018","unstructured":"Popa-Wyatt M, Wyatt J. Slurs, roles and power. Philos Stud. 2018;175:2879\u2013906. https:\/\/doi.org\/10.1007\/s11098-017-0986-2.","journal-title":"Philos Stud"},{"key":"457_CR70","unstructured":"Raehtka A. Recognizing the evolution of racial slurs. Democrat & Chronicle; 2014. https:\/\/eu.democratandchronicle.com\/story\/editorial\/2014\/01\/29\/recognizing-the-evolution-of-racial-slurs\/5017955\/. Accessed on 2020-04-28."},{"key":"457_CR71","doi-asserted-by":"publisher","unstructured":"Ross B, Rist M, Carbonell G, Cabrera B, Kurowsky N, Wojatzki M. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. In: Bei\u00dfwenger M, Wojatzki M, Zesch T (eds.) Proceedings of NLP4CMC III, pp. 6\u20139; 2016. https:\/\/doi.org\/10.17185\/duepublico\/42132.","DOI":"10.17185\/duepublico\/42132"},{"key":"457_CR72","doi-asserted-by":"publisher","unstructured":"Saha K, Chandrasekharan E, De\u00a0Choudhury M. Prevalence and psychological effects of hateful speech in online college communities. In: Proceedings of the 10th ACM Conference on Web Science, WebSci \u201919, p. 255\u2013264. Association for Computing Machinery, New York, NY, USA; 2019. https:\/\/doi.org\/10.1145\/3292522.3326032.","DOI":"10.1145\/3292522.3326032"},{"key":"457_CR73","doi-asserted-by":"crossref","unstructured":"Salminen J, Almerekhi H, Milenkovic M, JUNG SG, KWAK H, JANSEN BJ. Anatomy of online hate: developing a taxonomy and machine learning models for identifying and classifying hate in online news media; 2018.","DOI":"10.1609\/icwsm.v12i1.15028"},{"key":"457_CR74","unstructured":"Sanh V, Debut L, Chaumond J, Wolf T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter; 2019."},{"key":"457_CR75","unstructured":"Sap M, Card D, Gabriel S, Choi Y, Smith NA. The risk of racial bias in hate speech detection. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1668\u20131678. Association for Computational Linguistics, Florence, Italy; 2019. https:\/\/www.aclweb.org\/anthology\/P19-1163."},{"issue":"3","key":"457_CR76","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1023\/A:1007614523901","volume":"37","author":"RE Schapire","year":"1999","unstructured":"Schapire RE, Singer Y. Improved boosting algorithms using confidence-rated predictions. Mach Learn. 1999;37(3):297\u2013336.","journal-title":"Mach Learn"},{"key":"457_CR77","doi-asserted-by":"crossref","unstructured":"Seganti A, Sobol H, Orlova I, Kim H, Staniszewski J, Krumholc T, Koziel K. Nlpr@srpol at semeval-2019 task 6 and task 5: Linguistically enhanced deep learning offensive sentence classifier. In: SemEval@NAACL-HLT; 2019.","DOI":"10.18653\/v1\/S19-2126"},{"key":"457_CR78","doi-asserted-by":"crossref","unstructured":"Sun C, Qiu X, Xu Y, Huang X. How to fine-tune bert for text classification? arXiv:1905.05583; 2020.","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"457_CR79","unstructured":"Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks. CoRR arXiv:1703.01365; 2017."},{"key":"457_CR80","doi-asserted-by":"crossref","unstructured":"Topidi K. Words that Hurt (2): National and International Perspectives on Hate Speech Regulation; 2019.","DOI":"10.2139\/ssrn.3488718"},{"key":"457_CR81","doi-asserted-by":"publisher","unstructured":"Ullmann S, Tomalin M. Quarantining online hate speech: technical and ethical perspectives. Ethics and Information Technology; 2019. https:\/\/doi.org\/10.1007\/s10676-019-09516-z.","DOI":"10.1007\/s10676-019-09516-z"},{"key":"457_CR82","unstructured":"Van den Rul C. Why have resolutions of the un general assembly if they are not legally binding? 2020."},{"key":"457_CR83","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I. Attention is all you need. Advances in Neural Information Processing Systems 2017-Decem(Nips), 5999\u20136009; 2017."},{"key":"457_CR84","unstructured":"Wang B, Ding Y, Liu S, Zhou X. Ynu\\_wb at HASOC 2019: Ordered neurons LSTM with attention for identifying hate speech and offensive language. In: Mehta, P, Rosso, P, Majumder P, Mitra M (eds.) Working Notes of FIRE 2019 - Forum for Information Retrieval Evaluation, Kolkata, India, December 12-15, 2019, CEUR Workshop Proceedings, vol. 2517, pp. 191\u2013198. CEUR-WS.org; 2019. http:\/\/ceur-ws.org\/Vol-2517\/T3-2.pdf."},{"key":"457_CR85","unstructured":"Warner W, Hirschberg J. Detecting hate speech on the world wide web. In: Proceedings of the Second Workshop on Language in Social Media, pp. 19\u201326. Association for Computational Linguistics, Montr\u00e9al, Canada; 2012. https:\/\/www.aclweb.org\/anthology\/W12-2103."},{"key":"457_CR86","doi-asserted-by":"crossref","unstructured":"Waseem Z. Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter. In: Proceedings of the First Workshop on NLP and Computational Social Science, pp. 138\u2013142. Association for Computational Linguistics, Austin, Texas; 2016. https:\/\/www.aclweb.org\/anthology\/W16-5618.","DOI":"10.18653\/v1\/W16-5618"},{"key":"457_CR87","doi-asserted-by":"publisher","unstructured":"Waseem Z, Hovy D. Hateful symbols or hateful people? predictive features for hate speech detection on twitter. In: Proceedings of the NAACL Student Research Workshop, pp. 88\u201393. Association for Computational Linguistics, San Diego, California; 2016. https:\/\/doi.org\/10.18653\/v1\/N16-2013. https:\/\/www.aclweb.org\/anthology\/N16-2013.","DOI":"10.18653\/v1\/N16-2013"},{"key":"457_CR88","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3390\/info8030092","volume":"8","author":"X Wei","year":"2017","unstructured":"Wei X, Lin H, Yang L, Yu Y. A convolution-lstm-based deep neural network for cross-domain mooc forum post classification. Information. 2017;8:92. https:\/\/doi.org\/10.3390\/info8030092.","journal-title":"Information"},{"key":"457_CR89","unstructured":"Wiegand M, Siegel M, Ruppenhofer J. Overview of the germeval 2018 shared task on the identification of offensive language. In: Proceedings of the GermEval 2018 Workshop, pp. 1\u201311; 2018."},{"key":"457_CR90","doi-asserted-by":"crossref","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Brew J. Huggingface\u2019s transformers: State-of-the-art natural language processing. arXiv:1910.03771; 2019.","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"457_CR91","unstructured":"Wright RE. Logistic regression. In: Grimm LG, Yarnold PR (eds.) Reading and understanding multivariate statistics. American Psychological Association; 1995."},{"key":"457_CR92","unstructured":"Young T, Hazarika D, Poria S, Cambria E. Recent trends in deep learning based natural language processing; 2017. arXiv:1708.02709. Cite arXiv:1708.02709 Comment: Added BERT, ELMo, Transformer."},{"key":"457_CR93","doi-asserted-by":"publisher","unstructured":"Yuan S, Wu X, Xiang Y. A two phase deep learning model for identifying discrimination from tweets. In: Pitoura E, Maabout S, Koutrika G, Marian A, Tanca L, Manolescu I, Stefanidis K (eds.) Proc. EDBT 2016, Bordeaux, France, March 15-16, 2016, Bordeaux, France, March 15\u201316, 2016, pp. 696\u2013697. OpenProceedings.org; 2016. https:\/\/doi.org\/10.5441\/002\/edbt.2016.92.","DOI":"10.5441\/002\/edbt.2016.92"},{"key":"457_CR94","doi-asserted-by":"publisher","unstructured":"Zampieri M, Malmasi S, Nakov P, Rosenthal S, Farra N, Kumar R. Predicting the type and target of offensive posts in social media. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 1415\u20131420. Association for Computational Linguistics, Minneapolis, Minnesota; 2019. https:\/\/doi.org\/10.18653\/v1\/N19-1144. https:\/\/www.aclweb.org\/anthology\/N19-1144.","DOI":"10.18653\/v1\/N19-1144"},{"key":"457_CR95","doi-asserted-by":"crossref","unstructured":"Zampieri M, Malmasi S, Nakov P, Rosenthal S, Farra N, Kumar R. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). In: Proceedings of the 13th International Workshop on Semantic Evaluation, pp. 75\u201386; 2019.","DOI":"10.18653\/v1\/S19-2010"},{"key":"457_CR96","doi-asserted-by":"crossref","unstructured":"Zampieri M, Nakov P, Rosenthal S, Atanasova P, Karadzhov G, Mubarak H, Derczynski L, Pitenis Z, \u00c7\u00f6ltekin c. SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020). In: Proceedings of SemEval; 2020.","DOI":"10.18653\/v1\/2020.semeval-1.188"},{"issue":"2","key":"457_CR97","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/j.neuroimage.2007.02.044","volume":"36","author":"Y Zhang","year":"2007","unstructured":"Zhang Y, Zhou X, Witt RM, Sabatini BL, Adjeroh D, Wong ST. Dendritic spine detection using curvilinear structure detector and lda classifier. Neuroimage. 2007;36(2):346\u201360.","journal-title":"Neuroimage"},{"key":"457_CR98","doi-asserted-by":"publisher","unstructured":"Zhang Z, Luo L. Hate speech detection: A solved problem? the challenging case of long tail on twitter. Semantic Web Accepted; 2018. https:\/\/doi.org\/10.3233\/SW-180338.","DOI":"10.3233\/SW-180338"},{"key":"457_CR99","doi-asserted-by":"publisher","first-page":"745","DOI":"10.1007\/978-3-319-93417-4_48","volume-title":"The Semantic Web","author":"Z Zhang","year":"2018","unstructured":"Zhang Z, Robinson D, Tepper J. Detecting hate speech on twitter using a convolution-gru based deep neural network. In: Gangemi A, Navigli R, Vidal ME, Hitzler P, Troncy R, Hollink L, Tordai A, Alam M, editors. The Semantic Web. Cham: Springer; 2018. p. 745\u201360."},{"key":"457_CR100","doi-asserted-by":"publisher","unstructured":"Zhu X, Vondrick C, Ramanan D, Fowlkes CC. Do we need more training data or better models for object detection? In: Bowden R, Collomosse JP, Mikolajczyk K (eds.) British Machine Vision Conference, BMVC 2012, Surrey, UK, September 3-7, 2012, pp. 1\u201311. BMVA Press (2012). https:\/\/doi.org\/10.5244\/C.26.80.","DOI":"10.5244\/C.26.80"},{"key":"457_CR101","unstructured":"Zia T, Akram M, Nawaz M, Shahzad B, Abdullatif A, Mustafa R, Lali M. Identification of hatred speeches on twitter. In: Proceedings of 52nd The IRES International Conference, pp. 27\u201332; 2016."},{"key":"457_CR102","unstructured":"Zimbardo PG. The human choice: individuation, reason, and order versus deindividuation, impulse, and chaos. In: Nebraska Symposium on Motivation 17, pp. 237\u2013307; 1969."},{"key":"457_CR103","unstructured":"Zimmerman S, Kruschwitz U, Fox C. Improving hate speech detection with deep learning ensembles. In: Proc. LREC). European Language Resources Association (ELRA), Miyazaki, Japan; 2018. https:\/\/www.aclweb.org\/anthology\/L18-1404."}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00457-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s42979-021-00457-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-021-00457-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T07:55:59Z","timestamp":1671263759000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s42979-021-00457-3"}},"subtitle":["Data Scarcity, and Leveraging External Resources"],"short-title":[],"issued":{"date-parts":[[2021,2,13]]},"references-count":103,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,4]]}},"alternative-id":["457"],"URL":"https:\/\/doi.org\/10.1007\/s42979-021-00457-3","relation":{},"ISSN":["2662-995X","2661-8907"],"issn-type":[{"value":"2662-995X","type":"print"},{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,13]]},"assertion":[{"value":"4 May 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 January 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with Ethical Standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}}],"article-number":"95"}}