{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T18:30:49Z","timestamp":1791570649057,"version":"4.3.4"},"reference-count":36,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,12,22]],"date-time":"2020-12-22T00:00:00Z","timestamp":1608595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The last two decades have seen an exponential increase in the use of the Internet and social media, which has changed basic human interaction. This has led to many positive outcomes. At the same time, it has brought risks and harms. The volume of harmful content online, such as hate speech, is not manageable by humans. The interest in the academic community to investigate automated means for hate speech detection has increased. In this study, we analyse six publicly available datasets by combining them into a single homogeneous dataset. Having classified them into three classes, abusive, hateful or neither, we create a baseline model and improve model performance scores using various optimisation techniques. After attaining a competitive performance score, we create a tool that identifies and scores a page with an effective metric in near-real-time and uses the same feedback to re-train our model. We prove the competitive performance of our multilingual model in two languages, English and Hindi. This leads to comparable or superior performance to most monolingual models.<\/jats:p>","DOI":"10.3390\/info12010005","type":"journal-article","created":{"date-parts":[[2020,12,22]],"date-time":"2020-12-22T20:39:29Z","timestamp":1608669569000},"page":"5","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":69,"title":["Online Multilingual Hate Speech Detection: Experimenting with Hindi and English Social Media"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3569-5212","authenticated-orcid":false,"given":"Neeraj","family":"Vashistha","sequence":"first","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4583-3623","authenticated-orcid":false,"given":"Arkaitz","family":"Zubiaga","sequence":"additional","affiliation":[{"name":"School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Cortese, A.J.P. (2006). Opposing Hate Speech, Greenwood Publishing Group.","DOI":"10.5040\/9798400693625"},{"key":"ref_2","unstructured":"(2018). Abusive and Offensive Online Communications: A Scoping Report, The Law Commission."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Vidgen, B., and Derczynski, L. (2020). Directions in Abusive Language Training Data: Garbage In, Garbage Out. arXiv.","DOI":"10.1371\/journal.pone.0243300"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mandl, T., Modha, S., Majumder, P., Patel, D., Dave, M., Mandlia, C., and Patel, A. (2019, January 12\u201315). Overview of the HASOC Track at FIRE 2019: Hate Speech and Offensive Content Identification in Indo-European Languages. Proceedings of the 11th Forum for Information Retrieval Evaluation, Kolkata, India. FIRE \u201919.","DOI":"10.1145\/3368567.3368584"},{"key":"ref_5","unstructured":"Davidson, T., Warmsley, D., Macy, M.W., and Weber, I. (2017, January 15\u201318). Automated Hate Speech Detection and the Problem of Offensive Language. Proceedings of the Eleventh International Conference on Web and Social Media, ICWSM 2017, Montr\u00e9al, QC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"ElSherief, M., Nilizadeh, S., Nguyen, D., Vigna, G., and Belding, E. (2018). Peer to Peer Hate: Hate Speech Instigators and Their Targets. arXiv.","DOI":"10.1609\/icwsm.v12i1.15038"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ousidhoum, N., Lin, Z., Zhang, H., Song, Y., and Yeung, D.Y. (2019). Multilingual and Multi-Aspect Hate Speech Analysis. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics.","DOI":"10.18653\/v1\/D19-1474"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Basile, V., Bosco, C., Fersini, E., Nozza, D., Patti, V., Rangel Pardo, F.M., Rosso, P., and Sanguinetti, M. (2019). SemEval-2019 Task 5: Multilingual Detection of Hate Speech Against Immigrants and Women in Twitter. Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/S19-2007"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mathur, P., Sawhney, R., Ayyar, M., and Shah, R. (2018). Did you offend me? Classification of Offensive Tweets in Hinglish Language. Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-5118"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Waseem, Z., and Hovy, D. (2016). Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter. Proceedings of the NAACL Student Research Workshop, Association for Computational Linguistics.","DOI":"10.18653\/v1\/N16-2013"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"13825","DOI":"10.1109\/ACCESS.2018.2806394","article-title":"Hate Speech on Twitter: A Pragmatic Approach to Collect Hateful and Offensive Expressions and Perform Hate Speech Detection","volume":"6","author":"Watanabe","year":"2018","journal-title":"IEEE Access"},{"key":"ref_12","unstructured":"Smith, S.L., Turban, D.H.P., Hamblin, S., and Hammerla, N.Y. (2017). Offline bilingual word vectors, orthogonal transformations and the inverted softmax. arXiv."},{"key":"ref_13","unstructured":"Lample, G., Conneau, A., Denoyer, L., and Ranzato, M. (2017). Unsupervised Machine Translation Using Monolingual Corpora Only. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., and Gao, J. (2020). Deep Learning Based Text Classification: A Comprehensive Review. arXiv.","DOI":"10.1145\/3439726"},{"key":"ref_15","unstructured":"Ruder, S., Bingel, J., Augenstein, I., and S\u00f8gaard, A. (2017). Sluice networks: Learning what to share between loosely related tasks. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Bohra, A., Vijay, D., Singh, V., Akhtar, S.S., and Shrivastava, M. (2018). A Dataset of Hindi-English Code-Mixed Social Media Text for Hate Speech Detection. Proceedings of the Second Workshop on Computational Modeling of People\u2019s Opinions, Personality, and Emotions in Social Media, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-1105"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sawhney, R., Manchanda, P., Singh, R., and Aggarwal, S. (2018). A Computational Approach to Feature Extraction for Identification of Suicidal Ideation in Tweets. Proceedings of ACL 2018, Student Research Workshop, Association for Computational Linguistics.","DOI":"10.18653\/v1\/P18-3013"},{"key":"ref_18","first-page":"267","article-title":"The pragmatics of swearing","volume":"4","author":"Jay","year":"2008","journal-title":"J. Politeness Res. Lang. Behav. Cult."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Santosh, T., and Aravind, K. (2019, January 3\u20135). Hate Speech Detection in Hindi-English Code-Mixed Social Media Text. Proceedings of the ACM India Joint International Conference on Data Science and Management of Data, Kolkata, India.","DOI":"10.1145\/3297001.3297048"},{"key":"ref_20","unstructured":"Kamble, S., and Joshi, A. (2018). Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kshirsagar, R., Cukuvac, T., McKeown, K., and McGregor, S. (2018). Predictive Embeddings for Hate Speech Detection on Twitter. arXiv.","DOI":"10.18653\/v1\/W18-5104"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sohn, H., and Lee, H. (2019, January 8\u201311). MC-BERT4HATE: Hate Speech Detection using Multi-channel BERT for Different Languages and Translations. Proceedings of the 2019 International Conference on Data Mining Workshops (ICDMW), Beijing, China.","DOI":"10.1109\/ICDMW.2019.00084"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chen, W., Su, Y., Shen, Y., Chen, Z., Yan, X., and Wang, W.Y. (2019). How Large a Vocabulary Does Text Classification Need? A Variational Approach to Vocabulary Selection. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/N19-1352"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Baziotis, C., Pelekis, N., and Doulkeridis, C. (2017, January 3\u20134). DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), Vancouver, QC, Canada.","DOI":"10.18653\/v1\/S17-2126"},{"key":"ref_25","unstructured":"Kunchukuttan, A. (2020, December 20). The IndicNLP Library. Available online: https:\/\/github.com\/anoopkunchukuttan\/indic_nlp_library\/blob\/master\/docs\/indicnlp.pdf."},{"key":"ref_26","unstructured":"Loper, E., and Bird, S. (2020, January 7\u201312). NLTK: The Natural Language Toolkit. Proceedings of the ACL Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics, Philadelphia, PA, USA."},{"key":"ref_27","unstructured":"Zhang, X., and LeCun, Y. (2015). Text Understanding from Scratch. arXiv."},{"key":"ref_28","unstructured":"Jozefowicz, R., Vinyals, O., Schuster, M., Shazeer, N., and Wu, Y. (2016). Exploring the Limits of Language Modeling. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kim, Y., Jernite, Y., Sontag, D., and Rush, A.M. (2015). Character-Aware Neural Language Models. arXiv.","DOI":"10.1609\/aaai.v30i1.10362"},{"key":"ref_30","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics."},{"key":"ref_31","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv."},{"key":"ref_32","unstructured":"Cherifi, H., Gaito, S., Mendes, J.F., Moro, E., and Rocha, L.M. (2020). A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media. Complex Networks and Their Applications VIII, Springer International Publishing."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Badjatiya, P., Gupta, S., Gupta, M., and Varma, V. (2017, January 3\u20137). Deep Learning for Hate Speech Detection in Tweets. Proceedings of the 26th International Conference on World Wide Web Companion\u2014WWW \u201917 Companion, Perth, Australia.","DOI":"10.1145\/3041021.3054223"},{"key":"ref_34","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzm\u00e1n, F., Grave, E., Ott, M., Zettlemoyer, L., and Stoyanov, V. (2020). Unsupervised Cross-lingual Representation Learning at Scale. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1017\/S0008413100010677","article-title":"Grammatical Constraints on Code-Mixing: Evidence from Hindi-English","volume":"30","author":"Singh","year":"1985","journal-title":"Can. J. Linguist. Rev. Can. Linguist."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/1\/5\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:48:33Z","timestamp":1760179713000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/1\/5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,22]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["info12010005"],"URL":"https:\/\/doi.org\/10.3390\/info12010005","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202011.0646.v1","asserted-by":"object"}]},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,22]]}}}