{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,11]],"date-time":"2025-11-11T15:53:49Z","timestamp":1762876429717,"version":"3.37.3"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,3,1]],"date-time":"2023-03-01T00:00:00Z","timestamp":1677628800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T00:00:00Z","timestamp":1680048000000},"content-version":"vor","delay-in-days":28,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universit\u00e4t Hildesheim"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Datenbank Spektrum"],"published-print":{"date-parts":[[2023,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Hate speech is a\u00a0persistent issue in social media. Researchers have analyzed and developed detection methods for hate speech on the basis of example data, even though the phenomenon is only rather vaguely defined. This paper provides an approach to identify hate speech in terms of German laws, which are used as a\u00a0basis for annotation guidelines applied to real world data. We annotate six labels in a\u00a0corpus of 1,385 German short text messages: four subcategories of illegal hate speech, offensive language and a\u00a0neutral class. We consider hate speech expressions as illegal if the linguistic content could be interpreted in a\u00a0given context possibly violating a\u00a0specific law. This interpretation and a\u00a0check by lawyers would be the next step which is not yet included in our annotation. In this paper, I report on strategies to avoid certain biases in data for illegal hate speech. These strategies may serve as a\u00a0model for building a\u00a0larger dataset. In experiments, I investigate the capability of a\u00a0Transformer-based neural network model to learn our classification. The results show that this multiclass classification is still difficult to learn, probably due to the small size of the dataset. I suggest that it is crucial to be aware of data biases and to apply bias mitigation techniques when training hate speech detection systems on such data. Data and scripts of the experiments are made publicly available.<\/jats:p>","DOI":"10.1007\/s13222-023-00439-0","type":"journal-article","created":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T10:02:47Z","timestamp":1680084167000},"page":"41-51","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Bias Mitigation for Capturing Potentially Illegal Hate Speech"],"prefix":"10.1007","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5226-4206","authenticated-orcid":false,"given":"Johannes","family":"Sch\u00e4fer","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,3,29]]},"reference":[{"key":"439_CR1","doi-asserted-by":"publisher","first-page":"54","DOI":"10.18653\/v1\/S19-2007","volume-title":"Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA","author":"V Basile","year":"2019","unstructured":"Basile V, Bosco C, Fersini E, Nozza D, Patti V, Rangel Pardo FM, Rosso P, Sanguinetti M (2019) SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter. In: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, pp 54\u201363 https:\/\/doi.org\/10.18653\/v1\/S19-2007"},{"key":"439_CR2","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1145\/3308560.3317593","volume-title":"Companion proceedings of the 2019 world wide web conference","author":"D Borkan","year":"2019","unstructured":"Borkan D, Dixon L, Sorensen J, Thain N, Vasserman L (2019) Nuanced metrics for measuring unintended bias with real data for text classification. In: Companion proceedings of the 2019 world wide web conference, pp 491\u2013500 https:\/\/doi.org\/10.1145\/3308560.3317593"},{"doi-asserted-by":"crossref","unstructured":"Bosco C, Felice D, Poletto F, Sanguinetti M, Maurizio T (2018) Overview of the EVALITA 2018 hate speech detection task. In: EVALITA 2018-Sixth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian, CEUR, vol 2263, pp 1\u20139. http:\/\/ceur-ws.org\/Vol-2263\/paper010.pdf. Accessed 23 Mar 2023","key":"439_CR3","DOI":"10.4000\/books.aaccademia.4503"},{"key":"439_CR4","doi-asserted-by":"publisher","first-page":"6788","DOI":"10.18653\/v1\/2020.coling-main.598","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain (Online)","author":"B Chan","year":"2020","unstructured":"Chan B, Schweter S, M\u00f6ller T (2020) German\u2019s next language model. In: Proceedings of the 28th International Conference on Computational Linguistics, International Committee on Computational Linguistics, Barcelona, Spain (Online), pp 6788\u20136796 https:\/\/doi.org\/10.18653\/v1\/2020.coling-main.598"},{"issue":"1","key":"439_CR5","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J (1960) A\u00a0coefficient of agreement for nominal scales. Educ Psychol Meas 20(1):37\u201346. https:\/\/doi.org\/10.1177\/001316446002000104","journal-title":"Educ Psychol Meas"},{"key":"439_CR6","doi-asserted-by":"publisher","first-page":"25","DOI":"10.18653\/v1\/W19-3504","volume-title":"Proceedings of the Third Workshop on Abusive Language Online, Florence, Italy","author":"T Davidson","year":"2019","unstructured":"Davidson T, Bhattacharya D, Weber I (2019) Racial bias in hate speech and abusive language detection datasets. In: Proceedings of the Third Workshop on Abusive Language Online, Florence, Italy, pp 25\u201335 https:\/\/doi.org\/10.18653\/v1\/W19-3504"},{"key":"439_CR7","doi-asserted-by":"publisher","first-page":"143","DOI":"10.18653\/v1\/2022.woah-1.14","volume-title":"Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), Association for Computational Linguistics, Seattle, Washington (Hybrid)","author":"C Demus","year":"2022","unstructured":"Demus C, Pitz J, Sch\u00fctz M, Probol N, Siegel M, Labudde D (2022) Detox: A\u00a0comprehensive dataset for German offensive language and conversation analysis. In: Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH), Association for Computational Linguistics, Seattle, Washington (Hybrid), pp 143\u2013153 https:\/\/doi.org\/10.18653\/v1\/2022.woah-1.14"},{"key":"439_CR8","series-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi-asserted-by":"publisher","first-page":"4171","DOI":"10.18653\/v1\/N19-1423","volume-title":"Long and Short Papers","author":"J Devlin","year":"2019","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. In: Long and Short Papers. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol 1, pp 4171\u20134186 https:\/\/doi.org\/10.18653\/v1\/N19-1423"},{"key":"439_CR9","first-page":"42","volume-title":"Proceedings of the Twelfth International AAAI Conference on Web and Social Media (ICWSM 2018)","author":"M ElSherief","year":"2018","unstructured":"ElSherief M, Kulkarni V, Nguyen D, Wang WY, Belding E (2018) Hate Lingo: A\u00a0target-based linguistic analysis of hate speech in social media. In: Proceedings of the Twelfth International AAAI Conference on Web and Social Media (ICWSM 2018), pp 42\u201351"},{"issue":"5","key":"439_CR10","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","volume":"76","author":"JL Fleiss","year":"1971","unstructured":"Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76(5):378\u2013382. https:\/\/doi.org\/10.1037\/h0031619","journal-title":"Psychol Bull"},{"unstructured":"Kingma DP, Ba J (2014) Adam: A\u00a0method for stochastic optimization. arXiv preprint arXiv:14126980. https:\/\/arxiv.org\/abs\/1412.6980. Accessed 23 Mar 2023","key":"439_CR11"},{"issue":"1","key":"439_CR12","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33(1):159\u2013174. https:\/\/doi.org\/10.2307\/2529310","journal-title":"Biometrics"},{"key":"439_CR13","doi-asserted-by":"publisher","first-page":"87","DOI":"10.18653\/v1\/S19-2011","volume-title":"Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA","author":"P Liu","year":"2019","unstructured":"Liu P, Li W, Zou L (2019) NULI at SemEval-2019 Task 6: Transfer learning for offensive language detection using bidirectional transformers. In: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, pp 87\u201391 https:\/\/doi.org\/10.18653\/v1\/S19-2011"},{"unstructured":"Mandl T, Modha S, Shahi GK, Jaiswal AK, Nandini D, Patel D, Majumder P, Sch\u00e4fer J (2020) Overview of the HASOC track at FIRE 2020: Hate speech and offensive content identification in Indo-European languages. In: Working Notes of FIRE 2020 - Forum for Information Retrieval Evaluation, CEUR Workshop Proceedings, Hyderabad, India, pp 87\u2013111. http:\/\/ceur-ws.org\/Vol-2826\/T2-1.pdf. Accessed 23 Mar 2023","key":"439_CR14"},{"doi-asserted-by":"crossref","unstructured":"Mandl T, Modha S, Shahi GK, Madhu H, Satapara S, Majumder P, Sch\u00e4fer J, Ranasinghe T, Zampieri M, Nandini D, Jaiswal AK (2021) Overview of the HASOC subtrack at FIRE 2021: Hate speech and offensive content identification in English and Indo-Aryan Languages. In: Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, CEUR Workshop Proceedings, India, pp 1\u201319. http:\/\/ceur-ws.org\/Vol-3159\/T1-1.pdf. Accessed 23 Mar 2023","key":"439_CR15","DOI":"10.1145\/3503162.3503176"},{"unstructured":"Paraschiv A, Cercel DC (2019) UPB at GermEval-2019 Task 2: BERT-based offensive language classification of German Tweets. In: Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019). https:\/\/corpora.linguistik.uni-erlangen.de\/data\/konvens\/proceedings\/papers\/germeval\/Germeval_Task_2_2019_paper_9.UPB.pdf. Accessed 23 Mar 2023","key":"439_CR16"},{"unstructured":"Risch J, Stoll A, Ziegele M, Krestel R (2019) hpiDEDIS at GermEval 2019: Offensive language identification using a\u00a0German BERT model. In: Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019). https:\/\/corpora.linguistik.uni-erlangen.de\/data\/konvens\/proceedings\/papers\/germeval\/Germeval_Task_2_2019_paper_10.HPIDEDIS.pdf. Accessed 23 Mar 2023","key":"439_CR17"},{"unstructured":"Ruppenhofer J, Siegel M, Wiegand M (2018) Guidelines for IGGSA Shared Task on the identification of offensive language. http:\/\/www.melaniesiegel.de\/publications\/2018_GermEval_Guidelines.pdf. Accessed 23 Mar 2023","key":"439_CR18"},{"unstructured":"Sch\u00e4fer J, Boguslu K (2021) Towards annotating illegal hate speech: A\u00a0computational linguistic approach. Detect Then Act (DTCT) technical report 3. https:\/\/dtct.eu\/wp-content\/uploads\/2021\/10\/DTCT-TR3-CL.pdf. Accessed 23 Mar 2023","key":"439_CR19"},{"key":"439_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/w17-1101","volume-title":"Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media (SocialNLP@EACL 2017), Valencia, Spain","author":"A Schmidt","year":"2017","unstructured":"Schmidt A, Wiegand M (2017) A\u00a0survey on hate speech detection using natural language processing. In: Proceedings of the Fifth International Workshop on Natural Language Processing for Social Media (SocialNLP@EACL 2017), Valencia, Spain, pp 1\u201310 https:\/\/doi.org\/10.18653\/v1\/w17-1101"},{"key":"439_CR21","first-page":"354","volume-title":"Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019)","author":"J Stru\u00df","year":"2019","unstructured":"Stru\u00df J, Siegel M, Ruppenhofer J, Wiegand M, Klenner M (2019) Overview of GermEval task 2, 2019 shared task on the identification of offensive language. In: Proceedings of the 15th Conference on Natural Language Processing (KONVENS 2019). German Society for Computational Linguistics & Language Technology, Erlangen, pp 354\u2013365"},{"key":"439_CR22","volume-title":"Advances in neural information processing systems","author":"A Vaswani","year":"2017","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems, vol 30. Curran Associates, Red Hook"},{"key":"439_CR23","doi-asserted-by":"publisher","first-page":"1638","DOI":"10.18653\/v1\/2020.semeval-1.213","volume-title":"Proceedings of the Fourteenth Workshop on Semantic Evaluation (SemEval@COLING 2020)","author":"G Wiedemann","year":"2020","unstructured":"Wiedemann G, Yimam SM, Biemann C (2020) UHH-LT at SemEval-2020 Task 12: Fine-tuning of pre-trained transformer networks for offensive language detection. In: Proceedings of the Fourteenth Workshop on Semantic Evaluation (SemEval@COLING 2020), pp 1638\u20131644 https:\/\/doi.org\/10.18653\/v1\/2020.semeval-1.213"},{"key":"439_CR24","first-page":"1","volume-title":"Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018)","author":"M Wiegand","year":"2018","unstructured":"Wiegand M, Siegel M, Ruppenhofer J (2018) Overview of the GermEval 2018 shared task on the identification of offensive language. In: Proceedings of GermEval 2018, 14th Conference on Natural Language Processing (KONVENS 2018). \u00d6sterreichische Akademie der Wissenschaften, Vienna, pp 1\u201310"},{"key":"439_CR25","series-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","doi-asserted-by":"publisher","first-page":"602","DOI":"10.18653\/v1\/N19-1060","volume-title":"Long and short papers","author":"M Wiegand","year":"2019","unstructured":"Wiegand M, Ruppenhofer J, Kleinbauer T (2019) Detection of abusive language: the problem of biased datasets. In: Long and short papers. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol 1. Association for Computational Linguistics, Minneapolis, pp 602\u2013608 https:\/\/doi.org\/10.18653\/v1\/N19-1060"},{"key":"439_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18653\/v1\/2022.woah-1.1","volume-title":"Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH)","author":"S Yuan","year":"2022","unstructured":"Yuan S, Maronikolakis A, Sch\u00fctze H (2022) Separating hate speech and offensive language classes via adversarial debiasing. In: Proceedings of the Sixth Workshop on Online Abuse and Harms (WOAH). Association for Computational Linguistics, Seattle, pp 1\u201310 https:\/\/doi.org\/10.18653\/v1\/2022.woah-1.1"},{"key":"439_CR27","doi-asserted-by":"publisher","first-page":"75","DOI":"10.18653\/v1\/s19-2010","volume-title":"Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval@NAACL-HLT 2019)","author":"M Zampieri","year":"2019","unstructured":"Zampieri M, Malmasi S, Nakov P, Rosenthal S, Farra N, Kumar R (2019) SemEval-2019 Task 6: Identifying and categorizing offensive language in social media (OffensEval). In: Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval@NAACL-HLT 2019), pp 75\u201386 https:\/\/doi.org\/10.18653\/v1\/s19-2010"}],"container-title":["Datenbank-Spektrum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13222-023-00439-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13222-023-00439-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13222-023-00439-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,15]],"date-time":"2023-05-15T15:43:37Z","timestamp":1684165417000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13222-023-00439-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,3]]}},"alternative-id":["439"],"URL":"https:\/\/doi.org\/10.1007\/s13222-023-00439-0","relation":{},"ISSN":["1618-2162","1610-1995"],"issn-type":[{"type":"print","value":"1618-2162"},{"type":"electronic","value":"1610-1995"}],"subject":[],"published":{"date-parts":[[2023,3]]},"assertion":[{"value":"22 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}