{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T19:29:07Z","timestamp":1773862147784,"version":"3.50.1"},"reference-count":78,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T00:00:00Z","timestamp":1633910400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T00:00:00Z","timestamp":1633910400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010783","name":"Slovensk\u00e1 Akad\u00e9mia Vied","doi-asserted-by":"publisher","award":["APP0175"],"award-info":[{"award-number":["APP0175"]}],"id":[{"id":"10.13039\/100010783","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005357","name":"Agent\u00fara na Podporu V\u00fdskumu a V\u00fdvoja","doi-asserted-by":"publisher","award":["APVV-18-0140"],"award-info":[{"award-number":["APVV-18-0140"]}],"id":[{"id":"10.13039\/501100005357","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005357","name":"Agent\u00fara na Podporu V\u00fdskumu a V\u00fdvoja","doi-asserted-by":"publisher","award":["APVV-20-0319"],"award-info":[{"award-number":["APVV-20-0319"]}],"id":[{"id":"10.13039\/501100005357","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006109","name":"VEGA","doi-asserted-by":"crossref","award":["2\/0068\/19"],"award-info":[{"award-number":["2\/0068\/19"]}],"id":[{"id":"10.13039\/501100006109","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Hate speech should be tackled and prosecuted based on how it is operationalized. However, the existing theoretical definitions of hate speech are not sufficiently fleshed out or easily operable. To overcome this inadequacy, and with the help of interdisciplinary experts, we propose an empirical definition of hate speech by providing a list of 10 hate speech indicators and the rationale behind them (the indicators refer to specific, observable, and measurable characteristics that offer a practical definition of hate speech). A preliminary exploratory examination of the structure of hate speech, with the focus on comments related to migrants (one of the most reported grounds of hate speech), revealed that two indicators in particular, <jats:italic>denial of human rights<\/jats:italic> and <jats:italic>promoting violent behavior,<\/jats:italic> occupy a central role in the network of indicators. Furthermore, we discuss the practical implications of the proposed hate speech indicators\u2014especially (semi-)automatic detection using the latest natural language processing (NLP) and machine learning (ML) methods. Having a set of quantifiable indicators could benefit researchers, human right activists, educators, analysts, and regulators by providing them with a pragmatic approach to hate speech assessment and detection.<\/jats:p>","DOI":"10.1007\/s40747-021-00561-0","type":"journal-article","created":{"date-parts":[[2021,10,11]],"date-time":"2021-10-11T09:21:36Z","timestamp":1633944096000},"page":"2827-2842","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Hate speech operationalization: a preliminary examination of hate speech indicators and their structure"],"prefix":"10.1007","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9358-1419","authenticated-orcid":false,"given":"Jana","family":"Papcunov\u00e1","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4869-6900","authenticated-orcid":false,"given":"Marcel","family":"Marton\u010dik","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6816-1802","authenticated-orcid":false,"given":"Denisa","family":"Fed\u00e1kov\u00e1","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2341-2457","authenticated-orcid":false,"given":"Michal","family":"Kento\u0161","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4295-1464","authenticated-orcid":false,"given":"Miroslava","family":"Bozog\u00e1\u0148ov\u00e1","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3511-5337","authenticated-orcid":false,"given":"Ivan","family":"Srba","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3052-8290","authenticated-orcid":false,"given":"Robert","family":"Moro","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1353-9462","authenticated-orcid":false,"given":"Mat\u00fa\u0161","family":"Pikuliak","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2306-4408","authenticated-orcid":false,"given":"Mari\u00e1n","family":"\u0160imko","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9648-9108","authenticated-orcid":false,"given":"Mat\u00fa\u0161","family":"Adamkovi\u010d","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,10,11]]},"reference":[{"key":"561_CR1","unstructured":"EU Commission (2016) Eurobarometer, S. 452, media pluralism and democracy. Retrieved from https:\/\/ec.europa.eu"},{"key":"561_CR2","unstructured":"Eurostat (2020) Being young in Europe today\u2014digital world. Retrieved from https:\/\/ec.europa.eu\/eurostat"},{"key":"561_CR3","unstructured":"Council of Europe (2013) No hate speech movement. Retrieved from https:\/\/www.coe.int\/en\/web\/no-hate-campaign"},{"key":"561_CR4","unstructured":"ECRI (2016) General Policy Recommendation No. 15 on combating hate speech. Retrieved from https:\/\/coe.int"},{"key":"561_CR5","doi-asserted-by":"crossref","unstructured":"Mathew B, Saha P, Tharad H, Rajgaria S, Singhania P, Maity SK, Mukherjee A et al (2019) Thou shalt not hate: countering online hate speech. In: Proceedings of the international AAAI conference on web and social media, vol 13, pp 369\u2013380","DOI":"10.1609\/icwsm.v13i01.3237"},{"key":"561_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13673-019-0205-6","volume":"10","author":"J Salminen","year":"2020","unstructured":"Salminen J, Hopf M, Chowdhury SA et al (2020) Developing an online hate classifier for multiple social media platforms. Hum Cent Comput Inf Sci 10:1. https:\/\/doi.org\/10.1186\/s13673-019-0205-6","journal-title":"Hum Cent Comput Inf Sci"},{"issue":"3","key":"561_CR7","doi-asserted-by":"publisher","DOI":"10.1111\/spc3.12521","volume":"14","author":"R Sambaraju","year":"2020","unstructured":"Sambaraju R, McVittie C (2020) Examining abuse in online media. Soc Pers Psychol Compass 14(3):e12521. https:\/\/doi.org\/10.1111\/spc3.12521","journal-title":"Soc Pers Psychol Compass"},{"issue":"8","key":"561_CR8","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0221152","volume":"14","author":"S MacAvaney","year":"2019","unstructured":"MacAvaney S, Yao HR, Yang E, Russell K, Goharian N, Frieder O (2019) Hate speech detection: challenges and solutions. PLoS ONE 14(8):e0221152. https:\/\/doi.org\/10.1371\/journal.pone.0221152","journal-title":"PLoS ONE"},{"issue":"4","key":"561_CR9","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1007\/s10982-017-9297-1","volume":"36","author":"A Brown","year":"2017","unstructured":"Brown A (2017) What is hate speech? Part 1: the myth of hate. Law Philos 36(4):419\u2013468. https:\/\/doi.org\/10.1007\/s10982-017-9297-1","journal-title":"Law Philos"},{"issue":"4","key":"561_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3232676","volume":"51","author":"P Fortuna","year":"2018","unstructured":"Fortuna P, Nunes S (2018) A survey on automatic detection of hate speech in text. ACM Comput Surv (CSUR) 51(4):1\u201330. https:\/\/doi.org\/10.1145\/3232676","journal-title":"ACM Comput Surv (CSUR)"},{"key":"561_CR11","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-658-31793-5_14","author":"I Srba","year":"2021","unstructured":"Srba I, Lenzini G, Pikuliak M, Pecar S (2021) Addressing hate speech with data science: an overview from computer science perspective. Hate Speech Multidisziplin\u00e4re Analysen Und Handlungsoptionen. https:\/\/doi.org\/10.1007\/978-3-658-31793-5_14","journal-title":"Hate Speech Multidisziplin\u00e4re Analysen Und Handlungsoptionen"},{"key":"561_CR12","doi-asserted-by":"crossref","unstructured":"Waseem Z, Davidson T, Warmsley D, Weber I (2017) Understanding abuse: a typology of abusive language detection subtasks. arXiv preprint http:\/\/arxiv.org\/abs\/1705.09899","DOI":"10.18653\/v1\/W17-3012"},{"key":"561_CR13","doi-asserted-by":"publisher","unstructured":"Zampieri M, Malmasi S, Nakov P, Rosenthal S, Farra N, Kumar R (2019) Predicting the type and target of offensive posts in social media. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1, pp 1415\u20131420. https:\/\/doi.org\/10.18653\/v1\/n19-1144","DOI":"10.18653\/v1\/n19-1144"},{"key":"561_CR14","volume-title":"Diagnostic and statistical manual of mental disorders","author":"APA","year":"2013","unstructured":"APA (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Association, Arlington","edition":"5"},{"key":"561_CR15","unstructured":"World Health Organization (2018) International classification of diseases for mortality and morbidity statistics (11th revision). Retrieved from https:\/\/icd.who.int\/browse11\/l-m\/en"},{"key":"561_CR16","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2020.00003","author":"S-M Laaksonen","year":"2020","unstructured":"Laaksonen S-M, Haapoja J, Kinnunen T, Nelimarkka M, P\u00f6yht\u00e4ri R (2020) The datafication of hate: expectations and challenges in automated hate speech monitoring. Front Big Data. https:\/\/doi.org\/10.3389\/fdata.2020.00003","journal-title":"Front Big Data"},{"issue":"4","key":"561_CR17","doi-asserted-by":"publisher","first-page":"215824402097302","DOI":"10.1177\/2158244020973022","volume":"10","author":"MA Paz","year":"2020","unstructured":"Paz MA, Montero-D\u00edaz J, Moreno-Delgado A (2020) Hate speech: a systematized review. SAGE Open 10(4):215824402097302. https:\/\/doi.org\/10.1177\/2158244020973022","journal-title":"SAGE Open"},{"issue":"9","key":"561_CR18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0222194","volume":"14","author":"A Waqas","year":"2019","unstructured":"Waqas A, Salminen J, Jung S, Almerekhi H, Jansen BJ (2019) Mapping online hate: a scientometric analysis on research trends and hotspots in research on online hate. PLoS ONE 14(9):e0222194. https:\/\/doi.org\/10.1371\/journal.pone.0222194","journal-title":"PLoS ONE"},{"key":"561_CR19","unstructured":"Council of Europe (1997) Recommendation No. R (97) 20 of the Committee of Ministers to member states on \u201cHate Speech\u201d. Retrieved from https:\/\/rm.coe.int\/1680505d5b"},{"key":"561_CR20","unstructured":"EU Commission (2016) Code of conduct on countering illegal hate speech online. Retrieved from https:\/\/ec.europa.eu"},{"key":"561_CR21","unstructured":"UN General Assembly, International Covenant on Civil and Political Rights (1966) United Nations, Treaty Series, vol 999, p 171"},{"issue":"5","key":"561_CR22","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1007\/s10982-017-9300-x","volume":"36","author":"A Brown","year":"2017","unstructured":"Brown A (2017) What is hate speech? Part 2: family resemblances. Law Philos 36(5):561\u2013613. https:\/\/doi.org\/10.1007\/s10982-017-9300-x","journal-title":"Law Philos"},{"issue":"1","key":"561_CR23","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1111\/j.1460-2466.1997.tb02690.x","volume":"47","author":"C Calvert","year":"1997","unstructured":"Calvert C (1997) Hate speech and its harms: a communication theory perspective. J Commun 47(1):4\u201319. https:\/\/doi.org\/10.1111\/j.1460-2466.1997.tb02690.x","journal-title":"J Commun"},{"key":"561_CR24","doi-asserted-by":"publisher","unstructured":"Guterres A (2019) United Nations strategy and plan of action on hate speech. Taken from: https:\/\/www.un.org\/en\/genocideprevention\/documents\/U, (20Strategy). https:\/\/doi.org\/10.1163\/2210-7975_hrd-9915-20190006","DOI":"10.1163\/2210-7975_hrd-9915-20190006"},{"key":"561_CR25","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1017\/CBO9781139042871.006","volume-title":"The content and context of hate speech: rethinking regulation and responses","author":"B Parekh","year":"2012","unstructured":"Parekh B (2012) Is there a case for banning hate speech? In: Herz M, Molnar P (eds) The content and context of hate speech: rethinking regulation and responses. Cambridge University Press, Cambridge, pp 37\u201356. https:\/\/doi.org\/10.1017\/CBO9781139042871.006"},{"issue":"3","key":"561_CR26","doi-asserted-by":"publisher","first-page":"89","DOI":"10.2202\/1944-2866.1059","volume":"3","author":"R Cohen-Almagor","year":"2011","unstructured":"Cohen-Almagor R (2011) Fighting hate and bigotry on the internet. Policy Int 3(3):89\u2013114. https:\/\/doi.org\/10.2202\/1944-2866.1059","journal-title":"Policy Int"},{"key":"561_CR27","doi-asserted-by":"publisher","DOI":"10.1080\/13698230.2019.1576006","author":"K Gelber","year":"2019","unstructured":"Gelber K (2019) Differentiating hate speech: a systemic discrimination approach. Crit Rev Int Soc Pol Phil. https:\/\/doi.org\/10.1080\/13698230.2019.1576006","journal-title":"Crit Rev Int Soc Pol Phil"},{"issue":"S1","key":"561_CR28","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1111\/pops.12670","volume":"41","author":"M Bilewicz","year":"2020","unstructured":"Bilewicz M, Soral W (2020) Hate speech epidemic. The dynamic effects of derogatory language on intergroup relations and political radicalization. Polit Psychol 41(S1):3\u201333. https:\/\/doi.org\/10.1111\/pops.12670","journal-title":"Polit Psychol"},{"key":"561_CR29","doi-asserted-by":"crossref","unstructured":"Davidson T, Warmsley D, Macy M, Weber I (2017) Automated hate speech detection and the problem of offensive language. arXiv preprint http:\/\/arxiv.org\/abs\/1703.04009","DOI":"10.1609\/icwsm.v11i1.14955"},{"key":"561_CR30","unstructured":"Community standards, Facebook (2021) Available at https:\/\/www.facebook.com\/communitystandards\/hate_speech\/"},{"issue":"2","key":"561_CR31","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1177\/0163443720957562","volume":"43","author":"B Kalsnes","year":"2020","unstructured":"Kalsnes B, Ihleb\u00e6k KA (2020) Hiding hate speech: political moderation on Facebook. Media Cult Soc 43(2):326\u2013342. https:\/\/doi.org\/10.1177\/0163443720957562","journal-title":"Media Cult Soc"},{"key":"561_CR32","doi-asserted-by":"crossref","unstructured":"\u0160vec A, Pikuliak M, \u0160imko M, Bielikov\u00e1 M (2018) Improving moderation of online discussions via interpretable neural models. In: Proceedings of the second workshop on abusive language online\u2014ALW \u201918 at EMNLP, 2015, pp 60\u201365","DOI":"10.18653\/v1\/W18-5108"},{"key":"561_CR33","volume-title":"Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media","author":"T Gillespie","year":"2018","unstructured":"Gillespie T (2018) Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media. Yale University Press, New Haven"},{"issue":"1","key":"561_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1108\/AJIM-08-2019-0204","volume":"72","author":"AYK Chua","year":"2020","unstructured":"Chua AYK, Banerjee S (2020) The topic of terrorism on Yahoo! Answers: questions, answers and users\u2019 anonymity. Aslib J Inf Manag 72(1):1\u201316. https:\/\/doi.org\/10.1108\/AJIM-08-2019-0204","journal-title":"Aslib J Inf Manag"},{"key":"561_CR35","doi-asserted-by":"publisher","unstructured":"Guimar\u00e3es SS, Reis JCS, Ribeiro FN, Benevenuto F (2020) Characterizing toxicity on Facebook comments in Brazil. In: Proceedings of the Brazilian symposium on multimedia and the web (WebMedia '20). Association for Computing Machinery, New York, pp 253\u2013260. https:\/\/doi.org\/10.1145\/3428658.3430974","DOI":"10.1145\/3428658.3430974"},{"key":"561_CR36","doi-asserted-by":"crossref","unstructured":"Gao L, Huang R (2017) Detecting online hate speech using context aware models. http:\/\/arxiv.org\/abs\/1710.07395","DOI":"10.26615\/978-954-452-049-6_036"},{"key":"561_CR37","unstructured":"Wilson RA, Land MK (2021) Hate speech on social media: content moderation in context (September 10, 2020). Conn Law Rev 52:1029. Available at SSRN: https:\/\/ssrn.com\/abstract=3690616"},{"key":"561_CR38","doi-asserted-by":"publisher","DOI":"10.5539\/ass.v10n14p74","author":"T Tudkuea","year":"2014","unstructured":"Tudkuea T, Laeheem K (2014) Development of indicators of cyberbullying among youths in Songkhla Province. Asian Soc Sci. https:\/\/doi.org\/10.5539\/ass.v10n14p74","journal-title":"Asian Soc Sci"},{"issue":"2","key":"561_CR39","doi-asserted-by":"publisher","first-page":"180","DOI":"10.1177\/0002764219878224","volume":"65","author":"MD Molina","year":"2019","unstructured":"Molina MD, Sundar SS, Le T, Lee D (2019) \u201cFake News\u201d is not simply false information: a concept explication and taxonomy of online content. Am Behav Sci 65(2):180\u2013212. https:\/\/doi.org\/10.1177\/0002764219878224","journal-title":"Am Behav Sci"},{"issue":"3","key":"561_CR40","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1080\/19325037.2014.901113","volume":"45","author":"HK Hudson","year":"2014","unstructured":"Hudson HK, Fetro JV, Ogletree R (2014) Behavioral indicators and behaviors related to sexting among undergraduate students. Am J Health Educ 45(3):183\u2013195. https:\/\/doi.org\/10.1080\/19325037.2014.901113","journal-title":"Am J Health Educ"},{"key":"561_CR41","doi-asserted-by":"publisher","unstructured":"Ousidhoum N, Lin Z, Zhang H, Song Y, Yeung D (2019) Multilingual and multi-aspect hate speech analysis. IN: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 4667\u20134676. https:\/\/doi.org\/10.18653\/v1\/d19-1474","DOI":"10.18653\/v1\/d19-1474"},{"key":"561_CR42","doi-asserted-by":"publisher","unstructured":"Waseem Z (2016) Are you a racist or am I seeing things? Annotator influence on hate speech detection on Twitter. In: Proceedings of the first workshop on NLP and computational social science, pp 138\u2013142. https:\/\/doi.org\/10.18653\/v1\/w16-5618","DOI":"10.18653\/v1\/w16-5618"},{"key":"561_CR43","doi-asserted-by":"publisher","unstructured":"Waseem Z, Hovy D (2016) Hateful symbols or hateful people? Predictive features for hate speech detection on Twitter. In: Proceedings of the NAACL student research workshop. https:\/\/doi.org\/10.18653\/v1\/n16-2013","DOI":"10.18653\/v1\/n16-2013"},{"key":"561_CR44","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1016\/j.paid.2017.12.018","volume":"124","author":"D Azucar","year":"2018","unstructured":"Azucar D, Marengo D, Settanni M (2018) Predicting the big 5 personality traits from digital footprints on social media: a meta-analysis. Personal Individ Differ 124:150\u2013159","journal-title":"Personal Individ Differ"},{"key":"561_CR45","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.chb.2018.05.026","volume":"87","author":"AM Bliuc","year":"2018","unstructured":"Bliuc AM, Faulkner N, Jakubowicz A, McGarty C (2018) Online networks of racial hate: a systematic review of 10 years of research on cyber-racism. Comput Hum Behav 87:75\u201386","journal-title":"Comput Hum Behav"},{"key":"561_CR46","volume-title":"Communication Concepts 1: Explication","author":"SH Chaffee","year":"1991","unstructured":"Chaffee SH (1991) Communication Concepts 1: Explication. Sage Publications, Newbury Park, CA"},{"key":"561_CR47","volume-title":"Focus groups: A Practical Guide for Applied Research","author":"RA Krueger","year":"2009","unstructured":"Krueger RA, Casey MA (2009) Focus groups: A Practical Guide for Applied Research. Sage Publications, Thousand Oaks, CA"},{"key":"561_CR48","doi-asserted-by":"publisher","DOI":"10.4135\/9781483328171","volume-title":"Planning focus groups","author":"DL Morgan","year":"1998","unstructured":"Morgan DL, Krueger RA, Scannell AU (1998) Planning focus groups. Sage"},{"key":"561_CR49","unstructured":"EU Commission (2020) Countering illegal hate speech online. 5th evaluation of the Code of Conduct. Retrieved from https:\/\/ec.europa.eu\/info\/sites\/default\/files\/codeofconduct_2020_factsheet_12.pdf"},{"key":"561_CR50","unstructured":"The UN Refugee Agency (2020) Countering toxic narratives about refugees and migrants. Retrieved from: https:\/\/www.unhcr.org\/"},{"key":"561_CR51","unstructured":"CoE-ECRI\u2014Council of Europe\u2014European Commission against Racism and Intolerance: ECRI Report on the Slovak Republic (sixth monitoring cycle); Adopted on 1 October 2020; Published on 8 December 2020, 8 December 2020"},{"key":"561_CR52","unstructured":"World Bank (2019) The little data book on gender 2019. World Bank, Washington, DC. World Bank. https:\/\/openknowledge.worldbank.org\/handle\/10986\/31689"},{"issue":"3","key":"561_CR53","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1017\/s0033291719003404","volume":"50","author":"DJ Robinaugh","year":"2019","unstructured":"Robinaugh DJ, Hoekstra RHA, Toner ER, Borsboom D (2019) The network approach to psychopathology: a review of the literature 2008\u20132018 and an agenda for future research. Psychol Med 50(3):353\u2013366. https:\/\/doi.org\/10.1017\/s0033291719003404","journal-title":"Psychol Med"},{"issue":"1","key":"561_CR54","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1002\/wps.20375","volume":"16","author":"D Borsboom","year":"2017","unstructured":"Borsboom D (2017) A network theory of mental disorders. World Psychiatry 16(1):5\u201313. https:\/\/doi.org\/10.1002\/wps.20375","journal-title":"World Psychiatry"},{"issue":"2","key":"561_CR55","doi-asserted-by":"publisher","first-page":"58","DOI":"10.5204\/ijcjsd.v7i2.500","volume":"7","author":"V Pejchal","year":"2018","unstructured":"Pejchal V (2018) Hate speech regulation in post-communist countries: migrant crises in the Czech and Slovak republics. Int J Crime Justice Soc Democr 7(2):58. https:\/\/doi.org\/10.5204\/ijcjsd.v7i2.500","journal-title":"Int J Crime Justice Soc Democr"},{"key":"561_CR56","doi-asserted-by":"publisher","unstructured":"Golbeck J, Ashktorab Z, Banjo RO, Berlinger A, Bhagwan S, Buntain C, Cheakalos P, Geller AA, Gergory Q, Gnanasekaran RK, Gunasekaran RR, Hoffman KM, Hottle J, Jienjitlert V, Khare S, Lau R, Martindale MJ, Naik S, Nixon HL, Ramachandran P, Rogers KM, Rogers L, Sarin MS, Shahane G, Thanki J, Vengataraman P, Wan Z, Wu DM (2017) A large labeled corpus for online harassment research. In: Proceedings of the 2017 ACM on web science conference (WebSci '17). Association for Computing Machinery, New York, pp 229\u2013233. https:\/\/doi.org\/10.1145\/3091478.3091509","DOI":"10.1145\/3091478.3091509"},{"key":"561_CR57","doi-asserted-by":"publisher","unstructured":"Basile V, Bosco C, Fersini E, Nozza D, Patti V, Pardo FMR, Rosso P, Sanguinetti M (2019). SemEval-2019 task 5: multilingual detection of hate speech against immigrants and women in Twitter. In: Proceedings of the 13th international workshop on semantic evaluation. association for computational linguistics, pp 54\u201363. https:\/\/doi.org\/10.18653\/v1\/S19-2007","DOI":"10.18653\/v1\/S19-2007"},{"key":"561_CR58","unstructured":"Kumar R, Ojha AK, Malmasi S, Zampieri M (2018). Benchmarking aggression identification in social media. In: Proceedings of the first workshop on trolling, aggression and cyberbullying (TRAC-2018). Association for Computational Linguistics. p 11. https:\/\/aclanthology.org\/W18-4401\/"},{"key":"561_CR59","doi-asserted-by":"crossref","unstructured":"Poletto F, Stranisci M, Sanguinetti M, Patti V, Bosco C (2017) Hate speech annotation: analysis of an Italian Twitter corpus. In: Proceedings of the Fourth Italian conference on computational linguistics (CLiC-it 2017). CEUR-WS, vol 2006. http:\/\/ceur-ws.org\/Vol-2006\/paper024.pdf","DOI":"10.4000\/books.aaccademia.2448"},{"key":"561_CR60","unstructured":"Sanguinetti M, Poletto F, Bosco C, Patti V, Stranisci M (2018) An Italian Twitter corpus of hate speech against immigrants. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018). European Language Resources Association (ELRA). https:\/\/aclanthology.org\/L18-1443"},{"key":"561_CR61","doi-asserted-by":"publisher","unstructured":"Chung Y-L, Kuzmenko E, Tekiroglu SS, Guerini M (2019) CONAN\u2014COunter NArratives through Nichesourcing: a multilingual dataset of responses to fight online hate speech. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 2819\u20132829. Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/P19-1271","DOI":"10.18653\/v1\/P19-1271"},{"key":"561_CR62","unstructured":"Bosco C, Felice DO, Poletto F, Sanguinetti M, Maurizio T (2018) Overview of the evalita 2018 hate speech detection task. In: EVALITA 2018-sixth evaluation campaign of natural language processing and speech tools for Italian, vol 2263, pp 1\u20139. CEUR. https:\/\/iris.unito.it\/retrieve\/handle\/2318\/1686264\/465071\/paper010.pdf"},{"key":"561_CR63","doi-asserted-by":"crossref","unstructured":"Da San Martino G, Barr\u00f3n-Cede\u00f1o A, Wachsmuth H, Petrov R, Nakov P (2020). SemEval-2020 Task 11: detection of propaganda techniques in news articles. In: Proceedings of the fourteenth workshop on semantic evaluation, pp 1377\u20131414. International Committee for Computational Linguistics. https:\/\/aclanthology.org\/2020.semeval-1.186","DOI":"10.18653\/v1\/2020.semeval-1.186"},{"key":"561_CR64","doi-asserted-by":"publisher","unstructured":"Dimitrov D, Ali BB, Shaar S, Alam F, Silvestri F, Firooz H, Nakov P, Da San Martino G (2021) SemEval-2021 task 6: detection of persuasion techniques in texts and images. In: Proceedings of the 15th international workshop on semantic evaluation (SemEval-2021), pp 70\u201398. https:\/\/doi.org\/10.18653\/v1\/2021.semeval-1.7","DOI":"10.18653\/v1\/2021.semeval-1.7"},{"key":"561_CR65","doi-asserted-by":"publisher","unstructured":"Van Hee C, Lefever E, Hoste V (2018) SemEval-2018 task 3: irony detection in English tweets. In: Proceedings of the 12th international workshop on semantic evaluation. Association for Computational Linguistics, pp 39\u201350. https:\/\/doi.org\/10.18653\/v1\/S18-1005","DOI":"10.18653\/v1\/S18-1005"},{"key":"561_CR66","doi-asserted-by":"crossref","unstructured":"Meaney JA, Wilson SR, Chiruzzo L, Lopez A, Magdy W (2021). SemEval-2021 Task 7: HaHackathon, detecting and rating humor and offense. In: Proceedings of the 15th international workshop on semantic evaluation (SemEval-2021). Association for Computational Linguistics, pp 105\u2013119. https:\/\/aclanthology.org\/2021.semeval-1.9.pdf","DOI":"10.18653\/v1\/2021.semeval-1.9"},{"issue":"sup1","key":"561_CR67","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1080\/00031305.2018.1529623","volume":"73","author":"NC Brownstein","year":"2019","unstructured":"Brownstein NC, Louis TA, O\u2019Hagan A, Pendergast J (2019) The role of expert judgment in statistical inference and evidence-based decision-making. Am Stat 73(sup1):56\u201368. https:\/\/doi.org\/10.1080\/00031305.2018.1529623","journal-title":"Am Stat"},{"issue":"3","key":"561_CR68","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1108\/00251740910946714","volume":"47","author":"MF Cassidy","year":"2009","unstructured":"Cassidy MF, Buede D (2009) Does the accuracy of expert judgment comply with common sense:caveat emptor. Manag Decis 47(3):454\u2013469. https:\/\/doi.org\/10.1108\/00251740910946714","journal-title":"Manag Decis"},{"issue":"6","key":"561_CR69","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1177\/1745691617693393","volume":"12","author":"T Yarkoni","year":"2017","unstructured":"Yarkoni T, Westfall J (2017) Choosing prediction over explanation in psychology: lessons from machine learning. Perspect Psychol Sci 12(6):1100\u20131122. https:\/\/doi.org\/10.1177\/1745691617693393","journal-title":"Perspect Psychol Sci"},{"issue":"2","key":"561_CR70","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","volume":"86","author":"PE Shrout","year":"1979","unstructured":"Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86(2):420\u2013428. https:\/\/doi.org\/10.1037\/0033-2909.86.2.420","journal-title":"Psychol Bull"},{"key":"561_CR71","unstructured":"OSCE (2020) Annual report 2019. Retrieved from: http:\/\/osr.org"},{"issue":"4","key":"561_CR72","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1080\/1047840x.2020.1853461","volume":"31","author":"EI Fried","year":"2020","unstructured":"Fried EI (2020) Lack of theory building and testing impedes progress in the factor and network literature. Psychol Inq 31(4):271\u2013288. https:\/\/doi.org\/10.1080\/1047840x.2020.1853461","journal-title":"Psychol Inq"},{"key":"561_CR73","doi-asserted-by":"crossref","unstructured":"Fersini E, Nozza D, Rosso P (2018a) Overview of the Evalita 2018 task on automatic misogyny identification (AMI). EVALITA evaluation of NLP and speech tools for Italian, vol 12. p 59. https:\/\/boa.unimib.it\/retrieve\/handle\/10281\/219426\/452274\/10281-219426.pdf#page=68","DOI":"10.4000\/books.aaccademia.4497"},{"key":"561_CR74","doi-asserted-by":"crossref","unstructured":"Fersini E, Rosso P, Anzovino M (2018b) Overview of the task on automatic misogyny identification at IberEval 2018. In: Proceedings of the third workshop on evaluation of human language technologies for Iberian languages (IberEval 2018) co-located with 34th conference of the spanish society for natural language processing (SEPLN 2018). CEUR-WS. http:\/\/ceur-ws.org\/Vol-2150\/overview-AMI.pdf","DOI":"10.4000\/books.aaccademia.4497"},{"key":"561_CR75","doi-asserted-by":"publisher","unstructured":"Habernal I, Wachsmuth H, Gurevych I, Stein B (2018). Before name-calling: dynamics and triggers of ad hominem fallacies in web argumentation. In: Proceedings of the 2018 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, volume 1 (long papers), pp 386\u2013396. Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/N18-1036","DOI":"10.18653\/v1\/N18-1036"},{"key":"561_CR76","doi-asserted-by":"crossref","unstructured":"Rosenthal S, Atanasova P, Karadzhov G, Zampieri M, Nakov P (2021) A large-scale semi-supervised dataset for offensive language identification. arXiv preprint http:\/\/arxiv.org\/abs\/2004.14454","DOI":"10.18653\/v1\/2021.findings-acl.80"},{"key":"561_CR77","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/j.jrp.2014.07.003","volume":"54","author":"G Constantini","year":"2015","unstructured":"Constantini G, Epskamp S, Borsboom D, Perugini M, M\u00f5ttus R, Waldorp LJ, Cramer AO (2015) State of the aRt personality research: a tutorial on network analysis of personality data in R. J Res Pers 54:13\u201329. https:\/\/doi.org\/10.1016\/j.jrp.2014.07.003","journal-title":"J Res Pers"},{"issue":"6","key":"561_CR78","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1037\/abn0000181","volume":"125","author":"DJ Robinaugh","year":"2016","unstructured":"Robinaugh DJ, Millner AJ, McNally RJ (2016) Identifying highly influential nodes in the complicated grief network. J Abnorm Psychol 125(6):747. https:\/\/doi.org\/10.1037\/abn0000181","journal-title":"J Abnorm Psychol"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00561-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-021-00561-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-021-00561-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,9]],"date-time":"2023-06-09T17:35:32Z","timestamp":1686332132000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-021-00561-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,11]]},"references-count":78,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["561"],"URL":"https:\/\/doi.org\/10.1007\/s40747-021-00561-0","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,11]]},"assertion":[{"value":"25 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None declared.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}