{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T17:12:26Z","timestamp":1778346746249,"version":"3.51.4"},"reference-count":94,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2020,11,16]],"date-time":"2020-11-16T00:00:00Z","timestamp":1605484800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,11,16]],"date-time":"2020-11-16T00:00:00Z","timestamp":1605484800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003132","name":"Agentschap voor Innovatie door Wetenschap en Technologie","doi-asserted-by":"publisher","award":["120007"],"award-info":[{"award-number":["120007"]}],"id":[{"id":"10.13039\/501100003132","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007659","name":"Tilburg University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007659","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Lang Resources &amp; Evaluation"],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The detection of online cyberbullying has seen an increase in societal importance, popularity in research, and available open data. Nevertheless, while computational power and affordability of resources continue to increase, the access restrictions on high-quality data limit the applicability of state-of-the-art techniques. Consequently, much of the recent research uses small, heterogeneous datasets, without a thorough evaluation of applicability. In this paper, we further illustrate these issues, as we (i) evaluate many publicly available resources for this task and demonstrate difficulties with data collection. These predominantly yield small datasets that fail to capture the required complex social dynamics and impede direct comparison of progress. We (ii) conduct an extensive set of experiments that indicate a general lack of cross-domain generalization of classifiers trained on these sources, and openly provide this framework to replicate and extend our evaluation criteria. Finally, we (iii) present an effective crowdsourcing method: simulating real-life bullying scenarios in a lab setting generates plausible data that can be effectively used to enrich real data. This largely circumvents the restrictions on data that can be collected, and increases classifier performance. We believe these contributions can aid in improving the empirical practices of future research in the field.<\/jats:p>","DOI":"10.1007\/s10579-020-09509-1","type":"journal-article","created":{"date-parts":[[2020,11,16]],"date-time":"2020-11-16T07:03:35Z","timestamp":1605510215000},"page":"597-633","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["Current limitations in cyberbullying detection: On evaluation criteria, reproducibility, and data scarcity"],"prefix":"10.1007","volume":"55","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2179-559X","authenticated-orcid":false,"given":"Chris","family":"Emmery","sequence":"first","affiliation":[]},{"given":"Ben","family":"Verhoeven","sequence":"additional","affiliation":[]},{"given":"Guy","family":"De Pauw","sequence":"additional","affiliation":[]},{"given":"Gilles","family":"Jacobs","sequence":"additional","affiliation":[]},{"given":"Cynthia","family":"Van Hee","sequence":"additional","affiliation":[]},{"given":"Els","family":"Lefever","sequence":"additional","affiliation":[]},{"given":"Bart","family":"Desmet","sequence":"additional","affiliation":[]},{"given":"V\u00e9ronique","family":"Hoste","sequence":"additional","affiliation":[]},{"given":"Walter","family":"Daelemans","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,11,16]]},"reference":[{"key":"9509_CR1","unstructured":"Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat, S., Goodfellow, I., Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Man\u00e9, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Vi\u00e9gas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., & Zheng, X. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. http:\/\/tensorflow.org\/. Software available from tensorflow.org."},{"key":"9509_CR2","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1007\/978-3-319-76941-7_11","volume-title":"Advances in information retrieval","author":"S Agrawal","year":"2018","unstructured":"Agrawal, S., & Awekar, A. (2018). Deep learning for detecting cyberbullying across multiple social media platforms. In G. Pasi, B. Piwowarski, L. Azzopardi, & A. Hanbury (Eds.), Advances in information retrieval (pp. 141\u2013153). Cham: Springer International Publishing."},{"issue":"11","key":"9509_CR3","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1093\/bioinformatics\/15.11.937","volume":"15","author":"P Baldi","year":"1999","unstructured":"Baldi, P., Brunak, S., Frasconi, P., Soda, G., & Pollastri, G. (1999). Exploiting the past and the future in protein secondary structure prediction. Bioinformatics, 15(11), 937\u2013946.","journal-title":"Bioinformatics"},{"key":"9509_CR4","unstructured":"Bayzick, J., Kontostathis, A., & Edwards, L. (2011). Detecting the presence of cyberbullying using computer software. In Proceedings of the 3rd international web science conference. WebSci11; 2011."},{"issue":"2","key":"9509_CR5","doi-asserted-by":"publisher","first-page":"16","DOI":"10.21913\/JSW.v1i2.172","volume":"1","author":"T Beran","year":"2008","unstructured":"Beran, T., & Li, Q. (2008). The relationship between cyberbullying and school bullying. The Journal of Student Wellbeing, 1(2), 16\u201333.","journal-title":"The Journal of Student Wellbeing"},{"issue":"1","key":"9509_CR6","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1080\/13691180701858851","volume":"11","author":"K Bessiere","year":"2008","unstructured":"Bessiere, K., Kiesler, S., Kraut, R., & Boneva, B. S. (2008). Effects of internet use and social resources on changes in depression. Information, Community & Society, 11(1), 47\u201370.","journal-title":"Information, Community & Society"},{"key":"9509_CR7","unstructured":"Binns, A. (2013). Facebook\u2019s ugly sisters: Anonymity and abuse on formspring and ask. fm. Media Education Research Journal"},{"key":"9509_CR8","unstructured":"Blitzer, J., Dredze, M., & Pereira, F. (2007). Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of the 45th annual meeting of the association of computational linguistics (pp. 440\u2013447)."},{"issue":"1","key":"9509_CR9","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.csl.2013.04.007","volume":"28","author":"D Bogdanova","year":"2014","unstructured":"Bogdanova, D., Rosso, P., & Solorio, T. (2014). Exploring high-level features for detecting cyberpedophilia. Computer Speech & Language, 28(1), 108\u2013120.","journal-title":"Computer Speech & Language"},{"key":"9509_CR10","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","volume":"5","author":"P Bojanowski","year":"2017","unstructured":"Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135\u2013146.","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"9509_CR11","unstructured":"Bretschneider, U., W\u00f6hner, T., & Peters, R. (2014). Detecting Online Harassment in Social Networks. In 35th International Conference on Information Systems Li 2007 (pp. 1\u201314)."},{"key":"9509_CR12","unstructured":"Broadcasting Standards Authority: What not to swear: The acceptability of words in broadcasting. Retrieved March 03, 2016 from, http:\/\/bsa.govt.nz\/images\/assets\/Research\/Acceptability_of_Words_2013_WEB.pdf (2013)."},{"key":"9509_CR13","unstructured":"Broeck, E., Poels, K., Vandebosch, H., & Royen, K. (2014). Online perspective-taking as an intervention tool against cyberbullying. Annual review of cybertherapy and telemedicine (pp. 113\u2013117)."},{"key":"9509_CR14","doi-asserted-by":"crossref","unstructured":"Chan, Y.S., & Ng, H.T. (2006). Estimating class priors in domain adaptation for word sense disambiguation. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics (pp. 89\u201396). Association for Computational Linguistics.","DOI":"10.3115\/1220175.1220187"},{"key":"9509_CR15","unstructured":"Chen, M., Xu, Z., Weinberger, K., & Sha, F. (2012). Marginalized denoising autoencoders for domain adaptation. arXiv preprint arXiv:1206.4683."},{"key":"9509_CR16","doi-asserted-by":"crossref","unstructured":"Cheng, L., Guo, R., Candan, K.S., & Liu, H. (2020). Representation learning for imbalanced cross-domain classification. In Proceedings of the 2020 SIAM international conference on data mining. (pp. 478\u2013486). SIAM.","DOI":"10.1137\/1.9781611976236.54"},{"key":"9509_CR17","doi-asserted-by":"crossref","unstructured":"Cheng, L., Guo, R., Silva, Y., Hall, D., & Liu, H. (2019). Hierarchical attention networks for cyberbullying detection on the instagram social network. In Proceedings of the 2019 SIAM international conference on data mining (pp. 235\u2013243). SIAM.","DOI":"10.1137\/1.9781611975673.27"},{"key":"9509_CR18","unstructured":"Chollet, F., et\u00a0al. (2015). Keras. https:\/\/keras.io."},{"key":"9509_CR19","first-page":"2493","volume":"12","author":"R Collobert","year":"2011","unstructured":"Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12, 2493\u20132537.","journal-title":"Journal of Machine Learning Research"},{"issue":"3","key":"9509_CR20","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273\u2013297. https:\/\/doi.org\/10.1007\/BF00994018.","journal-title":"Machine Learning"},{"key":"9509_CR21","doi-asserted-by":"publisher","unstructured":"Dadvar, M. (2014). Experts and machines united against cyberbullying. Ph.D. thesis, University of Twente, Netherlands. https:\/\/doi.org\/10.3990\/1.9789036537391.","DOI":"10.3990\/1.9789036537391"},{"key":"9509_CR22","doi-asserted-by":"crossref","unstructured":"Dadvar, M., Trieschnigg, D., de\u00a0Jong, F. (2014). Experts and machines against bullies: A hybrid approach to detect cyberbullies. In Canadian conference on artificial intelligence (pp. 275\u2013281). Springer.","DOI":"10.1007\/978-3-319-06483-3_25"},{"key":"9509_CR23","unstructured":"Daum\u00e9\u00a0III, H. (2009). Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815."},{"issue":"2","key":"9509_CR24","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1089\/cpb.2007.0008","volume":"11","author":"F DeHue","year":"2008","unstructured":"DeHue, F., Bolman, C., & V\u00f6llink, T. (2008). Cyberbullying: Youngsters\u2019 experiences and parental perception. CyberPsychology & Behavior, 11(2), 217\u2013223.","journal-title":"CyberPsychology & Behavior"},{"key":"9509_CR25","unstructured":"Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805."},{"key":"9509_CR26","unstructured":"Dinakar, K., Reichart, R., & Lieberman, H. (2011). Modeling the detection of textual cyberbullying. the social mobile web. In 5th international AAAI conference on weblogs and social media, Barcelona, Catalonia, Spain."},{"key":"9509_CR27","volume-title":"Mobile messaging and social media 2015","author":"M Duggan","year":"2015","unstructured":"Duggan, M. (2015). Mobile messaging and social media 2015. Washington, D.C: Pew Research Center."},{"key":"9509_CR28","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.diin.2016.07.001","volume":"18","author":"M Ebrahimi","year":"2016","unstructured":"Ebrahimi, M., Suen, C. Y., & Ormandjieva, O. (2016). Detecting predatory conversations in social media by deep convolutional neural networks. Digital Investigation, 18, 33\u201349.","journal-title":"Digital Investigation"},{"key":"9509_CR29","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.eswa.2017.07.040","volume":"89","author":"HJ Escalante","year":"2017","unstructured":"Escalante, H. J., Villatoro-Tello, E., Garza, S. E., L\u00f3pez-Monroy, A. P., Montes-y G\u00f3mez, M., & Villase\u00f1or-Pineda, L. (2017). Early detection of deception and aggressiveness using profile-based representations. Expert Systems with Applications, 89, 99\u2013111.","journal-title":"Expert Systems with Applications"},{"key":"9509_CR30","doi-asserted-by":"publisher","unstructured":"Fan, R. E., Chang, K. W., Hsieh, C. J., Wang, X. R., & Lin, C. J. (2008). LIBLINEAR: A Library for Large Linear Classification. The Journal of Machine Learning, 9(2008), 1871\u20131874. https:\/\/doi.org\/10.1038\/oby.2011.351. http:\/\/dl.acm.org\/citation.cfm?id=1442794.","DOI":"10.1038\/oby.2011.351"},{"key":"9509_CR31","unstructured":"Feng, S., Banerjee, R., & Choi, Y. (2012). Syntactic stylometry for deception detection. In Proceedings of the 50th annual meeting of the association for computational linguistics (Volume 2: Short Papers) (pp. 171\u2013175)."},{"key":"9509_CR32","doi-asserted-by":"crossref","unstructured":"Finkel, J.R., & Manning, C.D. (2009). Hierarchical bayesian domain adaptation. In Proceedings of human language technologies: The 2009 annual conference of the North American Chapter of the Association for Computational Linguistics (pp. 602\u2013610).","DOI":"10.3115\/1620754.1620842"},{"key":"9509_CR33","first-page":"115","volume":"3","author":"FA Gers","year":"2002","unstructured":"Gers, F. A., Schraudolph, N. N., & Schmidhuber, J. (2002). Learning precise timing with lstm recurrent networks. Journal of Machine Learning Research, 3, 115\u2013143.","journal-title":"Journal of Machine Learning Research"},{"key":"9509_CR34","unstructured":"Glorot, X., Bordes, A., & Bengio, Y. (2011). Domain adaptation for large-scale sentiment classification: A deep learning approach."},{"key":"9509_CR35","unstructured":"Hee, C.V., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., Pauw, G.D., & Daelemans, W. (2015). Detection and Fine-Grained Classification of Cyberbullying Events. In International conference recent advances in natural language processing (RANLP) (pp. 672\u2013680)."},{"issue":"8","key":"9509_CR36","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735\u20131780.","journal-title":"Neural Computation"},{"key":"9509_CR37","unstructured":"Honnibal, M., & Montani, I. (2017). spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear."},{"key":"9509_CR38","doi-asserted-by":"crossref","unstructured":"Hosseinmardi, H., Mattson, S.A., Rafiq, R.I., Han, R., Lv, Q., Mishra, S. (2015). Prediction of cyberbullying incidents on the Instagram social network. In MobiSys, (p. 2014). arxiv:1503.03909.","DOI":"10.1007\/978-3-319-27433-1_4"},{"key":"9509_CR39","doi-asserted-by":"crossref","unstructured":"Jiang, J., & Zhai, C. (2007). A two-stage approach to domain adaptation for statistical classifiers. In Proceedings of the sixteenth ACM conference on Conference on information and knowledge management (pp. 401\u2013410).","DOI":"10.1145\/1321440.1321498"},{"key":"9509_CR40","doi-asserted-by":"crossref","unstructured":"Kaati, L., Omer, E., Prucha, N., & Shrestha, A. (2015). Detecting multipliers of jihadism on twitter. In 2015 IEEE international conference on data mining workshop (ICDMW) (pp. 954\u2013960). IEEE.","DOI":"10.1109\/ICDMW.2015.9"},{"issue":"Suppl 4","key":"9509_CR41","first-page":"1","volume":"63","author":"L Kann","year":"2014","unstructured":"Kann, L., Kinchen, S., Shanklin, S. L., Flint, K. H., Kawkins, J., Harris, W. A., et al. (2014). Youth risk behavior surveillance-united states, 2013. Morbidity and Mortality Weekly Report: Surveillance Summaries, 63(Suppl 4), 1\u2013168.","journal-title":"Morbidity and Mortality Weekly Report: Surveillance Summaries"},{"key":"9509_CR42","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882.","DOI":"10.3115\/v1\/D14-1181"},{"key":"9509_CR43","doi-asserted-by":"crossref","unstructured":"Kontostathis, A., Edwards, L., & Leatherman, A. (2010). Text mining and cybercrime. Text mining: Applications and theory. John Wiley & Sons, Ltd, Chichester, UK (pp. 149\u2013164).","DOI":"10.1002\/9780470689646.ch8"},{"key":"9509_CR44","doi-asserted-by":"crossref","unstructured":"Kontostathis, A., Reynolds, K. (2013). Detecting cyberbullying: Query terms and techniques. In Proceedings of the 3rd annual ACM web science conference, WebSci 2013 (2013) (pp. 195\u2013204).","DOI":"10.1145\/2464464.2464499"},{"key":"9509_CR45","unstructured":"Kowalski, R. M., Limber, S. P., & Agatston, P. W. (2012). Cyberbullying: Bullying in the digital age. John Wiley & Sons."},{"key":"9509_CR46","unstructured":"Lenhart, A., Madden, M., Smith, A., Purcell, K., Zickuhr, K., & Rainie, L. (2011). Teens, kindness and cruelty on social network sites: How american teens navigate the new world of. Pew Internet & American Life Project."},{"key":"9509_CR47","unstructured":"Li, S., Ju, S., Zhou, G., & Li, X. (2012). Active learning for imbalanced sentiment classification. In Proceedings of the 2012 Joint conference on empirical methods in natural language processing and computational natural language learning (pp. 139\u2013148). Association for Computational Linguistics."},{"key":"9509_CR48","unstructured":"Livingstone, S., Haddon, L., G\u00f6rzig, A., & \u00d3lafsson, K. (2011). EU Kids Online II: Final Report 2011. Tech. rep., EU Kids Online. http:\/\/www.lse.ac.uk\/collections\/EUKidsOnline\/ (p. 293)."},{"key":"9509_CR49","doi-asserted-by":"crossref","unstructured":"Ma, J., Zhang, Y., & Zhu, J. (2014). Tagging the web: Building a robust web tagger with neural network. In Proceedings of the 52nd annual meeting of the association for computational linguistics (Vol. 1: Long Papers) (pp. 144\u2013154).","DOI":"10.3115\/v1\/P14-1014"},{"key":"9509_CR50","unstructured":"Madden, M., Lenhart, A., Cortesi, S. (2013). Teens, social media, and privacy. Tech. rep., Pew Internet. http:\/\/www.lateledipenelope.it\/public\/52dff2e35b812.pdf."},{"key":"9509_CR51","doi-asserted-by":"crossref","unstructured":"McClosky, D., Charniak, E., Johnson, M.: Reranking and self-training for parser adaptation. In Proceedings of the 21st international conference on computational linguistics and the 44th annual meeting of the Association for Computational Linguistics (pp. 337\u2013344). Association for Computational Linguistics.","DOI":"10.3115\/1220175.1220218"},{"key":"9509_CR52","unstructured":"Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781."},{"key":"9509_CR53","unstructured":"Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111\u20133119)."},{"key":"9509_CR54","volume-title":"Delete expletives? research undertaken jointly by the advertising standards authority, british broadcasting corporation, broadcasting standards commission and the independent television commission","author":"A Millwood-Hargreave","year":"2000","unstructured":"Millwood-Hargreave, A. (2000). Delete expletives? research undertaken jointly by the advertising standards authority, british broadcasting corporation, broadcasting standards commission and the independent television commission. London: ASA, BBC, BSC and ITC."},{"issue":"May","key":"9509_CR55","first-page":"238","volume":"3","author":"V Nahar","year":"2013","unstructured":"Nahar, V., Li, X., & Pang, C. (2013). An effective approach for cyberbullying detection. Communications in Information Science and Management Engineering, 3(May), 238\u2013247.","journal-title":"Communications in Information Science and Management Engineering"},{"key":"9509_CR56","unstructured":"Noswearing.com: Bad Word List & Swear Filter. Retrieved March 03, 2016 from, http:\/\/www.noswearing.com (2016)."},{"key":"9509_CR57","unstructured":"\u00d3lafsson, K., Livingstone, S., & Haddon, L. (2013). Children\u2019s Use of Online Technologies in Europe: A review of the European evidence base. EU Kids Online: Tech. Rep. May."},{"key":"9509_CR58","doi-asserted-by":"crossref","unstructured":"Pan, S.J., Ni, X., Sun, J.T., Yang, Q., & Chen, Z. (2010). Cross-domain sentiment classification via spectral feature alignment. In Proceedings of the 19th international conference on World wide web (pp. 751\u2013760).","DOI":"10.1145\/1772690.1772767"},{"key":"9509_CR59","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825\u20132830.","journal-title":"Journal of Machine Learning Research"},{"key":"9509_CR60","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., & Manning, C.D. (2014). Glove: Global vectors for word representation. In Empirical methods in natural language processing (EMNLP) (pp. 1532\u20131543). http:\/\/www.aclweb.org\/anthology\/D14-1162.","DOI":"10.3115\/v1\/D14-1162"},{"issue":"4","key":"9509_CR61","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1089\/cpb.2009.0025","volume":"12","author":"C Privitera","year":"2009","unstructured":"Privitera, C., & Campbell, M. A. (2009). Cyberbullying: The new face of workplace bullying? CyberPsychology & Behavior, 12(4), 395\u2013400.","journal-title":"CyberPsychology & Behavior"},{"key":"9509_CR62","unstructured":"\u0158eh\u016f\u0159ek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks (pp. 45\u201350). ELRA, Valletta, Malta (2010). http:\/\/is.muni.cz\/publication\/884893\/en."},{"key":"9509_CR63","doi-asserted-by":"crossref","unstructured":"Reynolds, K., Kontostathis, A., & Edwards, L. (2011). Using machine learning to detect cyberbullying. In 10th international conference on machine learning and applications and workshops (2011) (pp. 241\u2013244).","DOI":"10.1109\/ICMLA.2011.152"},{"key":"9509_CR64","unstructured":"Robers, S., Zhang, A., & Morgan, R.E. (2015). Indicators of school crime and safety: 2014. nces 2015-072\/ncj 248036. National Center for Education Statistics."},{"key":"9509_CR65","doi-asserted-by":"crossref","unstructured":"Rosa, H., Carvalho, J.P., Calado, P., Martins, B., Ribeiro, R., & Coheur, L. (2018). Using fuzzy fingerprints for cyberbullying detection in social networks. In 2018 IEEE international conference on fuzzy systems (FUZZ-IEEE) (pp. 1\u20137). IEEE.","DOI":"10.1109\/FUZZ-IEEE.2018.8491557"},{"key":"9509_CR66","doi-asserted-by":"crossref","unstructured":"Rosa, H., Matos, D., Ribeiro, R., Coheur, L., & Carvalho, J.P. (2018). A \u201cdeeper\u201d look at detecting cyberbullying in social networks. In 2018 international joint conference on neural networks (IJCNN) (pp. 1\u20138). IEEE.","DOI":"10.1109\/IJCNN.2018.8489211"},{"key":"9509_CR67","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1016\/j.chb.2018.12.021","volume":"93","author":"H Rosa","year":"2019","unstructured":"Rosa, H., Pereira, N., Ribeiro, R., Ferreira, P. C., Carvalho, J. P., Oliveira, S., et al. (2019). Automatic cyberbullying detection: A systematic review. Computers in Human Behavior, 93, 333\u2013345.","journal-title":"Computers in Human Behavior"},{"key":"9509_CR68","unstructured":"Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. In NeurIPS EMC2 Workshop."},{"key":"9509_CR69","unstructured":"Sch\u00e4fer, R., & Bildhauer, F. (2012). Building large corpora from the web using a new efficient tool chain. In LREC, (pp. 486\u2013493)."},{"key":"9509_CR70","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1162\/tacl_a_00162","volume":"2","author":"T Schnabel","year":"2014","unstructured":"Schnabel, T., & Sch\u00fctze, H. (2014). Flors: Fast and simple domain adaptation for part-of-speech tagging. Transactions of the Association for Computational Linguistics, 2, 15\u201326.","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"11","key":"9509_CR71","doi-asserted-by":"publisher","first-page":"2673","DOI":"10.1109\/78.650093","volume":"45","author":"M Schuster","year":"1997","unstructured":"Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673\u20132681.","journal-title":"IEEE Transactions on Signal Processing"},{"key":"9509_CR72","doi-asserted-by":"crossref","unstructured":"Sharp, S., & Smith, P. K. (2002). School bullying: Insights and perspectives. : Routledge.","DOI":"10.4324\/9780203425497"},{"key":"9509_CR73","volume-title":"Social media use in 2018","author":"A Smith","year":"2018","unstructured":"Smith, A., & Anderson, M. (2018). Social media use in 2018. Washington, D.C.: Pew Research Center."},{"key":"9509_CR74","doi-asserted-by":"crossref","unstructured":"Squicciarini, A., Rajtmajer, S., Liu, Y., & Griffin, C. (2015). Identification and characterization of cyberbullying dynamics in an online social network. In Proceedings of the 2015 IEEE\/ACM international conference on advances in social networks analysis and mining 2015 (pp. 280\u2013285). ACM.","DOI":"10.1145\/2808797.2809398"},{"issue":"8","key":"9509_CR75","doi-asserted-by":"publisher","first-page":"582","DOI":"10.1089\/cyber.2012.0392","volume":"16","author":"WM Steijn","year":"2013","unstructured":"Steijn, W. M., & Schouten, A. P. (2013). Information sharing and relationships on social networking sites. Cyberpsychology, Behavior, and Social Networking, 16(8), 582\u2013587.","journal-title":"Cyberpsychology, Behavior, and Social Networking"},{"key":"9509_CR76","unstructured":"Sui, J. (2015). Understanding and fighting bullying with machine learning. Ph.D. thesis, The University of Wisconsin-Madison."},{"key":"9509_CR77","doi-asserted-by":"crossref","unstructured":"Sun, C., Qiu, X., Xu, Y., & Huang, X.(2019). How to fine-tune bert for text classification? In China national conference on Chinese computational linguistics (pp. 194\u2013206). Springer.","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"9509_CR78","doi-asserted-by":"publisher","unstructured":"Thain, N., Dixon, L., & Wulczyn, E. (2017). Wikipedia Talk Labels: Toxicity,. https:\/\/doi.org\/10.6084\/m9.figshare.4563973.v2. https:\/\/figshare.com\/articles\/Wikipedia_Talk_Labels_Toxicity\/4563973.","DOI":"10.6084\/m9.figshare.4563973.v2"},{"key":"9509_CR79","doi-asserted-by":"crossref","unstructured":"Tomkins, S., Getoor, L., Chen, Y., & Zhang, Y. (2018). A socio-linguistic model for cyberbullying detection. In 2018 IEEE\/ACM international conference on advances in social networks analysis and mining (ASONAM) (pp. 53\u201360). IEEE.","DOI":"10.1109\/ASONAM.2018.8508294"},{"key":"9509_CR80","unstructured":"Tulkens, S., Emmery, C., & Daelemans, W. (2016). Evaluating unsupervised dutch word embeddings as a linguistic resource. In: N.C.C. Chair), K.\u00a0Choukri, T.\u00a0Declerck, M.\u00a0Grobelnik, B.\u00a0Maegaard, J.\u00a0Mariani, A.\u00a0Moreno, J.\u00a0Odijk, S.\u00a0Piperidis (eds.) Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016). European Language Resources Association (ELRA), Paris, France."},{"key":"9509_CR81","doi-asserted-by":"crossref","unstructured":"Vaez, M., Ekberg, K., & LaFlamme, L. (2004). Abusive events at work among young working adults: Magnitude of the problem and its effect on self-rated health. Relations industrielles\/industrial relations (pp. 569\u2013584).","DOI":"10.7202\/010925ar"},{"issue":"2","key":"9509_CR82","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1037\/0012-1649.43.2.267","volume":"43","author":"PM Valkenburg","year":"2007","unstructured":"Valkenburg, P. M., & Peter, J. (2007). Preadolescents\u2019 and adolescents\u2019 online communication and their closeness to friends. Developmental Psychology, 43(2), 267.","journal-title":"Developmental Psychology"},{"issue":"10","key":"9509_CR83","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0203794","volume":"13","author":"C Van Hee","year":"2018","unstructured":"Van Hee, C., Jacobs, G., Emmery, C., Desmet, B., Lefever, E., Verhoeven, B., et al. (2018). Automatic detection of cyberbullying in social media text. PLoS ONE, 13(10), 1\u201322. https:\/\/doi.org\/10.1371\/journal.pone.0203794.","journal-title":"PLoS ONE"},{"key":"9509_CR84","unstructured":"Van\u00a0Hee, C., Verhoeven, B., Lefever, E., De\u00a0Pauw, G., Daelemans, W., & Hoste, V. (2015). Guidelines for the fine-grained analysis of cyberbullying. Tech. rep., version 1.0. Technical Report LT3 15-01, LT3, Language and Translation Technology Team\u2014Ghent University."},{"key":"9509_CR85","doi-asserted-by":"crossref","unstructured":"Vilain, M., Su, J., Lubar, S. (2007). Entity extraction is a boring solved problem\u2014or is it? In Human language technologies 2007: The conference of the North American chapter of the Association for Computational Linguistics; Companion Volume, Short Papers (pp. 181\u2013184).","DOI":"10.3115\/1614108.1614154"},{"key":"9509_CR86","unstructured":"Wang, S., & Manning, C.D. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th annual meeting of the association for computational linguistics: Short papers-volume 2 (pp. 90\u201394). Association for Computational Linguistics."},{"key":"9509_CR87","unstructured":"Willard, N. E. (2007). Cyberbullying and cyberthreats: Responding to the challenge of online social aggression, threats, and distress. : Research Press."},{"key":"9509_CR88","doi-asserted-by":"crossref","unstructured":"Wulczyn, E., Thain, N., & Dixon, L. (2017). Ex machina: Personal attacks seen at scale. In Proceedings of the 26th international conference on World Wide Web (pp. 1391\u20131399).","DOI":"10.1145\/3038912.3052591"},{"key":"9509_CR89","unstructured":"Xu, J., Jun, K., Zhu, X., & Bellmore, A. (2012). Learning from Bullying Traces in Social Media. In Proceedings of the 2012 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 656\u2013666). Association for Computational Linguistics."},{"key":"9509_CR90","unstructured":"Yin, D., Xue, Z., Hong, L., Davison, B.D., Kontostathis, A., & Edwards, L. (2009). Detection of Harassment on Web 2.0. In Proceedings of the content analysis in the WEB 2.0 (CAW2.0) Workshop at WWW2009."},{"key":"9509_CR91","doi-asserted-by":"crossref","unstructured":"Zhang, X., Tong, J., Vishwamitra, N., Whittaker, E., Mazer, J.P., Kowalski, R., Hu, H., Luo, F., Macbeth, J., & Dillon, E. (2016). Cyberbullying detection with a pronunciation based convolutional neural network. In 2016 15th IEEE international conference on machine learning and applications (ICMLA) (pp. 740\u2013745). IEEE.","DOI":"10.1109\/ICMLA.2016.0132"},{"issue":"3","key":"9509_CR92","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1109\/TAFFC.2016.2531682","volume":"8","author":"R Zhao","year":"2016","unstructured":"Zhao, R., & Mao, K. (2016). Cyberbullying detection based on semantic-enhanced marginalized denoising auto-encoder. IEEE Transactions on Affective Computing, 8(3), 328\u2013339.","journal-title":"IEEE Transactions on Affective Computing"},{"key":"9509_CR93","doi-asserted-by":"publisher","unstructured":"Zhao, R., Zhou, A., & Mao, K. (2016): Automatic detection of cyberbullying on social networks based on bullying features. In Proceedings of the 17th international conference on distributed computing and networking, ICDCN \u201916 (pp. 43:1\u201343:6). ACM, New York, NY, USA. https:\/\/doi.org\/10.1145\/2833312.2849567.","DOI":"10.1145\/2833312.2849567"},{"key":"9509_CR94","unstructured":"Zhou, C., Sun, C., Liu, Z., & Lau, F. (2015). A c-lstm neural network for text classification. arXiv preprint arXiv:1511.08630."}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-020-09509-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10579-020-09509-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-020-09509-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,17]],"date-time":"2024-08-17T12:40:22Z","timestamp":1723898422000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10579-020-09509-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,16]]},"references-count":94,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["9509"],"URL":"https:\/\/doi.org\/10.1007\/s10579-020-09509-1","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"value":"1574-020X","type":"print"},{"value":"1574-0218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,16]]},"assertion":[{"value":"24 September 2020","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 November 2020","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}