{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T22:18:30Z","timestamp":1783117110709,"version":"3.54.6"},"reference-count":32,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDT"],"published-print":{"date-parts":[[2021,9,27]]},"abstract":"<jats:p>Harmful information identification is a critical research topic in natural language processing. Existing approaches have been focused either on rule-based methods or harmful text identification of normal documents. In this paper, we propose a BERT-based model to identify harmful information from social media, called Topic-BERT. Firstly, Topic-BERT utilizes BERT to take additional information as input to alleviate the sparseness of short texts. The GPU-DMM topic model is used to capture hidden topics of short texts for attention weight calculation. Secondly, the proposed model divides harmful short text identification into two stages, and different granularity labels are identified by two similar sub-models. Finally, we conduct extensive experiments on a real-world social media dataset to evaluate our model. Experimental results demonstrate that our model can significantly improve the classification performance compared with baseline methods.<\/jats:p>","DOI":"10.3233\/idt-200094","type":"journal-article","created":{"date-parts":[[2021,9,10]],"date-time":"2021-09-10T13:15:04Z","timestamp":1631279704000},"page":"333-342","source":"Crossref","is-referenced-by-count":4,"title":["Topic-BERT: Detecting harmful information from social media"],"prefix":"10.1177","volume":"15","author":[{"given":"Wang","family":"Gao","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Jianghan University, Hubei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongtao","family":"Deng","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jianghan University, Hubei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xun","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Jianghan University, Hubei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuan","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Wuhan University of Technology, Hubei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","reference":[{"key":"10.3233\/IDT-200094_ref1","doi-asserted-by":"crossref","unstructured":"Xu G, Qi C, Yu H, Xu S, Zhao C, Yuan J. Detecting sensitive information of unstructured text Using convolutional neural network. In: International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC); 2019. pp. 474\u2013479.","DOI":"10.1109\/CyberC.2019.00087"},{"key":"10.3233\/IDT-200094_ref2","unstructured":"Devlin J, Chang M, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT); 2019. pp. 4171\u20134186."},{"issue":"4","key":"10.3233\/IDT-200094_ref3","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MMUL.2020.3012675","article-title":"Detecting disaster-related tweets via multimodal adversarial neural network","volume":"27","author":"Gao","year":"2020","journal-title":"IEEE MultiMedia"},{"key":"10.3233\/IDT-200094_ref4","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. In: Advances in Neural Information Processing Systems (NIPS); 2017. pp. 5998\u20136008."},{"issue":"2","key":"10.3233\/IDT-200094_ref5","doi-asserted-by":"crossref","first-page":"187","DOI":"10.3233\/IDT-170287","article-title":"A novel framework for augmenting the quality of explanations in recommender systems","volume":"11","author":"Karacapilidis","year":"2017","journal-title":"Intelligent Decision Technologies"},{"key":"10.3233\/IDT-200094_ref6","doi-asserted-by":"crossref","unstructured":"Zhou W, Ge T, Xu K, Wei F, Zhou M. BERT-based lexical substitution. In: Annual Meeting of the Association for Computational Linguistics (ACL); 2019. pp. 3368\u20133373.","DOI":"10.18653\/v1\/P19-1328"},{"issue":"1","key":"10.3233\/IDT-200094_ref7","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3233\/IDT-190361","article-title":"NICFS: A novel feature selection method applied to lexicon based sentiment analysis","volume":"13","author":"Mehta","year":"2019","journal-title":"Intelligent Decision Technologies"},{"key":"10.3233\/IDT-200094_ref8","doi-asserted-by":"crossref","unstructured":"Phan XH, Nguyen LM, Horiguchi S. Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In: International Conference on World Wide Web (WWW); 2008. pp. 91\u2013100.","DOI":"10.1145\/1367497.1367510"},{"key":"10.3233\/IDT-200094_ref9","doi-asserted-by":"crossref","unstructured":"Li C, Wang H, Zhang Z, Sun A, Ma Z. Topic modeling for short texts with auxiliary word embeddings. In: International ACM Conference on Research and Development in Information Retrieval (SIGIR); 2016. pp. 165\u2013174.","DOI":"10.1145\/2911451.2911499"},{"key":"10.3233\/IDT-200094_ref10","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.neucom.2016.03.088","article-title":"Multi-label maximum entropy model for social emotion classification over short text","volume":"210","author":"Li","year":"2016","journal-title":"Neurocomputing"},{"key":"10.3233\/IDT-200094_ref11","doi-asserted-by":"crossref","unstructured":"Li P, He L, Hu X, Zhang Y, Li L, Wu X. Concept based short text stream classification with topic drifting detection. In: IEEE International Conference on Data Mining (ICDM); 2016. pp.\u00a01009\u20131014.","DOI":"10.1109\/ICDM.2016.0128"},{"key":"10.3233\/IDT-200094_ref12","doi-asserted-by":"crossref","unstructured":"Ravi S, Kozareva Z. Self-governing neural networks for on-device short text classification. In: Conference on Empirical Methods in Natural Language Processing (EMNLP); 2018. pp.\u00a0804\u2013810.","DOI":"10.18653\/v1\/D18-1105"},{"key":"10.3233\/IDT-200094_ref13","first-page":"993","article-title":"Latent Dirichlet allocation","volume":"3","author":"Blei","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"10.3233\/IDT-200094_ref14","unstructured":"Qiuxing C, Lixiu Y, Jie Y. Short text classification based on LDA topic model. In: International Conference on Audio, Language and Image Processing (ICALIP); 2016. pp. 749\u2013753."},{"key":"10.3233\/IDT-200094_ref15","unstructured":"Boom CD, Canneyt SV, Bohez S, Demeester T, Dhoedt B. Learning semantic similarity for very short texts. In: IEEE International Conference on Data Mining Workshop (ICDMW); 2015. pp. 1229\u20131234."},{"key":"10.3233\/IDT-200094_ref16","unstructured":"Mikolov T, Yih W, Zweig G. Linguistic regularities in continuous space word representations. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT); 2013. pp. 746\u2013751."},{"key":"10.3233\/IDT-200094_ref17","doi-asserted-by":"crossref","first-page":"145","DOI":"10.14311\/NNW.2020.30.011","article-title":"Representation learning of knowledge graphs using convolutional neural networks","volume":"30","author":"Gao","year":"2020","journal-title":"Neural Network World"},{"key":"10.3233\/IDT-200094_ref18","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1016\/j.neucom.2015.09.096","article-title":"Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification","volume":"174","author":"Wang","year":"2016","journal-title":"Neurocomputing"},{"key":"10.3233\/IDT-200094_ref19","doi-asserted-by":"crossref","unstructured":"Lee JY, Dernoncourt F. Sequential short-Text classification with recurrent and convolutional neural networks. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT); 2016. pp. 515\u2013520.","DOI":"10.18653\/v1\/N16-1062"},{"key":"10.3233\/IDT-200094_ref20","doi-asserted-by":"crossref","unstructured":"Zhang H, Ni W, Zhao M, Lin Z. Cluster-gated convolutional neural network for short text classification. In: Conference on Computational Natural Language Learning (CoNLL); 2019. pp. 1002\u20131011.","DOI":"10.18653\/v1\/K19-1094"},{"key":"10.3233\/IDT-200094_ref21","doi-asserted-by":"crossref","unstructured":"Zhang H, Gong Y, Yan Y, Duan N, Xu J, Wang J, et al. Pretraining-based natural language generation for text summarization. In: Conference on Computational Natural Language Learning (CoNLL); 2019. pp. 789\u2013797.","DOI":"10.18653\/v1\/K19-1074"},{"key":"10.3233\/IDT-200094_ref22","doi-asserted-by":"crossref","unstructured":"Sun C, Qiu X, Xu Y, Huang X. How to fine-Tune BERT for text classification? In: Chinese Computational Linguistics \u2013 18th China National Conference, (CCL); 2019. pp. 194\u2013206.","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"10.3233\/IDT-200094_ref23","unstructured":"Mao J, Liu W. Factuality classification using the pre-trained language representation model BERT. In: Proceedings of the Iberian Languages Evaluation Forum (IberLEF); 2019. pp.\u00a0126\u2013131."},{"key":"10.3233\/IDT-200094_ref24","doi-asserted-by":"crossref","first-page":"62574","DOI":"10.1109\/ACCESS.2021.3073428","article-title":"Improving distantly-supervised relation extraction through BERT-based label and instance embeddings","volume":"9","author":"Christou","year":"2021","journal-title":"IEEE Access"},{"key":"10.3233\/IDT-200094_ref25","doi-asserted-by":"crossref","unstructured":"Giachanou A, Rosso P. The battle against online harmful information: The cases of fake news and hate speech. In: ACM International Conference on Information and Knowledge Management (CIKM); 2020. pp. 3503\u20133504.","DOI":"10.1145\/3340531.3412169"},{"issue":"2","key":"10.3233\/IDT-200094_ref26","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/s10115-020-01512-w","article-title":"A relative position attention network for aspect-based sentiment analysis","volume":"63","author":"Wu","year":"2021","journal-title":"Knowledge and Information Systems"},{"issue":"2","key":"10.3233\/IDT-200094_ref27","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1007\/s10115-018-1314-7","article-title":"Incorporating word embeddings into topic modeling of short text","volume":"61","author":"Gao","year":"2019","journal-title":"Knowledge and Information Systems"},{"key":"10.3233\/IDT-200094_ref28","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1016\/j.neucom.2019.11.077","article-title":"Generation of topic evolution graphs from short text streams","volume":"383","author":"Gao","year":"2020","journal-title":"Neurocomputing"},{"key":"10.3233\/IDT-200094_ref29","unstructured":"Mikolov T, Yih WT, Zweig G. Linguistic regularities in continuous space word representations. In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT); 2013. pp. 746\u2013751."},{"issue":"12","key":"10.3233\/IDT-200094_ref30","doi-asserted-by":"crossref","first-page":"2928","DOI":"10.1109\/TKDE.2014.2313872","article-title":"BTM: Topic modeling over short texts","volume":"26","author":"Cheng","year":"2014","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"10.3233\/IDT-200094_ref31","doi-asserted-by":"crossref","unstructured":"Yin J, Wang J. A dirichlet multinomial mixture model-based approach for short text clustering. In: ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD); 2014. pp. 233\u2013242.","DOI":"10.1145\/2623330.2623715"},{"key":"10.3233\/IDT-200094_ref33","unstructured":"Quan X, Kit C, Ge Y, Pan SJ. Short and sparse text topic modeling via self-aggregation. In: International Conference on Artificial Intelligence (IJCAI); 2015. pp. 2270\u20132276."}],"container-title":["Intelligent Decision Technologies"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDT-200094","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:22:55Z","timestamp":1777454575000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDT-200094"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,27]]},"references-count":32,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/idt-200094","relation":{},"ISSN":["1872-4981","1875-8843"],"issn-type":[{"value":"1872-4981","type":"print"},{"value":"1875-8843","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,27]]}}}