{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:45:25Z","timestamp":1740149125813,"version":"3.37.3"},"reference-count":51,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T00:00:00Z","timestamp":1665446400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T00:00:00Z","timestamp":1665446400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62066009"],"award-info":[{"award-number":["62066009"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"the Key Research and Development Project of Guilin","award":["2020010308"],"award-info":[{"award-number":["2020010308"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Intell Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Text classification is a crucial task in data mining and artificial intelligence. In recent years, deep learning-based text classification methods have made great development. The deep learning methods supervise model training by representing a label as a one-hot vector. However, the one-hot label representation cannot adequately reflect the relation between an instance and the labels, as labels are often not completely independent, and the instance may be associated with multiple labels in practice. Simply representing the labels as one-hot vectors leads to overconfidence in the model, making it difficult to distinguish some label confusions. In this paper, we propose a simulated label distribution method based on concepts (SLDC) to tackle this problem. This method captures the overlap between the labels by computing the similarity between an instance and the labels and generates a new simulated label distribution for assisting model training. In particular, we incorporate conceptual information from the knowledge base into the representation of instances and labels to address the surface mismatching problem when instances and labels are compared for similarity. Moreover, to fully use the simulated label distribution and the original label vector, we set up a multi-loss function to supervise the training process. Expensive experiments demonstrate the effectiveness of SLDC on five complex text classification datasets. Further experiments also verify that SLDC is especially helpful for confused datasets.<\/jats:p>","DOI":"10.1007\/s44196-022-00144-y","type":"journal-article","created":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T19:02:52Z","timestamp":1665514972000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Concept-Based Label Distribution Learning for Text Classification"],"prefix":"10.1007","volume":"15","author":[{"given":"Hui","family":"Li","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3056-4830","authenticated-orcid":false,"given":"Guimin","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Yiqun","family":"Li","sequence":"additional","affiliation":[]},{"given":"Xiaowei","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Yabing","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,11]]},"reference":[{"key":"144_CR1","doi-asserted-by":"crossref","unstructured":"Chen, J., Hu, Y., Liu, J., Xiao, Y., Jiang, H.: Deep short text classification with knowledge powered attention. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 6252\u2013 6259 ( 2019)","DOI":"10.1609\/aaai.v33i01.33016252"},{"key":"144_CR2","doi-asserted-by":"crossref","unstructured":"Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune bert for text classification. In: China National Conference on Chinese Computational Linguistics, pp. 194\u2013 206 ( 2019). Springer","DOI":"10.1007\/978-3-030-32381-3_16"},{"key":"144_CR3","doi-asserted-by":"crossref","unstructured":"Yao, L., Mao, C., Luo, Y.: Graph convolutional networks for text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 7370\u2013 7377 ( 2019)","DOI":"10.1609\/aaai.v33i01.33017370"},{"key":"144_CR4","unstructured":"Song, Y., Wang, H., Wang, Z., Li, H., Chen, W.: Short text conceptualization using a probabilistic knowledgebase. In: Twenty-second International Joint Conference on Artificial Intelligence ( 2011)"},{"key":"144_CR5","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818\u2013 2826 ( 2016)","DOI":"10.1109\/CVPR.2016.308"},{"key":"144_CR6","doi-asserted-by":"publisher","first-page":"5984","DOI":"10.1109\/TIP.2021.3089942","volume":"30","author":"C-B Zhang","year":"2021","unstructured":"Zhang, C.-B., Jiang, P.-T., Hou, Q., Wei, Y., Han, Q., Li, Z., Cheng, M.-M.: Delving deep into label smoothing. IEEE Trans. Image Process. 30, 5984\u20135996 (2021)","journal-title":"IEEE Trans. Image Process."},{"key":"144_CR7","doi-asserted-by":"crossref","unstructured":"Lukasik, M., Jain, H., Menon, A.K., Kim, S., Bhojanapalli, S., Yu, F., Kumar, S.: Semantic label smoothing for sequence to sequence problems. arXiv preprint arXiv:2010.07447 (2020)","DOI":"10.18653\/v1\/2020.emnlp-main.405"},{"key":"144_CR8","unstructured":"M\u00fcller, R., Kornblith, S., Hinton, G.E.: When does label smoothing help? Advances in neural information processing systems 32 (2019)"},{"key":"144_CR9","first-page":"12929","volume":"35","author":"B Guo","year":"2021","unstructured":"Guo, B., Han, S., Han, X., Huang, H., Lu, T.: Label confusion learning to enhance text classification models. Proc AAAI Conf rtificial Intell 35, 12929\u201312936 (2021)","journal-title":"Proc AAAI Conf rtificial Intell"},{"key":"144_CR10","doi-asserted-by":"crossref","unstructured":"Zhang, T., Mao, Y., Shen, F., Zhao, J.: Label distribution learning through exploring nonnegative components. Neurocomputing (2022)","DOI":"10.1016\/j.neucom.2022.06.017"},{"key":"144_CR11","doi-asserted-by":"crossref","unstructured":"Xiong, Y., Feng, Y., Wu, H., Kamigaito, H., Okumura, M.: Fusing label embedding into bert: An efficient improvement for text classification. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1743\u2013 1750 ( 2021)","DOI":"10.18653\/v1\/2021.findings-acl.152"},{"issue":"7","key":"144_CR12","doi-asserted-by":"publisher","first-page":"1734","DOI":"10.1109\/TKDE.2016.2545658","volume":"28","author":"X Geng","year":"2016","unstructured":"Geng, X.: Label distribution learning. IEEE Trans Knowl Data Eng 28(7), 1734\u20131748 (2016)","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"144_CR13","doi-asserted-by":"crossref","unstructured":"Hao, Y., Zhang, Y., Liu, K., He, S., Liu, Z., Wu, H., Zhao, J.: An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 221\u2013 231 ( 2017)","DOI":"10.18653\/v1\/P17-1021"},{"key":"144_CR14","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017)"},{"key":"144_CR15","doi-asserted-by":"crossref","unstructured":"Chorowski, J., Jaitly, N.: Towards better decoding and language model integration in sequence to sequence models. arXiv preprint arXiv:1612.02695 (2016)","DOI":"10.21437\/Interspeech.2017-343"},{"key":"144_CR16","doi-asserted-by":"crossref","unstructured":"Wang, G., Li, C., Wang, W., Zhang, Y., Shen, D., Zhang, X., Henao, R., Carin, L.: Joint embedding of words and labels for text classification. arXiv preprint arXiv:1805.04174 (2018)","DOI":"10.18653\/v1\/P18-1216"},{"key":"144_CR17","doi-asserted-by":"crossref","unstructured":"Zhang, H., Xiao, L., Chen, W., Wang, Y., Jin, Y.: Multi-task label embedding for text classification. arXiv preprint arXiv:1710.07210 (2017)","DOI":"10.18653\/v1\/D18-1484"},{"key":"144_CR18","first-page":"6359","volume":"33","author":"C Du","year":"2019","unstructured":"Du, C., Chen, Z., Feng, F., Zhu, L., Gan, T.: Explicit interaction model towards text classification. Proc AAAI Conf Artificial Intell 33, 6359\u20136366 (2019)","journal-title":"Proc AAAI Conf Artificial Intell"},{"key":"144_CR19","doi-asserted-by":"crossref","unstructured":"Wang, J., Geng, X.: Classification with label distribution learning. In: IJCAI, pp. 3712\u2013 3718 ( 2019)","DOI":"10.24963\/ijcai.2019\/515"},{"issue":"7","key":"144_CR20","doi-asserted-by":"publisher","first-page":"1734","DOI":"10.1109\/TKDE.2016.2545658","volume":"28","author":"X Geng","year":"2016","unstructured":"Geng, X.: Label distribution learning. IEEE Trans Knowl Data Eng 28(7), 1734\u20131748 (2016)","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"144_CR21","doi-asserted-by":"crossref","unstructured":"Jia, X., Li, W., Liu, J., Zhang, Y.: Label distribution learning by exploiting label correlations. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 ( 2018)","DOI":"10.1609\/aaai.v32i1.11664"},{"key":"144_CR22","doi-asserted-by":"crossref","unstructured":"Wang, J., Geng, X.: Classification with label distribution learning. In: IJCAI, pp. 3712\u2013 3718 ( 2019)","DOI":"10.24963\/ijcai.2019\/515"},{"issue":"4","key":"144_CR23","doi-asserted-by":"publisher","first-page":"1632","DOI":"10.1109\/TKDE.2019.2947040","volume":"33","author":"N Xu","year":"2019","unstructured":"Xu, N., Liu, Y.-P., Geng, X.: Label enhancement for label distribution learning. IEEE Trans Knowl Data Eng 33(4), 1632\u20131643 (2019)","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"144_CR24","doi-asserted-by":"crossref","unstructured":"Schreiber, A.T., Schreiber, G., Akkermans, H., Anjewierden, A., Shadbolt, N., de Hoog, R., Van\u00a0de Velde, W., Wielinga, B., Nigel, R., et al.: Knowledge Engineering and Management: the CommonKADS Methodology, ( 2000)","DOI":"10.7551\/mitpress\/4073.001.0001"},{"key":"144_CR25","unstructured":"Wang, Z., Zhao, K., Wang, H., Meng, X., Wen, J.-R.: Query understanding through knowledge-based conceptualization. In: Twenty-Fourth International Joint Conference on Artificial Intelligence ( 2015)"},{"key":"144_CR26","unstructured":"Kim, D., Wang, H., Oh, A., et al.: Context-dependent conceptualization (2013)"},{"issue":"3","key":"144_CR27","doi-asserted-by":"publisher","first-page":"499","DOI":"10.1109\/TKDE.2016.2571687","volume":"29","author":"W Hua","year":"2016","unstructured":"Hua, W., Wang, Z., Wang, H., Zheng, K., Zhou, X.: Understand short texts by harvesting and analyzing semantic knowledge. IEEE Trans. Knowl. Data Eng. 29(3), 499\u2013512 (2016)","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"144_CR28","doi-asserted-by":"crossref","unstructured":"Jiang, X., Shen, Y., Wang, Y., Jin, X., Cheng, X.: Bakgrastec: A background knowledge graph based method for short text classification. In: 2020 IEEE International Conference on Knowledge Graph (ICKG), pp. 360\u2013 366 ( 2020). IEEE","DOI":"10.1109\/ICBK50248.2020.00058"},{"key":"144_CR29","doi-asserted-by":"crossref","unstructured":"Xie, C., Liang, J., Xiao, Y., Hwang, S.-w.: Entity relationship explanation via conceptualization. Journal of Shanghai Jiaotong University (Science), 1\u20138 (2021)","DOI":"10.1007\/s12204-021-2394-1"},{"key":"144_CR30","doi-asserted-by":"crossref","unstructured":"Hearst, M.A.: Automatic acquisition of hyponyms from large text corpora. In: COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics ( 1992)","DOI":"10.3115\/992133.992154"},{"key":"144_CR31","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2019.105436","volume":"193","author":"P Li","year":"2020","unstructured":"Li, P., Mao, K., Xu, Y., Li, Q., Zhang, J.: Bag-of-concepts representation for document classification based on automatic knowledge acquisition from probabilistic knowledge base. Knowl-Based Syst 193, 105436 (2020)","journal-title":"Knowl-Based Syst"},{"key":"144_CR32","doi-asserted-by":"crossref","unstructured":"Wang, Z., Wang, H., Wen, J.-R., Xiao, Y.: An inference approach to basic level of categorization. In: Proceedings of the 24th Acm International on Conference on Information and Knowledge Management, pp. 653\u2013 662 ( 2015)","DOI":"10.1145\/2806416.2806533"},{"key":"144_CR33","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1162\/tacl_a_00179","volume":"2","author":"A Moro","year":"2014","unstructured":"Moro, A., Raganato, A., Navigli, R.: Entity linking meets word sense disambiguation: a unified approach. Trans Assoc Comput Linguistics 2, 231\u2013244 (2014)","journal-title":"Trans Assoc Comput Linguistics"},{"key":"144_CR34","doi-asserted-by":"publisher","unstructured":"Qi, P., Zhang, Y., Zhang, Y., Bolton, J., Manning, C.D.: Stanza: A python natural language processing toolkit for many human languages. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, ACL 2020, Online, July 5-10, 2020, pp. 101\u2013 108 ( 2020). https:\/\/doi.org\/10.18653\/v1\/2020.acl-demos.14","DOI":"10.18653\/v1\/2020.acl-demos.14"},{"key":"144_CR35","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)"},{"key":"144_CR36","unstructured":"Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)"},{"issue":"2","key":"144_CR37","first-page":"23","volume":"12","author":"P Gage","year":"1994","unstructured":"Gage, P.: A new algorithm for data compression. C Users J 12(2), 23\u201338 (1994)","journal-title":"C Users J"},{"key":"144_CR38","first-page":"20","volume":"1050","author":"P Velickovic","year":"2017","unstructured":"Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. Stat 1050, 20 (2017)","journal-title":"Stat"},{"issue":"1","key":"144_CR39","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1214\/aoms\/1177729694","volume":"22","author":"S Kullback","year":"1951","unstructured":"Kullback, S., Leibler, R.A.: On information and sufficiency. Ann Math Stat 22(1), 79\u201386 (1951)","journal-title":"Ann Math Stat"},{"key":"144_CR40","unstructured":"Le, Q., Mikolov, T.: Distributed representations of sentences and documents. In: International Conference on Machine Learning, pp. 1188\u2013 1196 ( 2014). PMLR"},{"key":"144_CR41","doi-asserted-by":"crossref","unstructured":"Tang, J., Qu, M., Mei, Q.: Pte: Predictive text embedding through large-scale heterogeneous text networks. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1165\u2013 1174 ( 2015)","DOI":"10.1145\/2783258.2783307"},{"key":"144_CR42","doi-asserted-by":"publisher","unstructured":"Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, Valencia, Spain, April 3-7, 2017, Volume 2: Short Papers, pp. 427\u2013 431 ( 2017). https:\/\/doi.org\/10.18653\/v1\/e17-2068","DOI":"10.18653\/v1\/e17-2068"},{"key":"144_CR43","doi-asserted-by":"publisher","unstructured":"Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A Meeting of SIGDAT, a Special Interest Group of The ACL, pp. 1746\u2013 1751 ( 2014). https:\/\/doi.org\/10.3115\/v1\/d14-1181","DOI":"10.3115\/v1\/d14-1181"},{"key":"144_CR44","unstructured":"Liu, P., Qiu, X., Huang, X.: Recurrent neural network for text classification with multi-task learning. In: Kambhampati, S. (ed.) Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016, pp. 2873\u2013 2879 ( 2016). http:\/\/www.ijcai.org\/Abstract\/16\/408"},{"key":"144_CR45","first-page":"12792","volume":"33","author":"M Ding","year":"2020","unstructured":"Ding, M., Zhou, C., Yang, H., Tang, J.: Cogltx: Applying bert to long texts. Adv Neural Inform Process Syst 33, 12792\u201312804 (2020)","journal-title":"Adv Neural Inform Process Syst"},{"key":"144_CR46","unstructured":"Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems 29 (2016)"},{"key":"144_CR47","unstructured":"Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs. In: Bengio, Y., LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings ( 2014). arxiv: org\/abs\/1312.6203"},{"key":"144_CR48","unstructured":"Henaff, M., Bruna, J., LeCun, Y.: Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163 (2015)"},{"key":"144_CR49","unstructured":"Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)"},{"issue":"1","key":"144_CR50","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/S0893-6080(98)00116-6","volume":"12","author":"N Qian","year":"1999","unstructured":"Qian, N.: On the momentum term in gradient descent learning algorithms. Neural Netw. 12(1), 145\u2013151 (1999)","journal-title":"Neural Netw."},{"key":"144_CR51","unstructured":"Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam (2018)"}],"container-title":["International Journal of Computational Intelligence Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-022-00144-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44196-022-00144-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-022-00144-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T19:28:36Z","timestamp":1665516516000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44196-022-00144-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,11]]},"references-count":51,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["144"],"URL":"https:\/\/doi.org\/10.1007\/s44196-022-00144-y","relation":{},"ISSN":["1875-6883"],"issn-type":[{"type":"electronic","value":"1875-6883"}],"subject":[],"published":{"date-parts":[[2022,10,11]]},"assertion":[{"value":"1 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interests"}},{"value":"This paper is an original work and has not been published nor has it been submitted simultaneously elsewhere. No data, text, or theories by others who are not list as coauthors are presented.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The datasets analysed during the current study are available from the corresponding author on reasonable request.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Availability of Data and Material"}}],"article-number":"85"}}