{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T09:33:55Z","timestamp":1768901635121,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,6,30]],"date-time":"2024-06-30T00:00:00Z","timestamp":1719705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Scientific Research Fund of Hunan Provincial Education Department","award":["21B0462"],"award-info":[{"award-number":["21B0462"]}]},{"name":"Scientific Research Fund of Hunan Provincial Education Department","award":["2024JJ6221"],"award-info":[{"award-number":["2024JJ6221"]}]},{"name":"Scientific Research Fund of Hunan Provincial Education Department","award":["U2003208"],"award-info":[{"award-number":["U2003208"]}]},{"name":"Natural Science Foundation of Hunan Province of China","award":["21B0462"],"award-info":[{"award-number":["21B0462"]}]},{"name":"Natural Science Foundation of Hunan Province of China","award":["2024JJ6221"],"award-info":[{"award-number":["2024JJ6221"]}]},{"name":"Natural Science Foundation of Hunan Province of China","award":["U2003208"],"award-info":[{"award-number":["U2003208"]}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["21B0462"],"award-info":[{"award-number":["21B0462"]}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["2024JJ6221"],"award-info":[{"award-number":["2024JJ6221"]}]},{"name":"Joint Funds of the National Natural Science Foundation of China","award":["U2003208"],"award-info":[{"award-number":["U2003208"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Distantly supervised relation extraction (DSRE), first used to address the limitations of manually annotated data via automatically annotating the data with triplet facts, is prone to issues such as mislabeled annotations due to the interference of noisy annotations. To address the interference of noisy annotations, we leveraged a novel knowledge distillation (KD) method which was different from the conventional models on DSRE. More specifically, we proposed a model-agnostic KD method, Multi-Level Knowledge Distillation with Adaptive Temperature (MKDAT), which mainly involves two modules: Adaptive Temperature Regulation (ATR) and Multi-Level Knowledge Distilling (MKD). ATR allocates adaptive entropy-based distillation temperatures to different training instances for providing a moderate softening supervision to the student, in which label hardening is possible for instances with great entropy. MKD combines the bag-level and instance-level knowledge of the teacher as supervisions of the student, and trains the teacher and student at the bag and instance levels, respectively, which aims at mitigating the effects of noisy annotation and improving the sentence-level prediction performance. In addition, we implemented three MKDAT models based on the CNN, PCNN, and ATT-BiLSTM neural networks, respectively, and the experimental results show that our distillation models outperform the baseline models on bag-level and instance-level evaluations.<\/jats:p>","DOI":"10.3390\/info15070382","type":"journal-article","created":{"date-parts":[[2024,7,1]],"date-time":"2024-07-01T09:06:07Z","timestamp":1719824767000},"page":"382","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["MKDAT: Multi-Level Knowledge Distillation with Adaptive Temperature for Distantly Supervised Relation Extraction"],"prefix":"10.3390","volume":"15","author":[{"given":"Jun","family":"Long","sequence":"first","affiliation":[{"name":"Big Data Institute, Central South University, Changsha 410075, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhuoying","family":"Yin","sequence":"additional","affiliation":[{"name":"Big Data Institute, Central South University, Changsha 410075, China"},{"name":"Guizhou Rural Credit Union, Guiyang 550000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Han","sequence":"additional","affiliation":[{"name":"School of Computer and Information Engineering, Guizhou University of Commerce, Guiyang 550025, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenti","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Hunan University of Science and Technology, Xiangtan 411100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Mintz, M., Bills, S., Snow, R., and Jurafsky, D. (August, January 2). Distant supervision for relation extraction without labeled data. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, Suntec, Singapore.","DOI":"10.3115\/1690219.1690287"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., and Ives, Z. (2007). Dbpedia: A nucleus for a web of open data. The Semantic Web, Springer.","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008, January 10\u201312). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada.","DOI":"10.1145\/1376616.1376746"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jiang, H., Cui, L., Xu, Z., Yang, D., Chen, J., Li, C., Liu, J., Liang, J., Wang, C., and Xiao, Y. (2019, January 10\u201316). Relation Extraction Using Supervision from Topic Knowledge of Relation Labels. Proceedings of the IJCAI, Macao, China.","DOI":"10.24963\/ijcai.2019\/698"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, N., Deng, S., Sun, Z., Wang, G., Chen, X., Zhang, W., and Chen, H. (2019, January 2\u20137). Long-tail Relation Extraction via Knowledge Graph Embeddings and Graph Convolution Networks. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.","DOI":"10.18653\/v1\/N19-1306"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, Y., Long, G., Shen, T., Zhou, T., Yao, L., Huo, H., and Jiang, J. (2020, January 7\u201312). Self-attention enhanced selective gate with entity-aware embedding for distantly supervised relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6342"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lin, X., Liu, T., Jia, W., and Gong, Z. (2021, January 7\u201311). Distantly Supervised Relation Extraction using Multi-Layer Revision Network and Confidence-based Multi-Instance Learning. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online, Punta Cana, Dominican Republic.","DOI":"10.18653\/v1\/2021.emnlp-main.15"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Riedel, S., Yao, L., and McCallum, A. (2010, January 20\u201324). Modeling relations and their mentions without labeled text. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Barcelona, Spain.","DOI":"10.1007\/978-3-642-15939-8_10"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wu, Y., Bamman, D., and Russell, S. (2017, January 9\u201311). Adversarial training for relation extraction. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1187"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Feng, J., Huang, M., Zhao, L., Yang, Y., and Zhu, X. (2018, January 2\u20137). Reinforcement learning for relation classification from noisy data. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12063"},{"key":"ref_11","unstructured":"He, Z., Chen, W., Wang, Y., Zhang, W., Wang, G., and Zhang, M. (2020, January 7\u201312). Improving neural relation extraction with positive and unlabeled learning. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chen, J., Guo, Z., and Yang, J. (March, January 26). Distant Supervision for Relation Extraction via Noise Filtering. Proceedings of the 2021 13th International Conference on Machine Learning and Computing, Shenzhen, China.","DOI":"10.1145\/3457682.3457743"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Shang, Y., Huang, H.Y., Mao, X.L., Sun, X., and Wei, W. (2020, January 7\u201312). Are noisy sentences useless for distant supervised relation extraction?. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6407"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tang, S., Zhang, J., Zhang, N., Wu, F., Xiao, J., and Zhuang, Y. (2017, January 7\u201311). ENCORE: External neural constraints regularized distant supervision for relation extraction. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan.","DOI":"10.1145\/3077136.3080735"},{"key":"ref_15","unstructured":"Lei, K., Chen, D., Li, Y., Du, N., Yang, M., Fan, W., and Shen, Y. (2018, January 20\u201326). Cooperative denoising for distantly supervised relation extraction. Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, NM, USA."},{"key":"ref_16","first-page":"1","article-title":"MiDTD: A Simple and Effective Distillation Framework for Distantly Supervised Relation Extraction","volume":"40","author":"Li","year":"2022","journal-title":"ACM Trans. Inf. Syst. (TOIS)"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zeng, D., Liu, K., Chen, Y., and Zhao, J. (2015, January 17\u201321). Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1203"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shi, G., Feng, C., Huang, L., Zhang, B., Ji, H., Liao, L., and Huang, H. (November, January 31). Genre Separation Network with Adversarial Training for Cross-genre Relation Extraction. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1125"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zeng, X., He, S., Liu, K., and Zhao, J. (2018, January 2\u20137). Large scaled relation extraction with reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11950"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"106480","DOI":"10.1016\/j.knosys.2020.106480","article-title":"CFSRE: Context-aware based on frame-semantics for distantly supervised relation extraction","volume":"210","author":"Zhao","year":"2020","journal-title":"Knowl.-Based Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, T., Wang, K., Chang, B., and Sui, Z. (2017, January 7\u201311). A soft-label method for noise-tolerant distantly supervised relation extraction. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1189"},{"key":"ref_22","unstructured":"Wu, S., Fan, K., and Zhang, Q. (February, January 27). Improving distantly supervised relation extraction with neural noise converter and conditional optimal selector. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Ye, Z.X., and Ling, Z.H. (2019, January 2\u20137). Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.","DOI":"10.18653\/v1\/N19-1288"},{"key":"ref_24","unstructured":"Yuan, Y., Liu, L., Tang, S., Zhang, Z., Zhuang, Y., Pu, S., Wu, F., and Ren, X. (February, January 27). Cross-relation cross-bag attention for distantly-supervised relation extraction. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1016\/j.neucom.2021.04.127","article-title":"Distant supervised relation extraction with position feature attention and selective bag attention","volume":"461","author":"Wang","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"100656","DOI":"10.1016\/j.websem.2021.100656","article-title":"Improved distant supervision relation extraction based on edge-reasoning hybrid graph model","volume":"70","author":"Shen","year":"2021","journal-title":"J. Web Semant."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"107565","DOI":"10.1016\/j.knosys.2021.107565","article-title":"Local-to-global GCN with knowledge-aware representation for distantly supervised relation extraction","volume":"234","author":"Huang","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"110108","DOI":"10.1016\/j.knosys.2022.110108","article-title":"Distantly supervised relation extraction with KB-enhanced reconstructed latent iterative graph networks","volume":"260","author":"Zhou","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alt, C., H\u00fcbner, M., and Hennig, L. (2019). Fine-tuning pre-trained transformer language models to distantly supervised relation extraction. arXiv.","DOI":"10.18653\/v1\/P19-1134"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"107637","DOI":"10.1016\/j.knosys.2021.107637","article-title":"Distantly supervised relation extraction using global hierarchy embeddings and local probability constraints","volume":"235","author":"Peng","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"105912","DOI":"10.1016\/j.knosys.2020.105912","article-title":"A dynamic parameter enhanced network for distant supervised relation extraction","volume":"197","author":"Gou","year":"2020","journal-title":"Knowl.-Based Syst."},{"key":"ref_32","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"114813","DOI":"10.1016\/j.eswa.2021.114813","article-title":"Data-free knowledge distillation in neural networks for regression","volume":"175","author":"Kang","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"116474","DOI":"10.1016\/j.eswa.2021.116474","article-title":"DistilledCTR: Accurate and scalable CTR prediction model through model distillation","volume":"193","author":"Jose","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, Y., Yang, J., Song, Y., Cao, L., Luo, J., and Li, L.J. (2017, January 22\u201329). Learning from noisy labels with distillation. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.211"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sarfraz, F., Arani, E., and Zonooz, B. (2021, January 10\u201315). Knowledge distillation beyond model compression. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9413016"},{"key":"ref_37","unstructured":"Helong, Z., Liangchen, S., Jiajie, C., Ye, Z., Guoli, W., Junsong, Y., and Zhang, Q. (2021, January 3\u20137). Rethinking soft labels for knowledge distillation: A bias-variance tradeoff perspective. Proceedings of the International Conference on Learning Representations (ICLR), Virtual Event, Austria."},{"key":"ref_38","unstructured":"Furlanello, T., Lipton, Z., Tschannen, M., Itti, L., and Anandkumar, A. (2018, January 10\u201315). Born again neural networks. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Pennington, J., Socher, R., and Manning, C.D. (2014, January 25\u201329). Glove: Global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1162"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhu, T., Wang, H., Yu, J., Zhou, X., Chen, W., Zhang, W., and Zhang, M. (2020, January 8\u201313). Towards Accurate and Consistent Evaluation: A Dataset for Distantly-Supervised Relation Extraction. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.","DOI":"10.18653\/v1\/2020.coling-main.566"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"117474","DOI":"10.1016\/j.eswa.2022.117474","article-title":"Teacher\u2013student knowledge distillation based on decomposed deep feature representation for intelligent mobile applications","volume":"202","author":"Sepahvand","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"115132","DOI":"10.1016\/j.eswa.2021.115132","article-title":"Online subclass knowledge distillation","volume":"181","author":"Tzelepi","year":"2021","journal-title":"Expert Syst. Appl."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/7\/382\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:08:14Z","timestamp":1760108894000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/7\/382"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,30]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["info15070382"],"URL":"https:\/\/doi.org\/10.3390\/info15070382","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,30]]}}}