{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:27:37Z","timestamp":1740122857570,"version":"3.37.3"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Joint Fund by Shaanxi Computer Society & Xi'an Xiangteng Microelectronics Technology Co., Ltd."},{"DOI":"10.13039\/501100017596","name":"Natural Science Basic Research Program of Shaanxi Province","doi-asserted-by":"publisher","award":["2023-JC-YB-568"],"award-info":[{"award-number":["2023-JC-YB-568"]}],"id":[{"id":"10.13039\/501100017596","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Scientific Research Program Funded by Shaanxi Provincial Education Department","award":["22JP028"],"award-info":[{"award-number":["22JP028"]}]},{"name":"Science and Technology Guidance Program of China National Textile and Apparel Council","award":["2020100"],"award-info":[{"award-number":["2020100"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61971339"],"award-info":[{"award-number":["61971339"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Continual relation extraction aims to learn new relations from a continuous stream of data while avoiding forgetting old relations. Existing methods typically use the BERT encoder to obtain semantic embeddings, ignoring the fact that the vector representations suffer from anisotropy and uneven distribution. Furthermore, the relation prototypes are usually computed by memory samples directly, resulting in the model being overly sensitive to memory samples. To solve these problems, we propose a new continual relation extraction method. Firstly, we modified the basic structure of the sample encoder to generate uniformly distributed semantic embeddings using the supervised SimCSE-BERT to obtain richer sample information. Secondly, we introduced static relation prototypes and dynamically adjust their proportion with dynamic relation prototypes to adapt to the feature space. Lastly, through experimental analysis on the widely used FewRel and TACRED datasets, the results demonstrate that the proposed method effectively enhances semantic embeddings and relation prototypes, resulting in a further alleviation of catastrophic forgetting in the model. The code will be soon released at <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/SuyueW\/SS-CRE\">https:\/\/github.com\/SuyueW\/SS-CRE<\/jats:ext-link>.<\/jats:p>","DOI":"10.1007\/s11063-024-11647-4","type":"journal-article","created":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T07:02:07Z","timestamp":1718866927000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["SS-CRE: A Continual Relation Extraction Method Through SimCSE-BERT and Static Relation Prototypes"],"prefix":"10.1007","volume":"56","author":[{"given":"Jinguang","family":"Chen","sequence":"first","affiliation":[]},{"given":"Suyue","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Lili","family":"Ma","sequence":"additional","affiliation":[]},{"given":"Bo","family":"Yang","sequence":"additional","affiliation":[]},{"given":"Kaibing","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,20]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Liu K (2020) A survey on neural relation extraction. In: Proceedings of the Science China Technological Sciences 63(10): 1971\u20131989","key":"11647_CR1","DOI":"10.1007\/s11431-020-1673-6"},{"doi-asserted-by":"crossref","unstructured":"Ji S, Pan S, Cambria E, et al. (2021) A survey on knowledge graphs: representation, acquisition, and applications. In: Proceedings of the IEEE Transactions on Neural Networks and Learning Systems 33(2): 494\u2013514","key":"11647_CR2","DOI":"10.1109\/TNNLS.2021.3070843"},{"doi-asserted-by":"crossref","unstructured":"Mishra A, Jain SK (2016) A survey on question answering systems with classification. In: Proceedings of the Journal of King Saud University-Computer and Information Sciences 28(3): 345\u2013361","key":"11647_CR3","DOI":"10.1016\/j.jksuci.2014.10.007"},{"doi-asserted-by":"crossref","unstructured":"Wang Z, Zhang Z, Lee CY, et al. (2022) Learning to prompt for continual learning. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition pp. 139\u2013149","key":"11647_CR4","DOI":"10.1109\/CVPR52688.2022.00024"},{"unstructured":"De Lange M, Aljundi R, Masana M, et al. (2021) A continual learning survey: defying forgetting in classification tasks. In: Proceedings of the IEEE Transactions on Pattern Analysis and Machine Intelligence 44(7): 3366\u20133385","key":"11647_CR5"},{"doi-asserted-by":"crossref","unstructured":"Parisi GI, Kemker R, Part JL, et al. (2019) Continual lifelong learning with neural networks: a review. In: Proceedings of the Neural Networks, 113: 54\u201371","key":"11647_CR6","DOI":"10.1016\/j.neunet.2019.01.012"},{"unstructured":"Good Fe Llow IJ, Mirza M, Xiao D, et al. (2013) An empirical investigation of catastrophic forgetting in gradient-based neural networks. In: Proceedings of the Computer Science, 84(12): 1387\u201391","key":"11647_CR7"},{"doi-asserted-by":"crossref","unstructured":"Han X, Dai Y, Gao T, et al. (2020) Continual relation learning via episodic memory activation and reconsolidation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6429\u20136440","key":"11647_CR8","DOI":"10.18653\/v1\/2020.acl-main.573"},{"doi-asserted-by":"crossref","unstructured":"Wu T, Li X, Li YF, et al. (2021) Curriculum-meta learning for order-robust continual relation extraction. In: Proceedings of the AAAI Conference on Artificial Intelligence 35(12): 10363\u201310369.","key":"11647_CR9","DOI":"10.1609\/aaai.v35i12.17241"},{"doi-asserted-by":"crossref","unstructured":"Cui L, Yang D, Yu J, et al. (2021) Refining sample embeddings with relation prototypes to enhance continual relation extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 232\u2013243","key":"11647_CR10","DOI":"10.18653\/v1\/2021.acl-long.20"},{"doi-asserted-by":"crossref","unstructured":"Zhao K, Xu H, Yang J, et al. (2022) Consistent representation learning for continual relation extraction. In: Proceedings of the Association for Computational Linguistics, pp. 3402\u20133411","key":"11647_CR11","DOI":"10.18653\/v1\/2022.findings-acl.268"},{"unstructured":"Hu CW, Yang DQ, Jin HL, et al. (2022) Improving continual relation extraction through prototypical contrastive learning. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 1885\u20131895.","key":"11647_CR12"},{"doi-asserted-by":"crossref","unstructured":"Wang P, Song Y, Liu T, et al. (2022) Learning robust representations for continual relation extraction via adversarial class augmentation. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 6264\u20136278","key":"11647_CR13","DOI":"10.18653\/v1\/2022.emnlp-main.420"},{"doi-asserted-by":"crossref","unstructured":"Zhao W, Cui Y, Hu W (2023) Improving Continual Relation Extraction by Distinguishing Analogous Semantics. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pp. 1162\u20131175.","key":"11647_CR14","DOI":"10.18653\/v1\/2023.acl-long.65"},{"doi-asserted-by":"crossref","unstructured":"Gao T, Yao X, Chen D (2021) Simcse: simple contrastive learning of sentence embeddings. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6894\u20136910","key":"11647_CR15","DOI":"10.18653\/v1\/2021.emnlp-main.552"},{"doi-asserted-by":"crossref","unstructured":"Mintz M, Bills S, Snow R et al. (2009) Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 1003\u20131011","key":"11647_CR16","DOI":"10.3115\/1690219.1690287"},{"doi-asserted-by":"crossref","unstructured":"Sun Q, Huang K, Yang XC et al. (2023) Uncertainty guided label denoising for document-level distant relation extraction. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15960\u201315973","key":"11647_CR17","DOI":"10.18653\/v1\/2023.acl-long.889"},{"doi-asserted-by":"crossref","unstructured":"Bekoulis G, Deleu J, Demeester T, et al. (2018) Joint entity recognition and relation extraction as a multi-head selection problem. In: Proceedings of the Expert Systems with Applications 114: 34\u201345","key":"11647_CR18","DOI":"10.1016\/j.eswa.2018.07.032"},{"doi-asserted-by":"crossref","unstructured":"Fu TJ, Li PH, Ma WY (2019) Graphrel: modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 1409\u20131418","key":"11647_CR19","DOI":"10.18653\/v1\/P19-1136"},{"doi-asserted-by":"crossref","unstructured":"Wang Y, Sun C, Wu Y et al. (2021) UniRE: a unified label space for entity relation extraction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pp. 220\u2013231","key":"11647_CR20","DOI":"10.18653\/v1\/2021.acl-long.19"},{"doi-asserted-by":"crossref","unstructured":"Qin P, Xu W, Wang WY (2018) DSGAN: generative adversarial training for distant supervision relation extraction. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 496\u2013505","key":"11647_CR21","DOI":"10.18653\/v1\/P18-1046"},{"doi-asserted-by":"crossref","unstructured":"Jung W, Shim K (2020) Dual supervision framework for relation extraction with distant supervision and human annotation. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 6411\u20136423","key":"11647_CR22","DOI":"10.18653\/v1\/2020.coling-main.564"},{"doi-asserted-by":"crossref","unstructured":"Kirkpatrick J, Pascanu R, Rabinowitz N et al. (2017) Overcoming catastrophic forgetting in neural networks. In: Proceedings of the National Academy of Sciences, 114(13): 3521\u20133526","key":"11647_CR23","DOI":"10.1073\/pnas.1611835114"},{"doi-asserted-by":"crossref","unstructured":"Dhar P, Singh RV, Peng KC, et al. (2019) Learning without memorizing. In: Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5138\u20135146","key":"11647_CR24","DOI":"10.1109\/CVPR.2019.00528"},{"unstructured":"Serra J, Suris D, Miron M, et al. (2018) Overcoming catastrophic forgetting with hard attention to the task. In: Proceedings of the 35th International Conference on Machine Learning, pp. 4548\u20134557","key":"11647_CR25"},{"unstructured":"Rusu AA, Rabinowitz NC, Desjardins G et al.2016)Progressive neural networks. arXiv preprint arXiv: 1606\u201304671","key":"11647_CR26"},{"unstructured":"Lopez-Paz D, Ranzato MA (2017) Gradient episodic memory for continual learning. In: Proceedings of the Neural Information Processing Systems, pp. 6467\u20136476","key":"11647_CR27"},{"doi-asserted-by":"crossref","unstructured":"Wang H, Xiong W, Yu M, et al. (2020) Sentence embedding alignment for lifelong relation extraction. In: Proceedings of the 2019 Conference of the North. Minneapolis, pp. 796\u2013806","key":"11647_CR28","DOI":"10.18653\/v1\/N19-1086"},{"unstructured":"Sun FK, Ho CH, Lee HY (2020) Lamol: Language modeling for lifelong language learning. In: Proceedings of the 8th International Conference on Learning Representations","key":"11647_CR29"},{"unstructured":"de Masson D'Autume C, Ruder S, Kong L, et al. (2019) Episodic memory in lifelong language learning. In: Proceedings of the Neural Information Processing Systems, pp. 13122\u201313131","key":"11647_CR30"},{"doi-asserted-by":"crossref","unstructured":"Han X, Zhu H, Yu P, et al. (2018) Fewrel: a large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4803\u20134809","key":"11647_CR31","DOI":"10.18653\/v1\/D18-1514"},{"doi-asserted-by":"crossref","unstructured":"Zhang Y, Zhong V, Chen D, et al (2017) Position-aware attention and supervised data improve slot filling. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 35\u201345","key":"11647_CR32","DOI":"10.18653\/v1\/D17-1004"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11647-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11647-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11647-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T16:08:40Z","timestamp":1725120520000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11647-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,20]]},"references-count":32,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["11647"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11647-4","relation":{},"ISSN":["1573-773X"],"issn-type":[{"type":"electronic","value":"1573-773X"}],"subject":[],"published":{"date-parts":[[2024,6,20]]},"assertion":[{"value":"8 May 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 June 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interests"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical and Informed Consent for Data Used"}},{"value":"The raw data elaborated during the current research are publicly available.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Data Availability"}}],"article-number":"203"}}