{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T02:57:29Z","timestamp":1788922649334,"version":"build-2803163510"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,21]],"date-time":"2024-02-21T00:00:00Z","timestamp":1708473600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,21]],"date-time":"2024-02-21T00:00:00Z","timestamp":1708473600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62166007"],"award-info":[{"award-number":["62166007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Prompt-tuning has been successfully applied to support classification tasks in natural language processing and has achieved promising performance. The main characteristic of prompt-tuning based classification is to verbalize class labels and predict masked tokens like a cloze-like task. It has the advantage to make use of knowledge in pre-trained language models (PLMs). Because prompt templates are manually designed, they are more prone to overfitting. Furthermore, traditional prompt templates are appended in the tail of an original sentence. They are far from some semantic units in a sentence. It is weak to decode semantic information of an input relevant to PLMs. To aggregate more semantic information from PLMs for masked token prediction, we propose an annotation-aware prompt-tuning model for relation extraction. In our method, entity type representations are used as entity annotations. They are implanted near the site of entities in a sentence for decoding semantic information of PLMs. It is effective to make full use of knowledge in PLMs for relation extraction. In the experiment section, our method is validated on the Chinese literature text and SemEval 2010 Task datasets and achieves 89.3% and 90.6% in terms of F1-score, respectively. It achieves the state-of-the-art performance on two public datasets. The result further demonstrates the effectiveness of our model to decode semantic information in PLMs.<\/jats:p>","DOI":"10.1007\/s11063-024-11437-y","type":"journal-article","created":{"date-parts":[[2024,2,21]],"date-time":"2024-02-21T19:02:31Z","timestamp":1708542151000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["APRE: Annotation-Aware Prompt-Tuning for Relation Extraction"],"prefix":"10.1007","volume":"56","author":[{"given":"Chao","family":"Wei","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanping","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongbin","family":"Qin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruizhang","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qinghua","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,2,21]]},"reference":[{"key":"11437_CR1","doi-asserted-by":"publisher","unstructured":"Zhang N, Deng S, Sun Z, Chen X, Zhang W, Chen H (2018) Attention-based capsule networks with dynamic routing for relation extraction. In: Proceedings of the 2018 conference on empirical methods in natural language processing. Association for Computational Linguistics, Brussels, Belgium, pp 986\u2013992. https:\/\/doi.org\/10.18653\/v1\/D18-1120; https:\/\/aclanthology.org\/D18-1120","DOI":"10.18653\/v1\/D18-1120"},{"key":"11437_CR2","doi-asserted-by":"publisher","first-page":"13195","DOI":"10.1109\/ACCESS.2020.2966303","volume":"8","author":"Y Chen","year":"2020","unstructured":"Chen Y, Wang K, Yang W, Qing Y, Huang R, Chen P (2020) A multi-channel deep neural network for relation extraction. IEEE Access 8:13195\u201313203. https:\/\/doi.org\/10.1109\/ACCESS.2020.2966303","journal-title":"IEEE Access"},{"key":"11437_CR3","doi-asserted-by":"publisher","unstructured":"Schick T, Sch\u00fctze H (2021) Exploiting cloze-questions for few-shot text classification and natural language inference. In: Proceedings of the 16th conference of the European chapter of the association for computational linguistics: main volume. Association for Computational Linguistics, Online, pp 255\u2013269. https:\/\/doi.org\/10.18653\/v1\/2021.eacl-main.20; https:\/\/aclanthology.org\/2021.eacl-main.20","DOI":"10.18653\/v1\/2021.eacl-main.20"},{"key":"11437_CR4","doi-asserted-by":"publisher","DOI":"10.3390\/sym13040539","author":"Y Qin","year":"2021","unstructured":"Qin Y, Yang W, Wang K, Huang R, Tian F, Ao S, Chen Y (2021) Entity relation extraction based on entity indicators. Symmetry. https:\/\/doi.org\/10.3390\/sym13040539","journal-title":"Symmetry"},{"key":"11437_CR5","doi-asserted-by":"publisher","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 1 (long and short papers). Association for Computational Linguistics, Minneapolis, Minnesota, pp 4171\u20134186. https:\/\/doi.org\/10.18653\/v1\/N19-1423; https:\/\/aclanthology.org\/N19-1423","DOI":"10.18653\/v1\/N19-1423"},{"key":"11437_CR6","unstructured":"Radford A, Narasimhan K (2018) Improving language understanding by generative pre-training"},{"key":"11437_CR7","doi-asserted-by":"publisher","unstructured":"Chen X, Zhang N, Xie X, Deng S, Yao Y, Tan C, Huang F, Si L, Chen H (2022) Knowprompt: knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In: Proceedings of the ACM web conference 2022. WWW \u201922. Association for Computing Machinery, New York, NY, USA, pp 2778\u20132788. https:\/\/doi.org\/10.1145\/3485447.3511998; https:\/\/doi.org\/10.1145\/3485447.3511998","DOI":"10.1145\/3485447.3511998"},{"key":"11437_CR8","doi-asserted-by":"publisher","unstructured":"Baldini\u00a0Soares L, FitzGerald N, Ling J, Kwiatkowski T (2019) Matching the blanks: distributional similarity for relation learning. In: Proceedings of the 57th annual meeting of the association for computational linguistics. Association for Computational Linguistics, Florence, Italy, pp 2895\u20132905. https:\/\/doi.org\/10.18653\/v1\/P19-1279; https:\/\/aclanthology.org\/P19-1279","DOI":"10.18653\/v1\/P19-1279"},{"key":"11437_CR9","doi-asserted-by":"publisher","unstructured":"Schick T, Sch\u00fctze H (2021) It\u2019s not just size that matters: small language models are also few-shot learners. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies. Association for Computational Linguistics, Online, pp 2339\u20132352. https:\/\/doi.org\/10.18653\/v1\/2021.naacl-main.185; https:\/\/aclanthology.org\/2021.naacl-main.185","DOI":"10.18653\/v1\/2021.naacl-main.185"},{"key":"11437_CR10","doi-asserted-by":"publisher","unstructured":"Schick T, Schmid H, Sch\u00fctze H (2020) Automatically identifying words that can serve as labels for few-shot text classification. In: Proceedings of the 28th international conference on computational linguistics. International Committee on Computational Linguistics, Barcelona, Spain, pp 5569\u20135578 (Online). https:\/\/doi.org\/10.18653\/v1\/2020.coling-main.488; https:\/\/aclanthology.org\/2020.coling-main.488","DOI":"10.18653\/v1\/2020.coling-main.488"},{"key":"11437_CR11","unstructured":"Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2020) Language models are few-shot learners. In: Proceedings of the 34th international conference on neural information processing systems. NIPS\u201920. Curran Associates Inc., Red Hook, NY, USA. https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf"},{"key":"11437_CR12","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1016\/j.aiopen.2022.11.003","volume":"3","author":"X Han","year":"2022","unstructured":"Han X, Zhao W, Ding N, Liu Z, Sun M (2022) Ptr: prompt tuning with rules for text classification. AI Open 3:182\u2013192. https:\/\/doi.org\/10.1016\/j.aiopen.2022.11.003","journal-title":"AI Open"},{"key":"11437_CR13","doi-asserted-by":"publisher","unstructured":"Lester B, Al-Rfou R, Constant N (2021) The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, pp 3045\u20133059. https:\/\/doi.org\/10.18653\/v1\/2021.emnlp-main.243; https:\/\/aclanthology.org\/2021.emnlp-main.243","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"11437_CR14","unstructured":"Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized BERT pretraining approach. arXiv:1907.11692"},{"key":"11437_CR15","doi-asserted-by":"publisher","unstructured":"Petroni F, Rockt\u00e4schel T, Riedel S, Lewis P, Bakhtin A, Wu Y, Miller A (2019) Language models as knowledge bases? In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, pp 2463\u20132473. https:\/\/doi.org\/10.18653\/v1\/D19-1250; https:\/\/aclanthology.org\/D19-1250","DOI":"10.18653\/v1\/D19-1250"},{"key":"11437_CR16","doi-asserted-by":"publisher","unstructured":"Wu S, He Y (2019) Enriching pre-trained language model with entity information for relation classification. In: Proceedings of the 28th ACM international conference on information and knowledge management. CIKM \u201919. Association for Computing Machinery, New York, NY, USA, pp 2361\u20132364. https:\/\/doi.org\/10.1145\/3357384.3358119","DOI":"10.1145\/3357384.3358119"},{"key":"11437_CR17","doi-asserted-by":"publisher","unstructured":"Peters ME, Neumann M, Logan R, Schwartz R, Joshi V, Singh S, Smith NA (2019) Knowledge enhanced contextual word representations. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, pp 43\u201354. https:\/\/doi.org\/10.18653\/v1\/D19-1005; https:\/\/aclanthology.org\/D19-1005","DOI":"10.18653\/v1\/D19-1005"},{"key":"11437_CR18","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/j.neunet.2021.04.010","volume":"141","author":"Y Chen","year":"2021","unstructured":"Chen Y, Yang W, Wang K, Qin Y, Huang R, Zheng Q (2021) A neuralized feature engineering method for entity relation extraction. Neural Netw 141:249\u2013260. https:\/\/doi.org\/10.1016\/j.neunet.2021.04.010","journal-title":"Neural Netw"},{"key":"11437_CR19","doi-asserted-by":"publisher","unstructured":"Lyu S, Chen H (2021) Relation classification with entity type restriction. In: Findings of the association for computational linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, Online, pp 390\u2013395. https:\/\/doi.org\/10.18653\/v1\/2021.findings-acl.34; https:\/\/aclanthology.org\/2021.findings-acl.34","DOI":"10.18653\/v1\/2021.findings-acl.34"},{"key":"11437_CR20","unstructured":"Zhou W, Chen M (2022) An improved baseline for sentence-level relation extraction. In: Proceedings of the 2nd conference of the Asia-Pacific chapter of the association for computational linguistics and the 12th international joint conference on natural language processing (volume 2: short papers). Association for Computational Linguistics, Online only, pp 161\u2013168. https:\/\/aclanthology.org\/2022.aacl-short.21"},{"key":"11437_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.106888","volume":"219","author":"K Zhao","year":"2021","unstructured":"Zhao K, Xu H, Cheng Y, Li X, Gao K (2021) Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction. Knowl-Based Syst 219:106888. https:\/\/doi.org\/10.1016\/j.knosys.2021.106888","journal-title":"Knowl-Based Syst"},{"key":"11437_CR22","doi-asserted-by":"crossref","unstructured":"Ding N, Chen Y, Han X, Xu G, Wang X, Xie P, Zheng H, Liu Z, Li J, Kim H-G (2022) Prompt-learning for fine-grained entity typing. In: Findings of the association for computational linguistics: EMNLP 2022. Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, pp 6888\u20136901. https:\/\/aclanthology.org\/2022.findings-emnlp.512","DOI":"10.18653\/v1\/2022.findings-emnlp.512"},{"key":"11437_CR23","doi-asserted-by":"publisher","unstructured":"Li XL, Liang P (2021) Prefix-tuning: optimizing continuous prompts for generation. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: long papers). Association for Computational Linguistics, Online, pp 4582\u20134597. https:\/\/doi.org\/10.18653\/v1\/2021.acl-long.353; https:\/\/aclanthology.org\/2021.acl-long.353","DOI":"10.18653\/v1\/2021.acl-long.353"},{"issue":"1","key":"11437_CR24","doi-asserted-by":"publisher","first-page":"2161478","DOI":"10.1080\/09540091.2022.2161478","volume":"35","author":"K Wang","year":"2023","unstructured":"Wang K, Chen Y, Wen K, Wei C, Dong B, Zheng Q, Qin Y (2023) Cue prompt adapting model for relation extraction. Connect Sci 35(1):2161478. https:\/\/doi.org\/10.1080\/09540091.2022.2161478","journal-title":"Connect Sci"},{"key":"11437_CR25","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Proceedings of the 31st international conference on neural information processing systems. NIPS\u201917. Curran Associates Inc., Red Hook, NY, USA, pp 6000\u20136010"},{"key":"11437_CR26","doi-asserted-by":"crossref","unstructured":"Hendrickx I, Kim SN, Kozareva Z, Nakov P, \u00d3\u00a0S\u00e9aghdha D, Pad\u00f3 S, Pennacchiotti M, Romano L, Szpakowicz S (2010) SemEval-2010 task 8: multi-way classification of semantic relations between pairs of nominals. In: Proceedings of the 5th international workshop on semantic evaluation. Association for Computational Linguistics, Uppsala, Sweden, pp 33\u201338. https:\/\/aclanthology.org\/S10-1006","DOI":"10.3115\/1621969.1621986"},{"key":"11437_CR27","unstructured":"Xu J, Wen J, Sun X, Su Q (2017) A discourse-level named entity recognition and relation extraction dataset for Chinese literature text, vol. abs\/1711.07010. arxiv:1711.07010"},{"key":"11437_CR28","doi-asserted-by":"publisher","unstructured":"Xie W (2021) A entity attention-based model for entity relation classification for Chinese literature text. In: 2021 IEEE 4th advanced information management, communicates, electronic and automation control conference (IMCEC), vol 4, pp 1104\u20131108. https:\/\/doi.org\/10.1109\/IMCEC51613.2021.9482227","DOI":"10.1109\/IMCEC51613.2021.9482227"},{"key":"11437_CR29","doi-asserted-by":"publisher","unstructured":"Dos\u00a0Santos C, Xiang B, Zhou B (2015) Classifying relations by ranking with convolutional neural networks. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, vol 1. https:\/\/doi.org\/10.3115\/v1\/P15-1061","DOI":"10.3115\/v1\/P15-1061"},{"key":"11437_CR30","doi-asserted-by":"publisher","unstructured":"Cai R, Zhang X, Wang H (2016) Bidirectional recurrent convolutional neural network for relation classification. In: Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers). Association for Computational Linguistics, Berlin, Germany, pp 756\u2013765. https:\/\/doi.org\/10.18653\/v1\/P16-1072; https:\/\/aclanthology.org\/P16-1072","DOI":"10.18653\/v1\/P16-1072"},{"key":"11437_CR31","doi-asserted-by":"publisher","unstructured":"Wen J, Sun X, Ren X, Su Q (2018) Structure regularized neural network for entity relation classification for Chinese literature text. In: Proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, vol 2 (short papers). Association for Computational Linguistics, New Orleans, Louisiana, pp 365\u2013370. https:\/\/doi.org\/10.18653\/v1\/N18-2059; https:\/\/aclanthology.org\/N18-2059","DOI":"10.18653\/v1\/N18-2059"},{"key":"11437_CR32","doi-asserted-by":"publisher","unstructured":"Li B, Yu D, Ye W, Zhang J, Zhang S (2022) Sequence generation with label augmentation for relation extraction. arXiv e-prints, pp. 2212\u201314266 https:\/\/doi.org\/10.48550\/arXiv.2212.14266; arXiv:2212.14266","DOI":"10.48550\/arXiv.2212.14266"},{"key":"11437_CR33","doi-asserted-by":"crossref","unstructured":"Li B, Ye W, Zhang J, Zhang S (2022) Reviewing labels: label graph network with top-k prediction set for relation extraction. arXiv:2212.14270","DOI":"10.1609\/aaai.v37i11.26533"},{"key":"11437_CR34","doi-asserted-by":"publisher","unstructured":"Li J, Katsis Y, Baldwin T, Kim H-C, Bartko A, McAuley J, Hsu C-N (2022) Spot: knowledge-enhanced language representations for information extraction. In: Proceedings of the 31st ACM international conference on information and knowledge management. CIKM \u201922. Association for Computing Machinery, New York, NY, USA, pp 1124\u20131134. https:\/\/doi.org\/10.1145\/3511808.3557459; https:\/\/doi.org\/10.1145\/3511808.3557459","DOI":"10.1145\/3511808.3557459"},{"key":"11437_CR35","doi-asserted-by":"publisher","unstructured":"Vig J (2019) A multiscale visualization of attention in the transformer model. In: Proceedings of the 57th annual meeting of the association for computational linguistics: system demonstrations. Association for Computational Linguistics, Florence, Italy, pp 37\u201342. https:\/\/doi.org\/10.18653\/v1\/P19-3007; https:\/\/aclanthology.org\/P19-3007","DOI":"10.18653\/v1\/P19-3007"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11437-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11437-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11437-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T20:24:45Z","timestamp":1715891085000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11437-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,21]]},"references-count":35,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["11437"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11437-y","relation":{},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,21]]},"assertion":[{"value":"30 October 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 February 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"I hereby declare that the authors have no competing interests as defined by Springer, nor do they have any other interests that could be perceived as influencing the results and\/or discussions presented in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Ethical approval is not required for this work.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}}],"article-number":"62"}}