{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T04:18:00Z","timestamp":1770783480542,"version":"3.50.0"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2023,7,25]],"date-time":"2023-07-25T00:00:00Z","timestamp":1690243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022ZD0160602"],"award-info":[{"award-number":["2022ZD0160602"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62122088"],"award-info":[{"award-number":["62122088"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>\n            Prompt learning has emerged as a new paradigm for leveraging pre-trained language models (PLMs) and has shown promising results in downstream tasks with only a slight increase in parameters. However, the current usage of fixed prompts, whether discrete or continuous, assumes that all samples within a task share the same prompt. This assumption may not hold for tasks with diverse samples that require different prompt information. To address this issue, we propose an instance-aware prompt learning method that learns a different prompt for each instance. Specifically, we suppose that each learnable prompt token has a different contribution to different instances, and we learn the contribution by calculating the relevance score between an instance and each prompt token. The contribution-weighted prompt would be instance aware. We apply our method to both unidirectional and bidirectional PLMs on both language understanding and generation tasks. Extensive experiments demonstrate that our method achieves comparable results using as few as 1.5% of the parameters of PLMs tuned and obtains considerable improvements compared with strong baselines. In particular, our method achieves state-of-the-art results using ALBERT-xxlarge-v2 on the SuperGLUE few-shot learning benchmark.\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>1<\/jats:sup>\n            <\/jats:xref>\n          <\/jats:p>","DOI":"10.1145\/3604613","type":"journal-article","created":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T11:27:08Z","timestamp":1686742028000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Instance-Aware Prompt Learning for Language Understanding and Generation"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5551-7723","authenticated-orcid":false,"given":"Feihu","family":"Jin","sequence":"first","affiliation":[{"name":"Institute of Automation, Chinese Academy of Sciences (CAS), The School of Artificial Intelligence, University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5395-2385","authenticated-orcid":false,"given":"Jinliang","family":"Lu","sequence":"additional","affiliation":[{"name":"Institute of Automation, Chinese Academy of Sciences (CAS), The School of Artificial Intelligence, University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5293-7434","authenticated-orcid":false,"given":"Jiajun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute of Automation, Chinese Academy of Sciences (CAS), The School of Artificial Intelligence, University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9864-3818","authenticated-orcid":false,"given":"Chengqing","family":"Zong","sequence":"additional","affiliation":[{"name":"Institute of Automation, Chinese Academy of Sciences (CAS), The School of Artificial Intelligence, University of Chinese Academy of Sciences, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,7,25]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-45321-2_19"},{"key":"e_1_3_2_3_1","volume-title":"EACL","author":"Belz Anja","year":"2006","unstructured":"Anja Belz and Ehud Reiter. 2006. Comparing automatic and human evaluation of NLG systems. In EACL."},{"key":"e_1_3_2_4_1","volume-title":"NeurIPS","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Sutskever Ilya, and Amodei Dario. 2020. Language models are few-shot learners. In NeurIPS. https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf."},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.449"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.02311"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1300"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.18148\/sub\/2019.v23i2.601"},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.292"},{"key":"e_1_3_2_10_1","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume":"1810","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. ArXiv abs\/1810.04805 (2019).","journal-title":"ArXiv"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.295"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-3518"},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-5409"},{"key":"e_1_3_2_14_1","article-title":"Response generation with context-aware prompt learning","volume":"2111","author":"Gu Xiaodong","year":"2021","unstructured":"Xiaodong Gu, Kang Min Yoo, and Sang-Woo Lee. 2021. Response generation with context-aware prompt learning. CoRR abs\/2111.02643 (2021). arXiv:2111.02643https:\/\/arxiv.org\/abs\/2111.02643.","journal-title":"CoRR"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.381"},{"key":"e_1_3_2_16_1","volume-title":"ICML","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In ICML."},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00324"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1023"},{"key":"e_1_3_2_19_1","volume-title":"ICLR","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In ICLR. https:\/\/openreview.net\/forum?id=H1eA7AEtvS."},{"key":"e_1_3_2_20_1","volume-title":"Proceedings of the 2nd Workshop on Statistical Machine Translation","author":"Lavie Alon","year":"2007","unstructured":"Alon Lavie and Abhaya Agarwal. 2007. METEOR: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the 2nd Workshop on Statistical Machine Translation. https:\/\/aclanthology.org\/W07-0734."},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_3_2_22_1","volume-title":"International Conference on the Principles of Knowledge Representation and Reasoning","author":"Levesque Hector","year":"2012","unstructured":"Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012. The Winograd schema challenge. In International Conference on the Principles of Knowledge Representation and Reasoning."},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"e_1_3_2_24_1","volume-title":"ACL","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In ACL."},{"key":"e_1_3_2_25_1","article-title":"P-Tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks","volume":"2110","author":"Liu Xiao","year":"2021","unstructured":"Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021a. P-Tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. CoRR abs\/2110.07602 (2021). arXiv:2110.07602https:\/\/arxiv.org\/abs\/2110.07602.","journal-title":"CoRR"},{"key":"e_1_3_2_26_1","article-title":"GPT understands, too","volume":"2103","author":"Liu Xiao","year":"2021","unstructured":"Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021b. GPT understands, too. ArXiv abs\/2103.10385 (2021).","journal-title":"ArXiv"},{"key":"e_1_3_2_27_1","volume-title":"SIGDIAL","author":"Novikova Jekaterina","year":"2017","unstructured":"Jekaterina Novikova, Ondrej Dusek, and Verena Rieser. 2017. The E2E dataset: New challenges for end-to-end generation. In SIGDIAL."},{"key":"e_1_3_2_28_1","volume-title":"ACL","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In ACL."},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1128"},{"key":"e_1_3_2_30_1","volume-title":"NAACL-HLT","author":"Radev Dragomir","year":"2021","unstructured":"Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chia-Hsuan Hsieh, Nazneen Rajani, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Murori Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, and Richard Socher. 2021. DART: Open-domain structured data record to text generation. In NAACL-HLT."},{"issue":"8","key":"e_1_3_2_31_1","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et\u00a0al. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019).","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_32_1","volume-title":"Logical Formalizations of Commonsense Reasoning, Papers from the 2011 AAAI Spring Symposium, Technical Report SS-11-06, Stanford, California, USA, March 21\u201323, 2011","author":"Roemmele Melissa","year":"2011","unstructured":"Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. 2011. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Logical Formalizations of Commonsense Reasoning, Papers from the 2011 AAAI Spring Symposium, Technical Report SS-11-06, Stanford, California, USA, March 21\u201323, 2011. AAAI. http:\/\/www.aaai.org\/ocs\/index.php\/SSS\/SSS11\/paper\/view\/2418."},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.eacl-main.20"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.185"},{"key":"e_1_3_2_35_1","volume-title":"EMNLP","author":"Shin Taylor","year":"2020","unstructured":"Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. AutoPrompt: Eliciting knowledge from language models with automatically generated prompts. In EMNLP."},{"key":"e_1_3_2_36_1","volume-title":"Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers","author":"Snover Matthew","year":"2006","unstructured":"Matthew Snover, Bonnie Dorr, Rich Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas: Technical Papers. https:\/\/aclanthology.org\/2006.amta-papers.25."},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.407"},{"key":"e_1_3_2_38_1","article-title":"A simple method for commonsense reasoning","volume":"1806","author":"Trinh Trieu H.","year":"2018","unstructured":"Trieu H. Trinh and Quoc V. Le. 2018. A simple method for commonsense reasoning. ArXiv abs\/1806.02847 (2018).","journal-title":"ArXiv"},{"key":"e_1_3_2_39_1","volume-title":"CVPR","author":"Vedantam Ramakrishna","year":"2015","unstructured":"Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. CIDEr: Consensus-based image description evaluation. In CVPR."},{"key":"e_1_3_2_40_1","volume-title":"NeurIPS","author":"Wang Alex","year":"2019","unstructured":"Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. In NeurIPS."},{"key":"e_1_3_2_41_1","volume-title":"EMNLP","author":"Wolf Thomas","year":"2020","unstructured":"Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R\u00e9mi Louf, Morgan Funtowicz, and Jamie Brew. 2020. Transformers: State-of-the-art natural language processing. In EMNLP."},{"key":"e_1_3_2_42_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.04497"},{"key":"e_1_3_2_43_1","article-title":"ReCoRD: Bridging the gap between human and machine commonsense reading comprehension","volume":"1810","author":"Zhang Sheng","year":"2018","unstructured":"Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. 2018. ReCoRD: Bridging the gap between human and machine commonsense reading comprehension. CoRR abs\/1810.12885 (2018). arXiv:1810.12885http:\/\/arxiv.org\/abs\/1810.12885.","journal-title":"CoRR"},{"key":"e_1_3_2_44_1","volume-title":"ICLR","author":"Zhang Tianyi","year":"2020","unstructured":"Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTScore: Evaluating text generation with BERT. In ICLR. https:\/\/openreview.net\/forum?id=SkeHuCVFDr."},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.398"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604613","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3604613","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:04Z","timestamp":1750178764000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604613"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,25]]},"references-count":44,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3604613"],"URL":"https:\/\/doi.org\/10.1145\/3604613","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,25]]},"assertion":[{"value":"2022-09-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}