{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T14:07:26Z","timestamp":1753884446197,"version":"3.41.2"},"reference-count":20,"publisher":"World Scientific Pub Co Pte Ltd","issue":"02","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Patt. Recogn. Artif. Intell."],"published-print":{"date-parts":[[2025,2]]},"abstract":"<jats:p> The attention mechanism has been successfully used in a sequence consisted of a series of word embeddings to improve the representation of the sequence. Inspired by this, we leverage the attention mechanism on a set of tasks to implement a multi-task transfer learning method called AMP (Attentions between Multiple Prompts). First, we encode a task into a prompt as task representation called task embedding. Second, we learn an attention component on all task embeddings to generate a combined prompt for each task, which is an attention-weighted sum of task embeddings. Each combined prompt incorporates the knowledge of all tasks. The word embedding is a vector, but the task embedding is a 2D matrix. The attention mechanism can be exploited on a set of vectors rather than on a set of matrices. The prior methods employ pooling or flattened method to transform the matrix to the vector for computing the attentions between matrices. We propose a method called DAM (Direct Attention Mechanism) which can compute attentions between matrices directly without transforming. DAM method can more exactly compute the attentions between matrices. Wide experiments demonstrate that AMP outperforms prompt-tuning method and prior prompt transfer methods. <\/jats:p>","DOI":"10.1142\/s0218001425570046","type":"journal-article","created":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T06:38:27Z","timestamp":1737527907000},"source":"Crossref","is-referenced-by-count":0,"title":["AMP: Multi-Task Transfer Learning via Leveraging Attention Mechanism on Task Embeddings"],"prefix":"10.1142","volume":"39","author":[{"given":"Yangyang","family":"Yu","sequence":"first","affiliation":[{"name":"School of Applied Engineering, Sanmenxia Vocational and Technical College, Sanmenxia 472000, Henan, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Keru","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Henan University of Science and Technology, Sanmenxia 472000, Henan, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2025,3,7]]},"reference":[{"key":"S0218001425570046BIB001","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.446"},{"key":"S0218001425570046BIB002","first-page":"642","volume-title":"Int. Conf. Machine Learning","author":"Bao H.","year":"2020"},{"key":"S0218001425570046BIB004","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3511998"},{"key":"S0218001425570046BIB005","first-page":"9355","volume":"34","author":"Chu X.","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"2","key":"S0218001425570046BIB007","first-page":"107","volume-title":"Proc. Sinn und Bedeutung","volume":"23","author":"De Marneffe M.-C.","year":"2019"},{"volume-title":"Proc. Third International Workshop on Paraphrasing (IWP2005)","year":"2005","author":"Dolan B.","key":"S0218001425570046BIB009"},{"key":"S0218001425570046BIB012","doi-asserted-by":"publisher","DOI":"10.3115\/1654536.1654538"},{"key":"S0218001425570046BIB015","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2022.11.003"},{"key":"S0218001425570046BIB018","first-page":"2790","volume-title":"Proc. 36th Int. Conf. Machine Learning","author":"Houlsby N.","year":"2019"},{"key":"S0218001425570046BIB020","first-page":"1022","volume":"34","author":"Mahabadi R. Karimi","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0218001425570046BIB022","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12022"},{"key":"S0218001425570046BIB032","first-page":"142","volume-title":"Proc. 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Maas A.","year":"2011"},{"issue":"1","key":"S0218001425570046BIB037","first-page":"5485","volume":"21","author":"Raffel C.","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"S0218001425570046BIB041","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33016949"},{"key":"S0218001425570046BIB044","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D13-1170"},{"key":"S0218001425570046BIB046","volume":"30","author":"Vaswani A.","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0218001425570046BIB049","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00061"},{"key":"S0218001425570046BIB052","volume":"32","author":"Yang Z.","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0218001425570046BIB055","volume":"28","author":"Zhang X.","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"S0218001425570046BIB056","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3070203"}],"container-title":["International Journal of Pattern Recognition and Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218001425570046","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T05:28:24Z","timestamp":1743139704000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/10.1142\/S0218001425570046"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2]]},"references-count":20,"journal-issue":{"issue":"02","published-print":{"date-parts":[[2025,2]]}},"alternative-id":["10.1142\/S0218001425570046"],"URL":"https:\/\/doi.org\/10.1142\/s0218001425570046","relation":{},"ISSN":["0218-0014","1793-6381"],"issn-type":[{"type":"print","value":"0218-0014"},{"type":"electronic","value":"1793-6381"}],"subject":[],"published":{"date-parts":[[2025,2]]},"article-number":"2557004"}}