{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T16:55:47Z","timestamp":1773939347148,"version":"3.50.1"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,8]]},"abstract":"<jats:p>Recently, the Transformer machine translation system has shown strong results by stacking attention layers on both the source and target-language sides. But the inference of this model is slow due to the heavy use of dot-product attention in auto-regressive decoding. In this paper we speed up Transformer via a fast and lightweight attention model. More specifically, we share attention weights in adjacent layers and enable the efficient re-use of hidden states in a vertical manner. Moreover, the sharing policy can be jointly learned with the MT model. We test our approach on ten WMT and NIST OpenMT tasks. Experimental results show that it yields an average of 1.3X speed-up (with almost no decrease in BLEU) on top of a state-of-the-art implementation that has already adopted a cache for fast inference. Also, our approach obtains a 1.8X speed-up when it works with the AAN model. This is even 16 times faster than the baseline with no use of the attention cache.<\/jats:p>","DOI":"10.24963\/ijcai.2019\/735","type":"proceedings-article","created":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T03:46:05Z","timestamp":1564285565000},"page":"5292-5298","source":"Crossref","is-referenced-by-count":14,"title":["Sharing Attention Weights for Fast Transformer"],"prefix":"10.24963","author":[{"given":"Tong","family":"Xiao","sequence":"first","affiliation":[{"name":"Northeastern University, Shenyang, China"},{"name":"NiuTrans Co., Ltd., Shenyang, China"}]},{"given":"Yinqiao","family":"Li","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"}]},{"given":"Jingbo","family":"Zhu","sequence":"additional","affiliation":[{"name":"Northeastern University, Shenyang, China"},{"name":"NiuTrans Co., Ltd., Shenyang, China"}]},{"given":"Zhengtao","family":"Yu","sequence":"additional","affiliation":[{"name":"Kunming University of Science and Technology, Kunming, China"}]},{"given":"Tongran","family":"Liu","sequence":"additional","affiliation":[{"name":"CAS Key Laboratory of Behavioral Science, Institute of Psychology, CAS, Beijing, China"}]}],"member":"10584","event":{"name":"Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}","theme":"Artificial Intelligence","location":"Macao, China","acronym":"IJCAI-2019","number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2019,8,10]]},"end":{"date-parts":[[2019,8,16]]}},"container-title":["Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T03:51:27Z","timestamp":1564285887000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2019\/735"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2019,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2019\/735","relation":{},"subject":[],"published":{"date-parts":[[2019,8]]}}}