{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T11:56:50Z","timestamp":1787313410545,"version":"build-2736575974"},"reference-count":35,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2021,5,8]],"date-time":"2021-05-08T00:00:00Z","timestamp":1620432000000},"content-version":"vor","delay-in-days":127,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Complexity"],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Transformer is a neural machine translation model which revolutionizes machine translation. Compared with traditional statistical machine translation models and other neural machine translation models, the recently proposed transformer model radically and fundamentally changes machine translation with its self\u2010attention and cross\u2010attention mechanisms. These mechanisms effectively model token alignments between source and target sentences. It has been reported that the transformer model provides accurate posterior alignments. In this work, we empirically prove the reverse effect, showing that prior alignments help transformer models produce better translations. Experiment results on Vietnamese\u2010English news translation task show not only the positive effect of manually annotated alignments on transformer models but also the surprising outperformance of statistically constructed alignments reinforced with the flexibility of token\u2010type selection over manual alignments in improving transformer models. Statistically constructed word\u2010to\u2010lemma alignments are used to train a word\u2010to\u2010word transformer model. The novel hybrid transformer model improves the baseline transformer model and transformer model trained with manual alignments by 2.53 and 0.79 BLEU, respectively. In addition to BLEU score, we make limited human judgment on translation results. Strong correlation between human and machine judgment confirms our findings.<\/jats:p>","DOI":"10.1155\/2021\/5515407","type":"journal-article","created":{"date-parts":[[2021,5,8]],"date-time":"2021-05-08T14:52:01Z","timestamp":1620485521000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Improving Transformer\u2010Based Neural Machine Translation with Prior Alignments"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7796-4778","authenticated-orcid":false,"given":"Thien","family":"Nguyen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3796-9734","authenticated-orcid":false,"given":"Lam","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0793-9590","authenticated-orcid":false,"given":"Phuoc","family":"Tran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5607-1938","authenticated-orcid":false,"given":"Huu","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2021,5,8]]},"reference":[{"key":"e_1_2_9_1_2","doi-asserted-by":"crossref","unstructured":"KoehnP. Moses: open source toolkit for statistical machine translation Proceedings of the 45th Annual Meeting of the ACL on interactive Poster and Demonstration Sessions June 2007 Stroudsburg PA USA 177\u2013180.","DOI":"10.3115\/1557769.1557821"},{"key":"e_1_2_9_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3342353"},{"key":"e_1_2_9_3_2","unstructured":"SongK. ZhangY. YuH. LuoW. WangK. andZhangM. Code-switching for enhancing NMT with pre-specified translation https:\/\/www.aclweb.org\/anthology\/N19-1044.pdf."},{"key":"e_1_2_9_4_2","unstructured":"ChenW. MatusovE. KhadiviS. andPeterJ. T. Guided alignment training for topic-aware neural machine translation Proceedings of the AMTA 2016: 12th Conference of the Association for Machine Translation in the Americas October 2016 Austin Texas USA 121\u2013134."},{"key":"e_1_2_9_5_2","doi-asserted-by":"crossref","unstructured":"GargS. PeitzS. NallasamyU. andPaulikM. Jointly learning to align and translate with transformer models Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) November 2019 Hong Kong China 4453\u20134462 https:\/\/doi.org\/10.18653\/v1\/d19-1453.","DOI":"10.18653\/v1\/D19-1453"},{"key":"e_1_2_9_6_2","first-page":"5998","volume-title":"Advances in Neural information Processing Systems","author":"Vaswani A.","year":"2017"},{"key":"e_1_2_9_7_2","unstructured":"VaswaniA. Tensor2tensor for neural machine translation Proceedings of the 13th Conference of the Association for Machine Translation in the Americas March 2018 Boston MA USA 193\u2013199."},{"key":"e_1_2_9_8_2","doi-asserted-by":"crossref","unstructured":"NgoQ. H.andWiniwarterW. Building an English-Vietnamese bilingual corpus for machine translation Proceedings of the 2012 International Conference on Asian Language Processing November 2012 Hanoi Vietnam 157\u2013160 https:\/\/doi.org\/10.1109\/IALP.2012.30 2-s2.0-84892988591.","DOI":"10.1109\/IALP.2012.30"},{"key":"e_1_2_9_9_2","unstructured":"NgoQ. H. WiniwarterW. andWlokaB. EVBCorpus-a multi-layer English-Vietnamese bilingual corpus for studying tasks in comparative linguistics Proceedings of the 11th Workshop on Asian Language Resources October 2013 Nagoya Japan 1\u20139."},{"key":"e_1_2_9_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30194-3_13"},{"key":"e_1_2_9_11_2","doi-asserted-by":"publisher","DOI":"10.1162\/089120103321337421"},{"key":"e_1_2_9_12_2","first-page":"263","article-title":"The mathematics of statistical machine translation: parameter estimation","volume":"19","author":"Brown P. F.","year":"1993","journal-title":"Computational Linguistics"},{"key":"e_1_2_9_13_2","unstructured":"DyerC. ChahuneauV. andSmithN. A. A simple fast and effective reparameterization of ibm model 2 Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies June 2013 Atlanta GA USA 644\u2013648."},{"key":"e_1_2_9_14_2","doi-asserted-by":"publisher","DOI":"10.1515\/pralin-2017-0006"},{"key":"e_1_2_9_15_2","doi-asserted-by":"crossref","unstructured":"AlkhouliT.andNeyH. Biasing attention-based recurrent neural networks using external alignment information Proceedings of the Second Conference on Machine Translation August 2017 Florence Italy 108\u2013117.","DOI":"10.18653\/v1\/W17-4711"},{"key":"e_1_2_9_16_2","doi-asserted-by":"crossref","unstructured":"ZengJ. Multi-domain neural machine translation with word-level domain context discrimination Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing October 2018 Brussels Belgium 447\u2013457.","DOI":"10.18653\/v1\/D18-1041"},{"key":"e_1_2_9_17_2","unstructured":"LiuL. UtiyamaM. FinchA. andSumitaE. Neural machine translation with supervised attention Proceedings of COLING 2016 the 26th International Conference on Computational Linguistics: Technical Papers December 2016 Osaka Japan 3093\u20133102."},{"key":"e_1_2_9_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2019.2914149"},{"key":"e_1_2_9_19_2","doi-asserted-by":"crossref","unstructured":"ZhangJ. Improving the transformer translation model with document-level context Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing November 2018 Brussels Belgium 533\u2013542.","DOI":"10.18653\/v1\/D18-1049"},{"key":"e_1_2_9_20_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-020-19266-y"},{"key":"e_1_2_9_21_2","unstructured":"ParmarN. Image transformer Proceedings of the International Conference on Machine Learning May 2018 Jinan China 4055\u20134064."},{"key":"e_1_2_9_22_2","unstructured":"WolfT. Transformers: state-of-the-art natural language processing Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations November 2020 Suzhou China 38\u201345."},{"key":"e_1_2_9_23_2","unstructured":"M\u00fcllerR. KornblithS. andHintonG. E. When does label smoothing help? Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 December 2019 Vancouver Canada 4696\u20134705."},{"key":"e_1_2_9_24_2","unstructured":"KoehnP. AxelrodA. MayneA. B. Callison-BurchC. OsborneM. andTalbotD. Edinburgh system description for the 2005 IWSLT speech translation evaluation Proceedings of the IWSLT-2005 2005 Pittsburgh PA USA."},{"key":"e_1_2_9_25_2","doi-asserted-by":"publisher","DOI":"10.1155\/2020\/8859452"},{"key":"e_1_2_9_26_2","doi-asserted-by":"crossref","unstructured":"VuT. NguyenD. Q. DrasM. andJohnsonM. VnCoreNLP: A Vietnamese natural language processing toolkit Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations June 2018 New Orleans LA USA 56\u201360.","DOI":"10.18653\/v1\/N18-5012"},{"key":"e_1_2_9_27_2","unstructured":"NguyenD. Q. VuT. DrasM. andJohnsonM. A Fast and Accurate Vietnamese Word Segmenter Proceedings of the 11th International Conference on Language Resources and Evaluation LREC 2018 May 2019 Miyazaki Japan 2582\u20132587."},{"key":"e_1_2_9_28_2","doi-asserted-by":"crossref","unstructured":"SennrichR. HaddowB. andBirchA. Neural machine translation of rare words with subword units Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics August 2016 Berlin Germany 1715\u20131725.","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_2_9_29_2","doi-asserted-by":"crossref","unstructured":"QiP. ZhangY. ZhangY. BoltonJ. andManningC. D. Stanza: A {Python} Natural Language Processing Toolkit for Many Human Languages 2020 https:\/\/nlp.stanford.edu\/pubs\/qi2020stanza.pdf.","DOI":"10.18653\/v1\/2020.acl-demos.14"},{"key":"e_1_2_9_30_2","doi-asserted-by":"publisher","DOI":"10.1155\/2020\/4795187"},{"key":"e_1_2_9_31_2","unstructured":"KleinG. KimY. DengY.et al. Neural machine translation toolkit Proceedings of the 13th Conference of the Association for Machine Translation in the Americas March 2018 Boston MA USA 177\u2013184."},{"key":"e_1_2_9_32_2","unstructured":"KleinG. HernandezF. NguyenV. andSenellartJ. The OpenNMT neural machine translation toolkit: 2020 edition Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (AMTA 2020) October 2020 102\u2013109."},{"key":"e_1_2_9_33_2","doi-asserted-by":"crossref","unstructured":"OttM. Fairseq: a fast extensible toolkit for sequence modeling Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations) June 2019 Minneapolis MN USA 48\u201353.","DOI":"10.18653\/v1\/N19-4009"},{"key":"e_1_2_9_34_2","doi-asserted-by":"crossref","unstructured":"PapineniK. RoukosS. WardT. andZhuW.-J. BLEU: a method for automatic evaluation of machine translation Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics July 2002 Philadelphia PA USA 311\u2013318.","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_2_9_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-005-4822-5"}],"container-title":["Complexity"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/complexity\/2021\/5515407.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/complexity\/2021\/5515407.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2021\/5515407","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T18:44:56Z","timestamp":1723229096000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2021\/5515407"}},"subtitle":[],"editor":[{"given":"Dr Shahzad","family":"Sarfraz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":35,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1155\/2021\/5515407"],"URL":"https:\/\/doi.org\/10.1155\/2021\/5515407","archive":["Portico"],"relation":{},"ISSN":["1076-2787","1099-0526"],"issn-type":[{"value":"1076-2787","type":"print"},{"value":"1099-0526","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1]]},"assertion":[{"value":"2021-01-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-30","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"5515407"}}