{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T18:56:28Z","timestamp":1770490588042,"version":"3.49.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,1,19]],"date-time":"2022-01-19T00:00:00Z","timestamp":1642550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Grant-in-Aid for Young Scientists","award":["#19K20343"],"award-info":[{"award-number":["#19K20343"]}]},{"name":"JSPS and Information\/AI\/Data Science Doctoral Fellowship of Kyoto University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2022,7,31]]},"abstract":"<jats:p>\n            In the present study, we propose novel sequence-to-sequence pre-training objectives for low-resource machine translation (NMT):\n            <jats:bold>Japanese-specific sequence to sequence (JASS)<\/jats:bold>\n            for language pairs involving Japanese as the source or target language, and\n            <jats:bold>English-specific sequence to sequence (ENSS)<\/jats:bold>\n            for language pairs involving English. JASS focuses on masking and reordering Japanese linguistic units known as bunsetsu, whereas ENSS is proposed based on phrase structure masking and reordering tasks. Experiments on ASPEC Japanese\u2013English &amp; Japanese\u2013Chinese, Wikipedia Japanese\u2013Chinese, News English\u2013Korean corpora demonstrate that JASS and ENSS outperform MASS and other existing language-agnostic pre-training methods by up to +2.9 BLEU points for the Japanese\u2013English tasks, up to +7.0 BLEU points for the Japanese\u2013Chinese tasks and up to +1.3 BLEU points for English\u2013Korean tasks. Empirical analysis, which focuses on the relationship between individual parts in JASS and ENSS, reveals the complementary nature of the subtasks of JASS and ENSS. Adequacy evaluation using LASER, human evaluation, and case studies reveals that our proposed methods significantly outperform pre-training methods without injected linguistic knowledge and they have a larger positive impact on the adequacy as compared to the fluency.\n          <\/jats:p>","DOI":"10.1145\/3491065","type":"journal-article","created":{"date-parts":[[2022,1,19]],"date-time":"2022-01-19T20:19:23Z","timestamp":1642623563000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Linguistically Driven Multi-Task Pre-Training for Low-Resource Neural Machine Translation"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5273-2738","authenticated-orcid":false,"given":"Zhuoyuan","family":"Mao","sequence":"first","affiliation":[{"name":"Graduate School of Informatics, Kyoto University, Japan"}]},{"given":"Chenhui","family":"Chu","sequence":"additional","affiliation":[{"name":"Graduate School of Informatics, Kyoto University, Japan"}]},{"given":"Sadao","family":"Kurohashi","sequence":"additional","affiliation":[{"name":"Graduate School of Informatics, Kyoto University, Japan"}]}],"member":"320","published-online":{"date-parts":[[2022,1,19]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.304"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00288"},{"key":"e_1_3_3_4_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR)","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR). (San Diego, CA)."},{"key":"e_1_3_3_5_2","first-page":"642","volume-title":"Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC\u201914)","author":"Chu Chenhui","year":"2014","unstructured":"Chenhui Chu, Toshiaki Nakazawa, and Sadao Kurohashi. 2014. Constructing a Chinese\u2014Japanese parallel corpus from Wikipedia. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC\u201914). European Language Resources Association (ELRA), Reykjavik, Iceland, 642\u2013647."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2833089"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454921"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1146"},{"key":"e_1_3_3_9_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171\u20134186."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1166"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1045"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.5555\/2392936.2392946"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-2703"},{"key":"e_1_3_3_14_2","first-page":"1062","volume-title":"Proceedings of the 6th International Joint Conference on Natural Language Processing","author":"Hoshino Sho","year":"2013","unstructured":"Sho Hoshino, Yusuke Miyao, Katsuhito Sudoh, and Masaaki Nagata. 2013. Two-stage pre-ordering for Japanese-to-English statistical machine translation. In Proceedings of the 6th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, Nagoya, Japan, 1062\u20131066."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/1868850.1868886"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00065"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00300"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-4012"},{"key":"e_1_3_3_19_2","first-page":"388","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing","author":"Koehn Philipp","year":"2004","unstructured":"Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Barcelona, Spain, 388\u2013395."},{"key":"e_1_3_3_20_2","first-page":"77","volume-title":"Proceedings of the 2006 International Workshop on Spoken Language Translation (IWSLT 2006) (Keihanna Science City, Kyoto, Japan, November 27-28, 2006)","author":"Komachi Mamoru","year":"2006","unstructured":"Mamoru Komachi, Yuji Matsumoto, and Masaaki Nagata. 2006. Phrase reordering for statistical machine translation based on predicate-argument structure. In Proceedings of the 2006 International Workshop on Spoken Language Translation (IWSLT 2006) (Keihanna Science City, Kyoto, Japan, November 27-28, 2006). 77\u201382."},{"key":"e_1_3_3_21_2","first-page":"22","volume-title":"Proceedings of the International Workshop on Sharable Natural Language Resources","author":"Kurohashi Sadao","year":"1994","unstructured":"Sadao Kurohashi, Toshihisa Nakamura, Yuji Matsumoto, and Makoto Nagao. 1994. Improvements of Japanese morphological analyzer JUMAN. In Proceedings of the International Workshop on Sharable Natural Language Resources. 22\u201328."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.210"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00343"},{"key":"e_1_3_3_25_2","first-page":"3683","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Mao Zhuoyuan","year":"2020","unstructured":"Zhuoyuan Mao, Fabien Cromieres, Raj Dabre, Haiyue Song, and Sadao Kurohashi. 2020. JASS: Japanese-specific sequence to sequence pre-training for neural machine translation. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, Marseille, France, 3683\u20133691."},{"key":"e_1_3_3_26_2","volume-title":"Conference Track Proceedings of the 6th International Conference on Learning Representations (ICLR 2018) (Vancouver, BC, Canada, April 30 - May 3, 2018)","author":"Micikevicius Paulius","year":"2018","unstructured":"Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory F. Diamos, Erich Elsen, David Garc\u00eda, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2018. Mixed precision training. In Conference Track Proceedings of the 6th International Conference on Learning Representations (ICLR 2018) (Vancouver, BC, Canada, April 30 - May 3, 2018)."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1276"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1387"},{"key":"e_1_3_3_29_2","first-page":"1","volume-title":"Proceedings of the 2nd Workshop on Asian Translation (WAT2015)","author":"Nakazawa Toshiaki","year":"2015","unstructured":"Toshiaki Nakazawa, Hideya Mino, Isao Goto, Graham Neubig, Sadao Kurohashi, and Eiichiro Sumita. 2015. Overview of the 2nd workshop on Asian translation. In Proceedings of the 2nd Workshop on Asian Translation (WAT2015). Workshop on Asian Translation, Kyoto, Japan, 1\u201328."},{"key":"e_1_3_3_30_2","volume-title":"Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation: 5th Workshop on Asian Translation","author":"Nakazawa Toshiaki","year":"2018","unstructured":"Toshiaki Nakazawa, Katsuhito Sudoh, Shohei Higashiyama, Chenchen Ding, Raj Dabre, Hideya Mino, Isao Goto, Win Pa Pa, Anoop Kunchukuttan, and Sadao Kurohashi. 2018. Overview of the 5th workshop on Asian translation. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation: 5th Workshop on Asian Translation. Association for Computational Linguistics, Hong Kong."},{"key":"e_1_3_3_31_2","first-page":"2204","volume-title":"Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916)","author":"Nakazawa Toshiaki","year":"2016","unstructured":"Toshiaki Nakazawa, Manabu Yaguchi, Kiyotaka Uchimoto, Masao Utiyama, Eiichiro Sumita, Sadao Kurohashi, and Hitoshi Isahara. 2016. ASPEC: Asian scientific paper excerpt corpus. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC\u201916). European Language Resources Association (ELRA), Portoro\u017e, Slovenia, 2204\u20132208."},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_3_33_2","first-page":"49","volume-title":"Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation: Oral Papers","author":"Park Jungyeul","year":"2016","unstructured":"Jungyeul Park, Jeen-Pyo Hong, and Jeong-Won Cha. 2016. Korean language resources for everyone. In Proceedings of the 30th Pacific Asia Conference on Language, Information and Computation: Oral Papers. Seoul, South Korea, 49\u201358."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.5555\/42293"},{"key":"e_1_3_3_36_2","volume-title":"Head-Driven Phrase Structure Grammar","author":"Pollard Carl","year":"1994","unstructured":"Carl Pollard and Ivan A. Sag. 1994. Head-Driven Phrase Structure Grammar. The University of Chicago Press, Chicago, IL."},{"key":"e_1_3_3_37_2","first-page":"2401","volume-title":"Findings of the Association for Computational Linguistics (EMNLP 2020)","author":"Qi Weizhen","year":"2020","unstructured":"Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, and Ming Zhou. 2020. ProphetNet: Predicting future N-gram for sequence-to-sequence pre-training. In Findings of the Association for Computational Linguistics (EMNLP 2020). Association for Computational Linguistics, Online, 2401\u20132410."},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2084"},{"key":"e_1_3_3_39_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. https:\/\/s3-us-west-2.amazonaws.com\/openai-assets\/research-covers\/language-unsupervised\/language_understanding_paper.pdf."},{"key":"e_1_3_3_40_2","first-page":"140:1\u2013140:67","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21 (2020), 140:1\u2013140:67.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1071"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2209"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1009"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1021"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.252"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-srw.37"},{"key":"e_1_3_3_48_2","first-page":"5926","volume-title":"Proceedings of the 36th International Conference on Machine Learning (ICML 2019) (9-15 June 2019, Long Beach, CA)","author":"Song Kaitao","year":"2019","unstructured":"Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. MASS: Masked sequence to sequence pre-training for language generation. In Proceedings of the 36th International Conference on Machine Learning (ICML 2019) (9-15 June 2019, Long Beach, CA). 5926\u20135936."},{"key":"e_1_3_3_49_2","first-page":"8968","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), The 32nd Innovative Applications of Artificial Intelligence Conference (IAAI 2020), The 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2020) (New York, NY, February 7-12, 2020)","author":"Sun Yu","year":"2020","unstructured":"Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. 2020. ERNIE 2.0: A continual pre-training framework for language understanding. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), The 32nd Innovative Applications of Artificial Intelligence Conference (IAAI 2020), The 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2020) (New York, NY, February 7-12, 2020). 8968\u20138975."},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.5555\/2969033.2969173"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1412"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.75"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454804"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.208"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-5410"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1160"},{"key":"e_1_3_3_58_2","first-page":"9628","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI 2020), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2020). (New York, NY, February 7-12, 2020)","author":"Zhang Zhuosheng","year":"2020","unstructured":"Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou. 2020. Semantics-aware BERT for language understanding. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), the 32nd Innovative Applications of Artificial Intelligence Conference (IAAI 2020), the 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI 2020). (New York, NY, February 7-12, 2020). 9628\u20139635."},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1143"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.399"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1163"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491065","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3491065","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:25:06Z","timestamp":1750195506000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3491065"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,19]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7,31]]}},"alternative-id":["10.1145\/3491065"],"URL":"https:\/\/doi.org\/10.1145\/3491065","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,19]]},"assertion":[{"value":"2021-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}