{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:33:27Z","timestamp":1762432407776,"version":"3.41.0"},"reference-count":80,"publisher":"Association for Computing Machinery (ACM)","issue":"9","license":[{"start":{"date-parts":[[2023,9,22]],"date-time":"2023-09-22T00:00:00Z","timestamp":1695340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2023,9,30]]},"abstract":"<jats:p>Neural approaches, which are currently state-of-the-art in many areas, have contributed significantly to the exciting advancements in machine translation. However, Neural Machine Translation (NMT) requires a substantial quantity and good quality parallel training data to train the best model. A large amount of training data, in turn, increases the underlying vocabulary exponentially. Therefore, several proposed methods have been devised for relatively limited vocabulary due to constraints of computing resources such as system memory. Encoding words as sequences of subword units for so-called open-vocabulary translation is an effective strategy for solving this problem. However, the conventional methods for splitting words into subwords focus on statistics-based approaches that mainly conform to agglutinative languages. In these languages, the morphemes have relatively clean boundaries. These methods still need to be thoroughly investigated for their applicability to fusion languages, which is the main focus of this article. Phonological and orthographic processes alter the borders of constituent morphemes of a word in fusion languages. Therefore, it makes it difficult to distinguish the actual morphemes that carry syntactic or semantic information from the word\u2019s surface form, the form of the word as it appears in the text. We, thus, resorted to a word segmentation method that segments words by restoring the altered morphemes. We also compared conventional and morpheme-based NMT subword models. We could prove that morpheme-based models outperform conventional subword models on a benchmark dataset.<\/jats:p>","DOI":"10.1145\/3610773","type":"journal-article","created":{"date-parts":[[2023,7,28]],"date-time":"2023-07-28T12:07:41Z","timestamp":1690546061000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Morpheme-Based Neural Machine Translation Models for Low-Resource Fusion Languages"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3424-755X","authenticated-orcid":false,"given":"Andargachew Mekonnen","family":"Gezmu","sequence":"first","affiliation":[{"name":"Otto von Guericke Universit\u00e4t Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4311-0624","authenticated-orcid":false,"given":"Andreas","family":"N\u00fcrnberger","sequence":"additional","affiliation":[{"name":"Otto von Guericke Universit\u00e4t Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,22]]},"reference":[{"key":"e_1_3_6_2_2","series-title":"Proceedings of the 9th International Conference on Text, Speech and Dialogue, TSD 2006, Brno, Czech Republic, September 11-15, 2006, Proceedings","first-page":"327","volume":"4188","author":"Allison Ben","year":"2006","unstructured":"Ben Allison, David Guthrie, and Louise Guthrie. 2006. Another look at the data sparsity problem. In Proceedings of the 9th International Conference on Text, Speech and Dialogue, TSD 2006, Brno, Czech Republic, September 11-15, 2006, Proceedings(Lecture Notes in Computer Science, Vol. 4188). Springer, 327\u2013334. 10.1007\/11846406_41"},{"key":"e_1_3_6_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.304"},{"key":"e_1_3_6_4_2","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1515\/pralin-2017-0031","article-title":"Linguistically motivated vocabulary reduction for neural machine translation from Turkish to English","volume":"108","author":"Ataman Duygu","year":"2017","unstructured":"Duygu Ataman, Matteo Negri, Marco Turchi, and Marcello Federico. 2017. Linguistically motivated vocabulary reduction for neural machine translation from Turkish to English. Prague Bull. Math. Linguistics 108 (2017), 331\u2013342. http:\/\/ufal.mff.cuni.cz\/pbml\/108\/art-ataman-negri-turchi-federico.pdf","journal-title":"Prague Bull. Math. Linguistics"},{"key":"e_1_3_6_5_2","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1409.0473"},{"key":"e_1_3_6_6_2","first-page":"840","volume-title":"Proceedings of the 13th Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20-25 June 2022","author":"Batsuren Khuyagbaatar","year":"2022","unstructured":"Khuyagbaatar Batsuren, Omer Goldman, Salam Khalifa, Nizar Habash, Witold Kieras, G\u00e1bor Bella, Brian Leonard, Garrett Nicolai, Kyle Gorman, Yustinus Ghanggo Ate, Maria Ryskina, Sabrina J. Mielke, Elena Budianskaya, Charbel El-Khaissi, Tiago Pimentel, Michael Gasser, William Abbott Lane, Mohit Raj, Matt Coler, Jaime Rafael Montoya Samame, Delio Siticonatzi Camaiteri, Esa\u00fa Zumaeta Rojas, Didier L\u00f3pez Francis, Arturo Oncevay, Juan L\u00f3pez Bautista, Gema Celeste Silva Villegas, Lucas Torroba Hennigen, Adam Ek, David Guriel, Peter Dirix, Jean-Philippe Bernardy, Andrey Scherbakov, Aziyana Bayyr-ool, Antonios Anastasopoulos, Roberto Zariquiey, Karina Sheifer, Sofya Ganieva, Hilaria Cruz, Ritv\u00e1n Karah\u00f3ga, Stella Markantonatou, George Pavlidis, Matvey Plugaryov, Elena Klyachko, Ali Salehi, Candy Angulo, Jatayu Baxi, Andrew Krizhanovsky, Natalia Krizhanovskaya, Elizabeth Salesky, Clara Vania, Sardana Ivanova, Jennifer White, Rowan Hall Maudslay, Josef Valvoda, Ran Zmigrod, Paula Czarnowska, Irene Nikkarinen, Aelita Salchak, Brijesh Bhatt, Christopher Straughn, Zoey Liu, Jonathan North Washington, Yuval Pinter, Duygu Ataman, Marcin Wolinski, Totok Suhardijanto, Anna Yablonskaya, Niklas Stoehr, Hossep Dolatian, Zahroh Nuriah, Shyam Ratan, Francis M. Tyers, Edoardo M. Ponti, Grant Aiton, Aryaman Arora, Richard J. Hatcher, Ritesh Kumar, Jeremiah Young, Daria Rodionova, Anastasia Yemelina, Taras Andrushko, Igor Marchenko, Polina Mashkovtseva, Alexandra Serova, Emily Prud\u2019hommeaux, Maria Nepomniashchaya, Fausto Giunchiglia, Eleanor Chodroff, Mans Hulden, Miikka Silfverberg, Arya D. McCarthy, David Yarowsky, Ryan Cotterell, Reut Tsarfaty, and Ekaterina Vylomova. 2022. UniMorph 4.0: Universal morphology. In Proceedings of the 13th Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20-25 June 2022. European Language Resources Association, 840\u2013855. https:\/\/aclanthology.org\/2022.lrec-1.89"},{"key":"e_1_3_6_7_2","article-title":"Finite-state morphology: Xerox tools and techniques","author":"Beesley Kenneth R.","year":"2003","unstructured":"Kenneth R. Beesley and Lauri Karttunen. 2003. Finite-state morphology: Xerox tools and techniques. CSLI, Stanford (2003).","journal-title":"CSLI, Stanford"},{"key":"e_1_3_6_8_2","first-page":"257","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1\u20134, 2016","author":"Bentivogli Luisa","year":"2016","unstructured":"Luisa Bentivogli, Arianna Bisazza, Mauro Cettolo, and Marcello Federico. 2016. Neural versus phrase-based machine translation quality: A case study. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1\u20134, 2016. The Association for Computational Linguistics, 257\u2013267. 10.18653\/v1\/d16-1025"},{"key":"e_1_3_6_9_2","article-title":"The word entropy of natural languages","volume":"1606","author":"Bentz Christian","year":"2016","unstructured":"Christian Bentz and Dimitrios Alikaniotis. 2016. The word entropy of natural languages. CoRR abs\/1606.06996 (2016). arXiv:1606.06996. http:\/\/arxiv.org\/abs\/1606.06996","journal-title":"CoRR"},{"key":"e_1_3_6_10_2","doi-asserted-by":"publisher","DOI":"10.3390\/e19060275"},{"key":"e_1_3_6_11_2","volume-title":"ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006","author":"Bird Steven","year":"2006","unstructured":"Steven Bird. 2006. NLTK: The natural language toolkit. In ACL 2006, 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Sydney, Australia, 17-21 July 2006. The Association for Computer Linguistics. 10.3115\/1225403.1225421"},{"key":"e_1_3_6_12_2","series-title":"Findings of ACL","first-page":"4617","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020","author":"Bostrom Kaj","year":"2020","unstructured":"Kaj Bostrom and Greg Durrett. 2020. Byte pair encoding is suboptimal for language model pretraining. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020(Findings of ACL, Vol. EMNLP 2020). Association for Computational Linguistics, 4617\u20134624. 10.18653\/v1\/2020.findings-emnlp.414"},{"key":"e_1_3_6_13_2","volume-title":"Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2006, April 3-7, 2006, Trento, Italy","author":"Callison-Burch Chris","year":"2006","unstructured":"Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluation the role of Bleu in machine translation research. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2006, April 3-7, 2006, Trento, Italy. The Association for Computer Linguistics. https:\/\/aclanthology.org\/E06-1032\/"},{"key":"e_1_3_6_14_2","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1515\/pralin-2017-0013","article-title":"Is neural machine translation the new state of the art?","volume":"108","author":"Castilho Sheila","year":"2017","unstructured":"Sheila Castilho, Joss Moorkens, Federico Gaspari, Iacer Calixto, John Tinsley, and Andy Way. 2017. Is neural machine translation the new state of the art? Prague Bull. Math. Linguistics 108 (2017), 109\u2013120. http:\/\/ufal.mff.cuni.cz\/pbml\/108\/art-castilho-moorkens-gaspari-tinsley-calixto-way.pdf","journal-title":"Prague Bull. Math. Linguistics"},{"key":"e_1_3_6_15_2","first-page":"4295","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018","author":"Cherry Colin","year":"2018","unstructured":"Colin Cherry, George F. Foster, Ankur Bapna, Orhan Firat, and Wolfgang Macherey. 2018. Revisiting character-based neural machine translation with capacity and compression. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 4295\u20134305. https:\/\/aclanthology.org\/D18-1461\/"},{"key":"e_1_3_6_16_2","first-page":"103","volume-title":"Proceedings of SSST@EMNLP 2014, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 25 October 2014","author":"Cho Kyunghyun","year":"2014","unstructured":"Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST@EMNLP 2014, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 25 October 2014. Association for Computational Linguistics, 103\u2013111. 10.3115\/v1\/W14-4012"},{"key":"e_1_3_6_17_2","volume-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers","author":"Costa-juss\u00e0 Marta R.","year":"2016","unstructured":"Marta R. Costa-juss\u00e0 and Jos\u00e9 A. R. Fonollosa. 2016. Character-based neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 2: Short Papers. The Association for Computer Linguistics. 10.18653\/v1\/p16-2058"},{"key":"e_1_3_6_18_2","first-page":"2325","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016","author":"Cotterell Ryan","year":"2016","unstructured":"Ryan Cotterell, Arun Kumar, and Hinrich Sch\u00fctze. 2016. Morphological segmentation inside-out. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016. The Association for Computational Linguistics, 2325\u20132330. 10.18653\/v1\/d16-1256"},{"key":"e_1_3_6_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1217098.1217101"},{"key":"e_1_3_6_20_2","volume-title":"Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation: 5th Workshop on Asian Translation, WAT@PACLIC 2018, Hong Kong, December 1-3, 2018","author":"Dabre Raj","year":"2018","unstructured":"Raj Dabre, Anoop Kunchukuttan, Atsushi Fujita, and Eiichiro Sumita. 2018. NICT\u2019s participation in WAT 2018: Approaches using multilingualism and recurrently stacked layers. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation: 5th Workshop on Asian Translation, WAT@PACLIC 2018, Hong Kong, December 1-3, 2018. Association for Computational Linguistics. https:\/\/aclanthology.org\/Y18-3003\/"},{"key":"e_1_3_6_21_2","first-page":"18","volume-title":"Proceedings of the 1st Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017","author":"Denkowski Michael J.","year":"2017","unstructured":"Michael J. Denkowski and Graham Neubig. 2017. Stronger baselines for trustable results in neural machine translation. In Proceedings of the 1st Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017. Association for Computational Linguistics, 18\u201327. 10.18653\/v1\/w17-3203"},{"key":"e_1_3_6_22_2","first-page":"126","volume-title":"Proceedings of the 5th Conference on Machine Translation, WMT@EMNLP 2020, Online, November 19-20, 2020","author":"Dhar Prajit","year":"2020","unstructured":"Prajit Dhar, Arianna Bisazza, and Gertjan van Noord. 2020. Linguistically motivated subwords for English-Tamil translation: University of Groningen\u2019s submission to WMT-2020. In Proceedings of the 5th Conference on Machine Translation, WMT@EMNLP 2020, Online, November 19-20, 2020. Association for Computational Linguistics, 126\u2013133. https:\/\/aclanthology.org\/2020.wmt-1.9\/"},{"key":"e_1_3_6_23_2","first-page":"204","volume-title":"Proceedings of Machine Translation Summit XVII Volume 1: Research Track, MTSummit 2019, Dublin, Ireland, August 19-23, 2019","author":"Ding Shuoyang","year":"2019","unstructured":"Shuoyang Ding, Adithya Renduchintala, and Kevin Duh. 2019. A call for prudent choice of subword merge operations in neural machine translation. In Proceedings of Machine Translation Summit XVII Volume 1: Research Track, MTSummit 2019, Dublin, Ireland, August 19-23, 2019. European Association for Machine Translation, 204\u2013213. https:\/\/aclanthology.org\/W19-6620\/"},{"key":"e_1_3_6_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0065-2458(08)60282-X"},{"key":"e_1_3_6_25_2","doi-asserted-by":"publisher","DOI":"10.2200\/S00994ED1V01Y202002HLT045"},{"key":"e_1_3_6_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-4541-9"},{"key":"e_1_3_6_27_2","first-page":"7112","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020","author":"Eskander Ramy","year":"2020","unstructured":"Ramy Eskander, Francesca Callejas, Elizabeth Nichols, Judith Klavans, and Smaranda Muresan. 2020. MorphAGram, evaluation and framework for unsupervised morphological segmentation. In Proceedings of the 12th Language Resources and Evaluation Conference, LREC 2020, Marseille, France, May 11-16, 2020. European Language Resources Association, 7112\u20137122. https:\/\/aclanthology.org\/2020.lrec-1.879\/"},{"key":"e_1_3_6_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-45358-8_1"},{"key":"e_1_3_6_29_2","first-page":"733","volume-title":"Proceedings of the 6th Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10-11, 2021","author":"Freitag Markus","year":"2021","unstructured":"Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, George F. Foster, Alon Lavie, and Ondrej Bojar. 2021. Results of the WMT21 metrics shared task: Evaluating metrics with expert-based human evaluations on TED and news domain. In Proceedings of the 6th Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10-11, 2021. Association for Computational Linguistics, 733\u2013774. https:\/\/aclanthology.org\/2021.wmt-1.73"},{"issue":"2","key":"e_1_3_6_30_2","first-page":"23","article-title":"A new algorithm for data compression","volume":"12","author":"Gage Philip","year":"1994","unstructured":"Philip Gage. 1994. A new algorithm for data compression. C Users Journal 12, 2 (1994), 23\u201338.","journal-title":"C Users Journal"},{"key":"e_1_3_6_31_2","volume-title":"Conference on Human Language Technology for Development, Alexandria, Egypt","author":"Gasser Michael","year":"2011","unstructured":"Michael Gasser. 2011. HornMorpho: A system for morphological processing of Amharic, Oromo, and Tigrinya. In Conference on Human Language Technology for Development, Alexandria, Egypt."},{"key":"e_1_3_6_32_2","doi-asserted-by":"publisher","DOI":"10.5220\/0010383905260532"},{"key":"e_1_3_6_33_2","first-page":"6644","volume-title":"Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022)","author":"Gezmu Andargachew Mekonnen","year":"2022","unstructured":"Andargachew Mekonnen Gezmu, Andreas N\u00fcrnberger, and Tesfaye Bayu Bati. 2022. Extended parallel corpus for Amharic-English machine translation. In Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022). European Language Resources Association, Marseille, France, 6644\u20136653. http:\/\/www.lrec-conf.org\/proceedings\/lrec2022\/pdf\/2022.lrec-1.716.pdf"},{"key":"e_1_3_6_34_2","first-page":"65","volume-title":"Proceedings of the 1st Workshop on Linguistic Resources for Natural Language Processing","author":"Gezmu Andargachew Mekonnen","year":"2018","unstructured":"Andargachew Mekonnen Gezmu, Binyam Ephrem Seyoum, Michael Gasser, and Andreas N\u00fcrnberger. 2018. Contemporary Amharic corpus: Automatically morpho-syntactically tagged Amharic corpus. In Proceedings of the 1st Workshop on Linguistic Resources for Natural Language Processing. Association for Computational Linguistics, Santa Fe, New Mexico, USA, 65\u201370. https:\/\/aclanthology.org\/W18-3809"},{"key":"e_1_3_6_35_2","series-title":"Findings of ACL","first-page":"3955","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020","author":"Gowda Thamme","year":"2020","unstructured":"Thamme Gowda and Jonathan May. 2020. Finding the optimal vocabulary size for neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020(Findings of ACL, Vol. EMNLP 2020). Association for Computational Linguistics, 3955\u20133964. 10.18653\/v1\/2020.findings-emnlp.352"},{"key":"e_1_3_6_36_2","doi-asserted-by":"crossref","first-page":"162","DOI":"10.18653\/v1\/2020.acl-srw.22","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, ACL 2020, Online, July 5-10, 2020","author":"Goyal Vikrant","year":"2020","unstructured":"Vikrant Goyal, Sourav Kumar, and Dipti Misra Sharma. 2020. Efficient neural machine translation for low-resource languages via exploiting related languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, 162\u2013168. 10.18653\/v1\/2020.acl-srw.22"},{"key":"e_1_3_6_37_2","first-page":"344","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)","author":"Gu Jiatao","year":"2018","unstructured":"Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O. K. Li. 2018. Universal neural machine translation for extremely low resource languages. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers). Association for Computational Linguistics, 344\u2013354. 10.18653\/v1\/n18-1032"},{"key":"e_1_3_6_38_2","first-page":"3454","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19-23, 2021","author":"Gutierrez-Vasques Ximena","year":"2021","unstructured":"Ximena Gutierrez-Vasques, Christian Bentz, Olga Sozinova, and Tanja Samardzic. 2021. From characters to words: The turning point of BPE merges. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19-23, 2021. Association for Computational Linguistics, 3454\u20133468. 10.18653\/v1\/2021.eacl-main.302"},{"key":"e_1_3_6_39_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00446"},{"key":"e_1_3_6_40_2","doi-asserted-by":"publisher","DOI":"10.1515\/LINGTY.2007.011"},{"key":"e_1_3_6_41_2","first-page":"56","volume-title":"Proceedings of the 2nd Conference on Machine Translation, WMT 2017, Copenhagen, Denmark, September 7-8, 2017","author":"Huck Matthias","year":"2017","unstructured":"Matthias Huck, Simon Riess, and Alexander M. Fraser. 2017. Target-side word segmentation strategies for neural machine translation. In Proceedings of the 2nd Conference on Machine Translation, WMT 2017, Copenhagen, Denmark, September 7-8, 2017. Association for Computational Linguistics, 56\u201367. 10.18653\/v1\/w17-4706"},{"key":"e_1_3_6_42_2","first-page":"1","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers","author":"Jean S\u00e9bastien","year":"2015","unstructured":"S\u00e9bastien Jean, KyungHyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. The Association for Computer Linguistics, 1\u201310. 10.3115\/v1\/p15-1001"},{"key":"e_1_3_6_43_2","volume-title":"Speech and Language Processing (2nd Edition)","author":"Jurafsky Daniel","year":"2009","unstructured":"Daniel Jurafsky and James H. Martin. 2009. Speech and Language Processing (2nd Edition). Prentice-Hall, Inc., USA."},{"key":"e_1_3_6_44_2","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_6_45_2","first-page":"478","volume-title":"Proceedings of the 6th Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10-11, 2021","author":"Kocmi Tom","year":"2021","unstructured":"Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, and Arul Menezes. 2021. To ship or not to ship: An extensive evaluation of automatic metrics for machine translation. In Proceedings of the 6th Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10-11, 2021. Association for Computational Linguistics, 478\u2013494. https:\/\/aclanthology.org\/2021.wmt-1.57"},{"key":"e_1_3_6_46_2","first-page":"388","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, EMNLP 2004, A meeting of SIGDAT, a Special Interest Group of the ACL, held in conjunction with ACL 2004, 25-26 July 2004, Barcelona, Spain","author":"Koehn Philipp","year":"2004","unstructured":"Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, EMNLP 2004, A meeting of SIGDAT, a Special Interest Group of the ACL, held in conjunction with ACL 2004, 25-26 July 2004, Barcelona, Spain. ACL, 388\u2013395. https:\/\/aclanthology.org\/W04-3250\/"},{"key":"e_1_3_6_47_2","volume-title":"Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, ACL 2007, June 23-30, 2007, Prague, Czech Republic","author":"Koehn Philipp","year":"2007","unstructured":"Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, ACL 2007, June 23-30, 2007, Prague, Czech Republic. The Association for Computational Linguistics. https:\/\/aclanthology.org\/P07-2045\/"},{"key":"e_1_3_6_48_2","first-page":"28","volume-title":"Proceedings of the 1st Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017","author":"Koehn Philipp","year":"2017","unstructured":"Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the 1st Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017. Association for Computational Linguistics, 28\u201339. 10.18653\/v1\/w17-3204"},{"key":"e_1_3_6_49_2","first-page":"66","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers","author":"Kudo Taku","year":"2018","unstructured":"Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers. Association for Computational Linguistics, 66\u201375. 10.18653\/v1\/P18-1007"},{"key":"e_1_3_6_50_2","first-page":"66","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 - November 4, 2018","author":"Kudo Taku","year":"2018","unstructured":"Taku Kudo and John Richardson. 2018. SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018: System Demonstrations, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 66\u201371. 10.18653\/v1\/d18-2012"},{"key":"e_1_3_6_51_2","first-page":"48","volume-title":"Proceedings of the 18th Biennial Machine Translation Summit - Volume 1: Research Track, MTSummit 2021 Virtual, August 16-20, 2021","author":"Lankford Seamus","year":"2021","unstructured":"Seamus Lankford, Haithem Alfi, and Andy Way. 2021. Transformers for low-resource languages: Is F\u00e9idir linn!. In Proceedings of the 18th Biennial Machine Translation Summit - Volume 1: Research Track, MTSummit 2021 Virtual, August 16-20, 2021. Association for Machine Translation in the Americas, 48\u201360. https:\/\/aclanthology.org\/2021.mtsummit-research.5"},{"key":"e_1_3_6_52_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00067"},{"key":"e_1_3_6_53_2","first-page":"11","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers","author":"Luong Thang","year":"2015","unstructured":"Thang Luong, Ilya Sutskever, Quoc V. Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers. The Association for Computer Linguistics, 11\u201319. 10.3115\/v1\/p15-1002"},{"key":"e_1_3_6_54_2","series-title":"Text, Speech, and Dialogue - 21st International Conference, TSD 2018, Brno, Czech Republic, September 11-14, 2018, Proceedings","first-page":"277","volume":"11107","author":"Mach\u00e1cek Dominik","year":"2018","unstructured":"Dominik Mach\u00e1cek, Jon\u00e1s Vidra, and Ondrej Bojar. 2018. Morphological and language-agnostic word segmentation for NMT. In Text, Speech, and Dialogue - 21st International Conference, TSD 2018, Brno, Czech Republic, September 11-14, 2018, Proceedings(Lecture Notes in Computer Science, Vol. 11107). Springer, 277\u2013284. 10.1007\/978-3-030-00794-2_30"},{"key":"e_1_3_6_55_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.566"},{"key":"e_1_3_6_56_2","article-title":"Between words and characters: A brief history of open-vocabulary modeling and tokenization in NLP","volume":"2112","author":"Mielke Sabrina J.","year":"2021","unstructured":"Sabrina J. Mielke, Zaid Alyafeai, Elizabeth Salesky, Colin Raffel, Manan Dey, Matthias Gall\u00e9, Arun Raja, Chenglei Si, Wilson Y. Lee, Beno\u00eet Sagot, and Samson Tan. 2021. Between words and characters: A brief history of open-vocabulary modeling and tokenization in NLP. CoRR abs\/2112.10508. (2021). arXiv:2112.10508https:\/\/arxiv.org\/abs\/2112.10508","journal-title":"CoRR"},{"key":"e_1_3_6_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10590-020-09255-9"},{"key":"e_1_3_6_58_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA. ACL, 311\u2013318. 10.3115\/1073083.1073135"},{"key":"e_1_3_6_59_2","doi-asserted-by":"crossref","first-page":"43","DOI":"10.2478\/pralin-2018-0002","article-title":"Training tips for the Transformer model","volume":"110","author":"Popel Martin","year":"2018","unstructured":"Martin Popel and Ondrej Bojar. 2018. Training tips for the Transformer model. Prague Bull. Math. Linguistics 110 (2018), 43\u201370. http:\/\/ufal.mff.cuni.cz\/pbml\/110\/art-popel-bojar.pdf","journal-title":"Prague Bull. Math. Linguistics"},{"key":"e_1_3_6_60_2","doi-asserted-by":"crossref","first-page":"186","DOI":"10.18653\/v1\/W18-6319","volume-title":"Proceedings of the 3rd Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018","author":"Post Matt","year":"2018","unstructured":"Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the 3rd Conference on Machine Translation: Research Papers, WMT 2018, Belgium, Brussels, October 31 - November 1, 2018. Association for Computational Linguistics, 186\u2013191. 10.18653\/v1\/w18-6319"},{"key":"e_1_3_6_61_2","first-page":"2685","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020","author":"Rei Ricardo","year":"2020","unstructured":"Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020. Association for Computational Linguistics, 2685\u20132702. 10.18653\/v1\/2020.emnlp-main.213"},{"key":"e_1_3_6_62_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00322"},{"key":"e_1_3_6_63_2","doi-asserted-by":"publisher","DOI":"10.1142\/0822"},{"key":"e_1_3_6_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10590-019-09243-8"},{"key":"e_1_3_6_65_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.eacl-srw.22"},{"issue":"2","key":"e_1_3_6_66_2","doi-asserted-by":"crossref","first-page":"155","DOI":"10.2307\/2967612","article-title":"A note on measurement of utility","volume":"4","author":"Samuelson Paul A.","year":"1937","unstructured":"Paul A. Samuelson. 1937. A note on measurement of utility. The Review of Economic Studies 4, 2 (1937), 155\u2013161.","journal-title":"The Review of Economic Studies"},{"key":"e_1_3_6_67_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/w19-5339"},{"key":"e_1_3_6_68_2","first-page":"5149","volume-title":"2012 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2012, Kyoto, Japan, March 25-30, 2012","author":"Schuster Mike","year":"2012","unstructured":"Mike Schuster and Kaisuke Nakajima. 2012. Japanese and Korean voice search. In 2012 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2012, Kyoto, Japan, March 25-30, 2012. IEEE, 5149\u20135152. 10.1109\/ICASSP.2012.6289079"},{"key":"e_1_3_6_69_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p16-1162"},{"key":"e_1_3_6_70_2","first-page":"211","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers","author":"Sennrich Rico","year":"2019","unstructured":"Rico Sennrich and Biao Zhang. 2019. Revisiting low-resource neural machine translation: A case study. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 211\u2013221. 10.18653\/v1\/p19-1021"},{"key":"e_1_3_6_71_2","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"e_1_3_6_72_2","first-page":"21","volume-title":"Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, April 26-30, 2014, Gothenburg, Sweden","author":"Smit Peter","year":"2014","unstructured":"Peter Smit, Sami Virpioja, Stig-Arne Gr\u00f6nroos, and Mikko Kurimo. 2014. Morfessor 2.0: Toolkit for statistical morphological segmentation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, April 26-30, 2014, Gothenburg, Sweden. The Association for Computer Linguistics, 21\u201324. 10.3115\/v1\/e14-2006"},{"key":"e_1_3_6_73_2","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_3_6_74_2","first-page":"3104","volume-title":"Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13, 2014, Montreal, Quebec, Canada","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13, 2014, Montreal, Quebec, Canada. 3104\u20133112. https:\/\/proceedings.neurips.cc\/paper\/2014\/hash\/a14ac55a4f27472c5d894ec1c3c743d2-Abstract.html"},{"key":"e_1_3_6_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_6_76_2","first-page":"386","volume-title":"Proceedings of the 4th Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers, Day 1","author":"Toral Antonio","year":"2019","unstructured":"Antonio Toral, Lukas Edman, Galiya Yeshmagambetova, and Jennifer Spenader. 2019. Neural machine translation for English-Kazakh with morphological segmentation and synthetic data. In Proceedings of the 4th Conference on Machine Translation, WMT 2019, Florence, Italy, August 1-2, 2019 - Volume 2: Shared Task Papers, Day 1. Association for Computational Linguistics, 386\u2013392. 10.18653\/v1\/w19-5343"},{"key":"e_1_3_6_77_2","first-page":"193","volume-title":"Proceedings of the 13th Conference of the Association for Machine Translation in the Americas, AMTA 2018, Boston, MA, USA, March 17-21, 2018 - Volume 1: Research Papers","author":"Vaswani Ashish","year":"2018","unstructured":"Ashish Vaswani, Samy Bengio, Eugene Brevdo, Fran\u00e7ois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Lukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, and Jakob Uszkoreit. 2018. Tensor2Tensor for neural machine translation. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas, AMTA 2018, Boston, MA, USA, March 17-21, 2018 - Volume 1: Research Papers. Association for Machine Translation in the Americas, 193\u2013199. https:\/\/aclanthology.org\/W18-1819\/"},{"key":"e_1_3_6_78_2","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 5998\u20136008. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_6_79_2","article-title":"Google\u2019s neural machine translation system: Bridging the gap between human and machine translation","volume":"1609","author":"Wu Yonghui","year":"2016","unstructured":"Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google\u2019s neural machine translation system: Bridging the gap between human and machine translation. CoRR abs\/1609.08144 (2016). arXiv:1609.08144. http:\/\/arxiv.org\/abs\/1609.08144","journal-title":"CoRR"},{"key":"e_1_3_6_80_2","first-page":"7361","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021","author":"Xu Jingjing","year":"2021","unstructured":"Jingjing Xu, Hao Zhou, Chun Gan, Zaixiang Zheng, and Lei Li. 2021. Vocabulary learning via optimal transport for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL\/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021. Association for Computational Linguistics, 7361\u20137373. 10.18653\/v1\/2021.acl-long.571"},{"key":"e_1_3_6_81_2","series-title":"Databases and Information Systems - 13th International Baltic Conference, DB&IS 2018, Trakai, Lithuania, July 1-4, 2018, Proceedings","first-page":"289","volume":"838","author":"Zuters Janis","year":"2018","unstructured":"Janis Zuters, Gus Strazds, and Karlis Immers. 2018. Semi-automatic quasi-morphological word segmentation for neural machine translation. In Databases and Information Systems - 13th International Baltic Conference, DB&IS 2018, Trakai, Lithuania, July 1-4, 2018, Proceedings(Communications in Computer and Information Science, Vol. 838). Springer, 289\u2013301. 10.1007\/978-3-319-97571-9_23"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610773","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3610773","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:11Z","timestamp":1750178231000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3610773"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,22]]},"references-count":80,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,9,30]]}},"alternative-id":["10.1145\/3610773"],"URL":"https:\/\/doi.org\/10.1145\/3610773","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"type":"print","value":"2375-4699"},{"type":"electronic","value":"2375-4702"}],"subject":[],"published":{"date-parts":[[2023,9,22]]},"assertion":[{"value":"2022-07-05","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}