{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T13:44:16Z","timestamp":1784382256241,"version":"3.55.0"},"reference-count":67,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T00:00:00Z","timestamp":1617321600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["13\/RC\/2106"],"award-info":[{"award-number":["13\/RC\/2106"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Digital"],"abstract":"<jats:p>Phrase-based statistical machine translation (PB-SMT) has been the dominant paradigm in machine translation (MT) research for more than two decades. Deep neural MT models have been producing state-of-the-art performance across many translation tasks for four to five years. To put it another way, neural MT (NMT) took the place of PB-SMT a few years back and currently represents the state-of-the-art in MT research. Translation to or from under-resourced languages has been historically seen as a challenging task. Despite producing state-of-the-art results in many translation tasks, NMT still poses many problems such as performing poorly for many low-resource language pairs mainly because of its learning task\u2019s data-demanding nature. MT researchers have been trying to address this problem via various techniques, e.g., exploiting source- and\/or target-side monolingual data for training, augmenting bilingual training data, and transfer learning. Despite some success, none of the present-day benchmarks have entirely overcome the problem of translation in low-resource scenarios for many languages. In this work, we investigate the performance of PB-SMT and NMT on two rarely tested under-resourced language pairs, English-To-Tamil and Hindi-To-Tamil, taking a specialised data domain into consideration. This paper demonstrates our findings and presents results showing the rankings of our MT systems produced via a social media-based human evaluation scheme.<\/jats:p>","DOI":"10.3390\/digital1020007","type":"journal-article","created":{"date-parts":[[2021,4,2]],"date-time":"2021-04-02T04:13:51Z","timestamp":1617336831000},"page":"86-102","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Comparing Statistical and Neural Machine Translation Performance on Hindi-To-Tamil and English-To-Tamil"],"prefix":"10.3390","volume":"1","author":[{"given":"Akshai","family":"Ramesh","sequence":"first","affiliation":[{"name":"ADAPT Centre, School of Computing, Dublin City University, Dublin 9, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Venkatesh Balavadhani","family":"Parthasarathy","sequence":"additional","affiliation":[{"name":"ADAPT Centre, School of Computing, Dublin City University, Dublin 9, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1680-0099","authenticated-orcid":false,"given":"Rejwanul","family":"Haque","sequence":"additional","affiliation":[{"name":"ADAPT Centre, School of Computing, Dublin City University, Dublin 9, Ireland"},{"name":"School of Computing, National College of Ireland, Dublin 1, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5736-5930","authenticated-orcid":false,"given":"Andy","family":"Way","sequence":"additional","affiliation":[{"name":"ADAPT Centre, School of Computing, Dublin City University, Dublin 9, Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sennrich, R., Haddow, B., and Birch, A. (2016, January 7\u201312). Improving Neural Machine Translation Models with Monolingual Data. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-1009"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chen, P.J., Shen, J., Le, M., Chaudhary, V., El-Kishky, A., Wenzek, G., Ott, M., and Ranzato, M. (2019, January 4). Facebook AI\u2019s WAT19 Myanmar-English Translation Task Submission. Proceedings of the 6th Workshop on Asian Translation, Hong Kong, China.","DOI":"10.18653\/v1\/D19-5213"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Artetxe, M., Labaka, G., and Agirre, E. (November, January 31). Unsupervised Statistical Machine Translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1399"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Lample, G., Ott, M., Conneau, A., Denoyer, L., and Ranzato, M. (November, January 31). Phrase-Based & Neural Unsupervised Machine Translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium.","DOI":"10.18653\/v1\/D18-1549"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.csl.2016.10.006","article-title":"Multi-way, multilingual neural machine translation","volume":"45","author":"Firat","year":"2017","journal-title":"Comput. Speech Lang."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1162\/tacl_a_00065","article-title":"Google\u2019s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation","volume":"5","author":"Johnson","year":"2017","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Niehues, J., and Cho, E. (2017, January 7\u20138). Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning. Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4708"},{"key":"ref_8","unstructured":"Sennrich, R., and Zhang, B. (August, January 28). Revisiting Low-Resource Neural Machine Translation: A Case Study. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Liu, Y., Gu, J., Goyal, N., Li, X., Edunov, S., Ghazvininejad, M., Lewis, M., and Zettlemoyer, L. (2020). Multilingual Denoising Pre-training for Neural Machine Translation. arXiv.","DOI":"10.1162\/tacl_a_00343"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Currey, A., Miceli Barone, A.V., and Heafield, K. (2017, January 7\u20138). Copied Monolingual Data Improves Low-Resource Neural Machine Translation. Proceedings of the Second Conference on Machine Translation, Association for Computational Linguistics, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4715"},{"key":"ref_11","unstructured":"Marie, B., and Fujita, A. (2018). Unsupervised neural machine translation initialized by unsupervised statistical machine translation. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"S\u00f8gaard, A., Ruder, S., and Vuli\u0107, I. (2018, January 15\u201320). On the Limitations of Unsupervised Bilingual Dictionary Induction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Melbourne, Australia.","DOI":"10.18653\/v1\/P18-1072"},{"key":"ref_13","unstructured":"Montoya, H.E.G., Rojas, K.D.R., and Oncevay, A. (2019, January 19\u201323). A Continuous Improvement Framework of Machine Translation for Shipibo-Konibo. Proceedings of the 2nd Workshop on Technologies for MT of Low Resource Languages, Dublin, Ireland."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bentivogli, L., Bisazza, A., Cettolo, M., and Federico, M. (2016, January 1\u20134). Neural versus Phrase-Based Machine Translation Quality: A Case Study. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, USA.","DOI":"10.18653\/v1\/D16-1025"},{"key":"ref_15","unstructured":"Castilho, S., Moorkens, J., Gaspari, F., Sennrich, R., Sosoni, V., Georgakopoulou, P., Lohar, P., Way, A., Valerio, A., and Barone, M. (2017, January 18\u201322). A Comparative Quality Evaluation of PBSMT and NMT using Professional Translators. Proceedings of the MT Summit XVI, the 16th Machine Translation Summit, Nagoya, Japan."},{"key":"ref_16","unstructured":"Koehn, P., and Knowles, R. (August, January 30). Six Challenges for Neural Machine Translation. Proceedings of the First Workshop on Neural Machine Translation, Vancouver, BC, Canada."},{"key":"ref_17","unstructured":"\u00d6stling, R., and Tiedemann, J. (2017). Neural machine translation for low-resource languages. arXiv."},{"key":"ref_18","unstructured":"Dowling, M., Lynn, T., Poncelas, A., and Way, A. (2018, January 21). SMT versus NMT: Preliminary comparisons for Irish. Proceedings of the AMTA 2018 Workshop on Technologies for MT of Low Resource Languages (LoResMT 2018), Boston, MA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Casas, N., Fonollosa, J.A., Escolano, C., Basta, C., and Costa-juss\u00e0, M.R. (2019, January 1\u20132). The TALP-UPC machine translation systems for WMT19 news translation task: Pivoting techniques for low resource MT. Proceedings of the Fourth Conference on Machine Translation, Florence, Italy.","DOI":"10.18653\/v1\/W19-5311"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sen, S., Gupta, K.K., Ekbal, A., and Bhattacharyya, P. (2019, January 1\u20132). IITP-MT System for Gujarati-English News Translation Task at WMT 2019. Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), Florence, Italy.","DOI":"10.18653\/v1\/W19-5346"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ramesh, A., Parthasarathy, V.B., Haque, R., and Way, A. (2020, January 4\u20137). An Error-based Investigation of Statistical and Neural Machine Translation Performance on Hindi-To-Tamil and English-To-Tamil. Proceedings of the 7th Workshop on Asian Translation (WAT2020), Suzhou, China.","DOI":"10.20944\/preprints202012.0580.v1"},{"key":"ref_22","unstructured":"Ramesh, A., Parthasarathy, V.B., Haque, R., and Way, A. (2020, January 4\u20137). Investigating Low-resource Machine Translation for English-To-Tamil. Proceedings of the AACL-IJCNLP 2020 Workshop on Technologies for MT of Low Resource Languages (LoResMT 2020), Suzhou, China."},{"key":"ref_23","unstructured":"Junczys-Dowmunt, M., Dwojak, T., and Hoang, H. (2016). Is neural machine translation ready for deployment? A case study on 30 translation directions. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Toral, A., and S\u00e1nchez-Cartagena, V.M. (2017). A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions. arXiv.","DOI":"10.18653\/v1\/E17-1100"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1515\/pralin-2017-0021","article-title":"Comparing Language Related Issues for NMT and PBMT between German and English","volume":"108","year":"2017","journal-title":"Prague Bull. Math. Linguist."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Toral, A., and Way, A. (2018). What level of quality can Neural Machine Translation attain on literary text?. Translation Quality Assessment, Springer.","DOI":"10.1007\/978-3-319-91241-7_12"},{"key":"ref_27","unstructured":"Specia, L., Harris, K., Blain, F., Burchardt, A., Macketanz, V., Skadi\u0146a, I., Negri, M., and Turchi, M. (2017, January 18\u201322). Translation Quality and Productivity: A Study on Rich Morphology Languages. Proceedings of the MT Summit XVI, the 16th Machine Translation Summit, Nagoya, Japan."},{"key":"ref_28","unstructured":"Shterionov, D., Nagle, P., Casanellas, L., Superbo, R., and O\u2019Dowd, T. (2017, January 29\u201331). Empirical evaluation of NMT and PBSMT quality for large-scale translation production. Proceedings of the User Track of the 20th Annual Conference of the European Association for Machine Translation (EAMT), Prague, Czech Republic."},{"key":"ref_29","unstructured":"Long, Z., Utsuro, T., Mitsuhashi, T., and Yamamoto, M. (2016, January 11\u201316). Translation of Patent Sentences with a Large Vocabulary of Technical Terms Using Neural Machine Translation. Proceedings of the 3rd Workshop on Asian Translation (WAT2016), Osaka, Japan."},{"key":"ref_30","unstructured":"Kinoshita, S., Oshio, T., and Mitsuhashi, T. (December, January 27). Comparison of SMT and NMT trained with large Patent Corpora: Japio at WAT2017. Proceedings of the 4th Workshop on Asian Translation (WAT2017), Asian Federation of Natural Language Processing, Taipei, Taiwan."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Klubi\u010dka, F., Toral, A., and S\u00e1nchez-Cartagena, V.M. (2017). Fine-grained human evaluation of neural versus phrase-based machine translation. arXiv.","DOI":"10.1515\/pralin-2017-0014"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Klubi\u010dka, F., Toral, A., and S\u00e1nchez-Cartagena, V.M. (2018). Quantitative Fine-Grained Human Evaluation of Machine Translation Systems: A Case Study on English to Croatian. arXiv.","DOI":"10.1007\/s10590-018-9214-x"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Isabelle, P., Cherry, C., and Foster, G.F. (2017). A Challenge Set Approach to Evaluating Machine Translation. arXiv.","DOI":"10.18653\/v1\/D17-1263"},{"key":"ref_34","unstructured":"Beyer, A.M., Macketanz, V., Burchardt, A., and Williams, P. (2017, January 29\u201331). Can out-of-the-box NMT Beat a Domain-trained Moses on Technical Data?. Proceedings of the EAMT User Studies and Project\/Product Descriptions, Prague, Czech Republic."},{"key":"ref_35","unstructured":"Nunez, J.C.R., Seddah, D., and Wisniewski, G. (October, January 30). Comparison between NMT and PBSMT Performance for Translating Noisy User-Generated Content. Proceedings of the NEAL 22nd Nordic Conference on Computional Linguistics (NoDaLiDa), Turku, Finland."},{"key":"ref_36","unstructured":"Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., and Zens, R. (2007, January 25\u201327). Moses: Open Source Toolkit for Statistical Machine Translation. Proceedings of the ACL 2007, Proceedings of the Interactive Poster and Demonstration Sessions, Prague, Czech Republic."},{"key":"ref_37","unstructured":"Kneser, R., and Ney, H. (1995, January 9\u201312). Improved backing-off for M-gram language modeling. Proceedings of the 1995 International Conference on Acoustics Speech, and Signal Processing, Detroit, MI, USA."},{"key":"ref_38","unstructured":"Durrani, N., Schmid, H., and Fraser, A. (2011, January 21). A Joint Sequence Translation Model with Integrated Reordering. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA."},{"key":"ref_39","unstructured":"Cherry, C., and Foster, G. (2012, January 3\u20138). Batch tuning strategies for statistical machine translation. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Montreal, QC, Canada."},{"key":"ref_40","unstructured":"Huang, L., and Chiang, D. (2007, January 23\u201330). Forest Rescoring: Faster Decoding with Integrated Language Models. Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Klein, G., Kim, Y., Deng, Y., Senellart, J., and Rush, A. (\u2013, January July). OpenNMT: Open-Source Toolkit for Neural Machine Translation. Proceedings of the ACL 2017, System Demonstrations, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/P17-4012"},{"key":"ref_42","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017). Attention Is All You Need. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sennrich, R., Haddow, B., and Birch, A. (2016, January 7\u201312). Neural Machine Translation of Rare Words with Subword Units. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Berlin, Germany.","DOI":"10.18653\/v1\/P16-1162"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (, January July). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the ACL-2002: 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_45","unstructured":"Tiedemann, J. (2012, January 23\u201325). Parallel Data, Tools and Interfaces in OPUS. Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC\u20192012), Istanbul, Turkey."},{"key":"ref_46","unstructured":"Schwenk, H., Chaudhary, V., Sun, S., Gong, H., and Guzm\u00e1n, F. (2019). Wikimatrix: Mining 135m parallel sentences in 1620 language pairs from wikipedia. arXiv."},{"key":"ref_47","unstructured":"Haddow, B., and Kirefu, F. (2020). PMIndia\u2014A Collection of Parallel Corpora of Languages of India. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Popovi\u0107, M. (2015, January 17\u201318). chrF: Character n-gram F-score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation, Lisbon, Portugal.","DOI":"10.18653\/v1\/W15-3049"},{"key":"ref_49","unstructured":"Koehn, P. (2004, January 25\u201326). Statistical Significance Tests for Machine Translation Evaluation. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP), Barcelona, Spain."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Popovi\u0107, M. (2017, January 7\u20138). chrF++: Words helping character n-grams. Proceedings of the Second Conference on Machine Translation, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4770"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Angelone, E., Ehrensberger-Dow, M., and Massey, G. (2019). Machine Translation: Where are we at today?. The Bloomsbury Companion to Language Industry Studies, Bloomsbury Academic Publishing.","DOI":"10.5040\/9781350024960"},{"key":"ref_52","unstructured":"Burlot, F., and Yvon, F. (November, January 31). Using Monolingual Data in Neural Machine Translation: A Systematic Study. Proceedings of the Third Conference on Machine Translation: Research Papers, Belgium, Brussels."},{"key":"ref_53","unstructured":"Bogoychev, N., and Sennrich, R. (2019). Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation. arXiv."},{"key":"ref_54","unstructured":"Kunchukuttan, A., Kakwani, D., Golla, S., Bhattacharyya, A., Khapra, M.M., and Kumar, P. (2020). AI4Bharat-IndicNLP Corpus: Monolingual Corpora and Word Embeddings for Indic Languages. arXiv."},{"key":"ref_55","unstructured":"Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. MT Summit, Citeseer."},{"key":"ref_56","unstructured":"Dinu, G., Mathur, P., Federico, M., and Al-Onaizan, Y. (August, January 28). Training Neural Machine Translation to Apply Terminology Constraints. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Haque, R., Hasanuzzaman, M., and Way, A. (2019, January 2\u20134). Investigating Terminology Translation in Statistical and Neural Machine Translation: A Case Study on English-to-Hindi and Hindi-to-English. Proceedings of the International Conference on Recent Advances in Natural Language Processing, Varna, Bulgaria.","DOI":"10.26615\/978-954-452-056-4_052"},{"key":"ref_58","unstructured":"Exel, M., Buschbeck, B., Brandt, L., and Doneva, S. (2020, January 3\u20135). Terminology-Constrained Neural Machine Translation at SAP. Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, Lisboa, Portugal."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Farajian, M.A., Turchi, M., Negri, M., Bertoldi, N., and Federico, M. (2017, January 3\u20137). Neural vs. Phrase-Based Machine Translation in a Multi-Domain Scenario. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain. Short Papers.","DOI":"10.18653\/v1\/E17-2045"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Banerjee, T., and Bhattacharyya, P. (2018, January 6). Meaningless yet meaningful: Morphology grounded subword-level NMT. Proceedings of the Second Workshop on Subword\/Character Level Models, New Orleans, LA, USA.","DOI":"10.18653\/v1\/W18-1207"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10590-018-9220-z","article-title":"Human versus automatic quality evaluation of NMT and PBSMT","volume":"32","author":"Shterionov","year":"2018","journal-title":"Mach. Transl."},{"key":"ref_62","unstructured":"Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, Ann Arbor, MI, USA."},{"key":"ref_63","unstructured":"Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006, January 8\u201312). A study of translation edit rate with targeted human annotation. Proceedings of the Association for Machine Translation in the Americas, Cambridge, MA, USA."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Castilho, S., Moorkens, J., Gaspari, F., and Doherty, S. (2018). Quality expectations of machine translation. Translation Quality Assessment, Springer.","DOI":"10.1007\/978-3-319-91241-7"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Lommel, A.R., Uszkoreit, H., and Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics. Tradum\u00c1Tica Tecnol. Traducci\u00d3, 455\u2013463.","DOI":"10.5565\/rev\/tradumatica.77"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Rei, R., Stewart, C., Farinha, A.C., and Lavie, A. (2020). COMET: A Neural Framework for MT Evaluation. arXiv.","DOI":"10.18653\/v1\/2020.emnlp-main.213"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Ma, Q., Wei, J., Bojar, O., and Graham, Y. (2019, January 1\u20132). Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges. Proceedings of the Fourth Conference on Machine Translation (Volume 2: Shared Task Papers, Day 1), Florence, Italy.","DOI":"10.18653\/v1\/W19-5302"}],"container-title":["Digital"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-6470\/1\/2\/7\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:26:05Z","timestamp":1760361965000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-6470\/1\/2\/7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,2]]},"references-count":67,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["digital1020007"],"URL":"https:\/\/doi.org\/10.3390\/digital1020007","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202012.0580.v1","asserted-by":"object"}]},"ISSN":["2673-6470"],"issn-type":[{"value":"2673-6470","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,2]]}}}