{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T03:30:57Z","timestamp":1773804657895,"version":"3.50.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"2-3","license":[{"start":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T00:00:00Z","timestamp":1598918400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,3]],"date-time":"2020-09-03T00:00:00Z","timestamp":1599091200000},"content-version":"vor","delay-in-days":2,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"publisher","award":["Grant 13\/RC\/2106"],"award-info":[{"award-number":["Grant 13\/RC\/2106"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010665","name":"H2020 Marie Sklodowska-Curie Actions","doi-asserted-by":"publisher","award":["713567"],"award-info":[{"award-number":["713567"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Machine Translation"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In a translation workflow, machine translation (MT) is almost always followed by a human post-editing step, where the raw MT output is corrected to meet required quality standards. To reduce the number of errors human translators need to correct, automatic post-editing (APE) methods have been developed and deployed in such workflows. With the advances in deep learning, neural APE (NPE) systems have outranked more traditional, statistical, ones. However, the plethora of options, variables and settings, as well as the relation between NPE performance and train\/test data makes it difficult to select the most suitable approach for a given use case. In this article, we systematically analyse these different parameters with respect to NPE performance. We build an NPE \u201croadmap\u201d to trace the different decision points and train a set of systems selecting different options through the roadmap. We also propose a novel approach for APE with data augmentation. We then analyse the performance of 15 of these systems and identify the best ones. In fact, the best systems are the ones that follow the newly-proposed method. The work presented in this article follows from a collaborative project between Microsoft and the ADAPT centre. The data provided by Microsoft originates from phrase-based statistical MT (PBSMT) systems employed in production. All tested NPE systems significantly increase the translation quality, proving the effectiveness of neural post-editing in the context of a commercial translation workflow that leverages PBSMT.<\/jats:p>","DOI":"10.1007\/s10590-020-09249-7","type":"journal-article","created":{"date-parts":[[2020,9,3]],"date-time":"2020-09-03T08:03:33Z","timestamp":1599120213000},"page":"67-96","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["A roadmap to neural automatic post-editing: an empirical approach"],"prefix":"10.1007","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6300-797X","authenticated-orcid":false,"given":"Dimitar","family":"Shterionov","sequence":"first","affiliation":[]},{"given":"F\u00e9lix do","family":"Carmo","sequence":"additional","affiliation":[]},{"given":"Joss","family":"Moorkens","sequence":"additional","affiliation":[]},{"given":"Murhaf","family":"Hossari","sequence":"additional","affiliation":[]},{"given":"Joachim","family":"Wagner","sequence":"additional","affiliation":[]},{"given":"Eric","family":"Paquin","sequence":"additional","affiliation":[]},{"given":"Dag","family":"Schmidtke","sequence":"additional","affiliation":[]},{"given":"Declan","family":"Groves","sequence":"additional","affiliation":[]},{"given":"Andy","family":"Way","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,9,3]]},"reference":[{"key":"9249_CR1","doi-asserted-by":"crossref","unstructured":"Ataman D, Federico M (2018) Compositional representation of morphologically-rich input for neural machine translation. In: Proceedings of the 56th annual meeting of the association for computational linguistics (ACL 2018), July 15\u201320, 2018, Melbourne, Australia, vol 2: short papers, pp 305\u2013311","DOI":"10.18653\/v1\/P18-2049"},{"key":"9249_CR2","doi-asserted-by":"crossref","unstructured":"Bojar O, Chatterjee R, Federmann C, Graham Y, Haddow B, Huang S, Huck M, Koehn P, Liu Q, Logacheva V, Monz C, Negri M, Post M, Rubino R, Specia L, Turchi M (2017) Findings of the 2017 Conference on Machine Translation (WMT17). In: Proceedings of the second conference on machine translation (WMT 2017), September 7\u20138, 2017, Copenhagen, Denmark, vol 2: shared task papers, pp 169\u2013214","DOI":"10.18653\/v1\/W17-4717"},{"key":"9249_CR4","doi-asserted-by":"crossref","unstructured":"Chatterjee R, Negri M, Rubino R, Turchi M (2018) Findings of the WMT 2018 shared task on automatic post-editing. In: Proceedings of the 3rd conference on machine translation (WMT 2018) (shared task), October 31\u2013November 1, 2018, Brussels, Belgium, pp 710\u2013725","DOI":"10.18653\/v1\/W18-6452"},{"key":"9249_CR5","unstructured":"Crego JM, Kim J, Klein G, Rebollo A, Yang K, Senellart J, Akhanov E, Brunelle P, Coquard A, Deng Y, Enoue S, Geiss C, Johanson J, Khalsa A, Khiari R, Ko B, Kobus C, Lorieux J, Martins L, Nguyen D, Priori A, Riccardi T, Segal N, Servan C, Tiquet C, Wang B, Yang J, Zhang D, Zhou J, Zoldan P (2016) Systran\u2019s pure neural machine translation systems. CoRR abs\/1610.05540"},{"key":"9249_CR6","doi-asserted-by":"crossref","unstructured":"Creutz M, Lagus K, Virpioja S (2005) Unsupervised morphology induction using morfessor. In: Finite-state methods and natural language processing, 5th international workshop (FSMNLP 2005), revised papers, September 1\u20132, 2005, Helsinki, Finland. Lecture notes in computer science, vol 4002, pp 300\u2013301","DOI":"10.1007\/11780885_34"},{"key":"9249_CR3","doi-asserted-by":"crossref","unstructured":"do\u00a0Carmo F, Shterionov D, Wagner J, Hossari M, Paquin E, Moorkens J (2020) A review of the state-of-the-art in automatic post-editing. Mach Transl 34 (in press)","DOI":"10.1007\/s10590-020-09249-7"},{"key":"9249_CR7","doi-asserted-by":"crossref","unstructured":"Hokamp C (2017) Ensembling factored neural machine translation models for automatic post-editing and quality estimation. In: Proceedings of the second conference on machine translation (WMT 2017), September 7\u20138, 2017, Copenhagen, Denmark, pp 647\u2013654","DOI":"10.18653\/v1\/W17-4775"},{"key":"9249_CR8","doi-asserted-by":"crossref","unstructured":"Johnson M, Schuster M, Le QV, Krikun M, Wu Y, Chen Z, Thorat N, Vi\u00e9gas FB, Wattenberg M, Corrado G, Hughes M, Dean J (2017) Google\u2019s multilingual neural machine translation system: enabling zero-shot translation. Transactions of the association for computational linguistics (TACL) vol 5, pp 339\u2013351","DOI":"10.1162\/tacl_a_00065"},{"key":"9249_CR9","doi-asserted-by":"crossref","unstructured":"Junczys-Dowmunt M, Grundkiewicz R (2016) Log-linear combinations of monolingual and bilingual neural machine translation models for automatic post-editing. In: Proceedings of the first conference on machine translation (WMT 2016), August 11\u201312, 2016, Berlin, Germany, vol 2, pp 751\u2013758","DOI":"10.18653\/v1\/W16-2378"},{"key":"9249_CR10","unstructured":"Junczys-Dowmunt M, Grundkiewicz R (2017) An exploration of neural sequence-to-sequence architectures for automatic post-editing. In: Proceedings of the eighth international joint conference on natural language processing (IJCNLP 2017), November 27\u2013December 1, 2017, Taipei, Taiwan, vol 1: long papers, pp 120\u2013129"},{"key":"9249_CR11","doi-asserted-by":"crossref","unstructured":"Koehn P, Knowles R (2017) Six challenges for neural machine translation. In: Proceedings of the first workshop on neural machine translation (NMT@ACL 2017), August 4, 2017, Vancouver, Canada, pp 28\u201339","DOI":"10.18653\/v1\/W17-3204"},{"key":"9249_CR12","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1162\/tacl_a_00067","volume":"5","author":"J Lee","year":"2017","unstructured":"Lee J, Cho K, Hofmann T (2017) Fully character-level neural machine translation without explicit segmentation. Trans Assoc Comput Linguist (TACL) 5:365\u2013378","journal-title":"Trans Assoc Comput Linguist (TACL)"},{"key":"9249_CR13","doi-asserted-by":"crossref","unstructured":"Mathur P, Ueffing N, Leusch G (2017) Generating titles for millions of browse pages on an e-commerce site. In: Proceedings of the 10th international conference on natural language generation (INLG 2017), September 4\u20137, 2017, Santiago de Compostela, Spain, pp 158\u2013167","DOI":"10.18653\/v1\/W17-3525"},{"key":"9249_CR14","unstructured":"Mattoni G, Nagle P, Collantes C, Shterionov D (2017) Zero-shot translation for Indian languages with sparse data. In: Proceedings of the 16th machine translation summit (MTSummit 2017), September 18\u201322, 2017, vol 2: users and translators track, pp 1\u201310"},{"key":"9249_CR15","unstructured":"Negri M, Turchi M, Chatterjee R, Bertoldi N (2018) ESCAPE: a large-scale synthetic corpus for automatic post-editing. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), May 7\u201312, 2018, European Language Resources Association (ELRA), Miyazaki, Japan, pp 24\u201330"},{"key":"9249_CR16","unstructured":"Papineni K, Roukos S, Ward T, Zhu WJ (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on association for computational linguistics (ACL 2002), July 6\u201312, 2002, Philadelphia, Pennsylvania, USA, pp 311\u2013318"},{"key":"9249_CR17","unstructured":"Pascanu R, Mikolov T, Bengio Y (2012) Understanding the exploding gradient problem. CoRR abs\/1211.5063"},{"key":"9249_CR18","unstructured":"Poncelas A, Shterionov D, Way A, de\u00a0Buy\u00a0Wenniger GM, Passban P (2018) Investigating backtranslation in neural machine translation. In: Proceedings of the 21st annual conference of the european association for machine translation (EAMT 2018), May 28\u201330, 2018, Alacant, Spain, pp 249\u2013258"},{"key":"9249_CR19","doi-asserted-by":"crossref","unstructured":"Sennrich R, Haddow B, Birch A (2016a) Controlling politeness in neural machine translation via side constraints. In: The 2016 conference of the North American Chapter of the association for computational linguistics: human language technologies (NAACL HLT 2016), June 12\u201317, 2016. San Diego, California, USA, pp 35\u201340","DOI":"10.18653\/v1\/N16-1005"},{"key":"9249_CR20","doi-asserted-by":"crossref","unstructured":"Sennrich R, Haddow B, Birch A (2016b) Improving neural machine translation models with monolingual data. In: Proceedings of the 54th annual meeting of the association for computational linguistics, ACL 2016, Berlin, Germany, vol 1: long papers, pp 86\u201396","DOI":"10.18653\/v1\/P16-1009"},{"key":"9249_CR21","doi-asserted-by":"crossref","unstructured":"Sennrich R, Haddow B, Birch A (2016c) Neural machine translation of rare words with subword units. In: Proceedings of the 54th annual meeting of the association for computational linguistics (ACL 2016), August 7\u201312, 2016, Berlin, Germany, vol 1: long papers, pp 1715\u20131725","DOI":"10.18653\/v1\/P16-1162"},{"key":"9249_CR22","doi-asserted-by":"crossref","unstructured":"Smit P, Virpioja S, Gr\u00f6nroos S, Kurimo M (2014) Morfessor 2.0: Toolkit for statistical morphological segmentation. In: Proceedings of the 14th conference of the european chapter of the association for computational linguistics (EACL (2014) April 26\u201330, 2014. Gothenburg, Sweden, pp 21\u201324","DOI":"10.3115\/v1\/E14-2006"},{"key":"9249_CR23","unstructured":"Snover M, Dorr B, Schwartz R, Micciulla L, Makhoul J (2006) A study of translation edit rate with targeted human annotation. In: Proceedings of the 7th conference of the association for machine translation of the Americas (AMTA 2006) visions for the future of machine translation, August 8\u201312, 2006. Massachusetts, USA, Cambridge, pp 223\u2013231"},{"key":"9249_CR24","unstructured":"Toral A (2019) Post-editese: an exacerbated translationese. In: Proceedings of machine translation summit XVII (MTSummit 2019), August 19\u201323, 2019, Dublin, Ireland, vol 1: research track, pp 273\u2013281"},{"key":"9249_CR25","doi-asserted-by":"crossref","unstructured":"Vanmassenhove E, Hardmeier C, Way A (2018) Getting gender right in neural MT. In: Proceedings of the 2018 conference on empirical methods in natural language processing (EMNLP2018), October 31\u2013 November 4, 2018, Brussels, Belgium, pp 3003\u20133008","DOI":"10.18653\/v1\/D18-1334"},{"key":"9249_CR26","unstructured":"Vanmassenhove E, Shterionov D, Way A (2019) Lost in translation: loss and decay of linguistic richness in machine translation. In: Proceedings of machine translation summit XVII (MTSummit 2019), August 19\u201323, 2019, Vol 1: research track, Dublin, Ireland, pp 222\u2013232"},{"key":"9249_CR27","doi-asserted-by":"crossref","unstructured":"Varis D, Bojar O (2017) CUNI system for WMT17 automatic post-editing task. In: Proceedings of the second conference on machine translation (WMT 2017), September 7\u20138, 2017, Copenhagen, Denmark, pp 661\u2013666","DOI":"10.18653\/v1\/W17-4777"}],"container-title":["Machine Translation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10590-020-09249-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10590-020-09249-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10590-020-09249-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,3]],"date-time":"2021-09-03T16:35:00Z","timestamp":1630686900000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10590-020-09249-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9]]},"references-count":27,"journal-issue":{"issue":"2-3","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["9249"],"URL":"https:\/\/doi.org\/10.1007\/s10590-020-09249-7","relation":{},"ISSN":["0922-6567","1573-0573"],"issn-type":[{"value":"0922-6567","type":"print"},{"value":"1573-0573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9]]},"assertion":[{"value":"2 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 July 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 September 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}