{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T16:26:17Z","timestamp":1782923177726,"version":"3.54.5"},"reference-count":30,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,2,15]],"date-time":"2022-02-15T00:00:00Z","timestamp":1644883200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Slovenian Research Agency","doi-asserted-by":"publisher","award":["P6-0215"],"award-info":[{"award-number":["P6-0215"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Neural machine translation systems have revolutionized translation processes in terms of quantity and speed in recent years, and they have even been claimed to achieve human parity. However, the quality of their output has also raised serious doubts and concerns, such as loss in lexical variation, evidence of \u201cmachine translationese\u201d, and its effect on post-editing, which results in \u201cpost-editese\u201d. In this study, we analyze the outputs of three English to Slovenian machine translation systems in terms of lexical diversity in three different genres. Using both quantitative and qualitative methods, we analyze one statistical and two neural systems, and we compare them to a human reference translation. Our quantitative analyses based on lexical diversity metrics show diverging results; however, translation systems, particularly neural ones, mostly exhibit larger lexical diversity than their human counterparts. Nevertheless, a qualitative method shows that these quantitative results are not always a reliable tool to assess true lexical diversity and that a lot of lexical \u201ccreativity\u201d, especially by neural translation systems, is often unreliable, inconsistent, and misguided.<\/jats:p>","DOI":"10.3390\/info13020093","type":"journal-article","created":{"date-parts":[[2022,2,15]],"date-time":"2022-02-15T22:44:47Z","timestamp":1644965087000},"page":"93","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Lexical Diversity in Statistical and Neural Machine Translation"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8806-0942","authenticated-orcid":false,"given":"Mojca","family":"Brglez","sequence":"first","affiliation":[{"name":"Faculty of Arts, University of Ljubljana, A\u0161ker\u010deva Cesta 2, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1934-0200","authenticated-orcid":false,"given":"\u0160pela","family":"Vintar","sequence":"additional","affiliation":[{"name":"Faculty of Arts, University of Ljubljana, A\u0161ker\u010deva Cesta 2, 1000 Ljubljana, Slovenia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,15]]},"reference":[{"key":"ref_1","unstructured":"Hassan, H., Aue, A., Chen, C., Chowdhary, V., Clark, J., Federmann, C., Huang, X., Junczys-Dowmunt, M., Lewis, W., and Li, M. (2018). Achieving Human Parity on Automatic Chinese to English News Translation. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 6\u201312). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, Philadephia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_3","unstructured":"Banerjee, S., and Lavie, A. (2005, January 29). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, Ann Arbor, MI, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1515\/pralin-2017-0013","article-title":"Is Neural Machine Translation the New State of the Art?","volume":"108","author":"Castilho","year":"2017","journal-title":"Prague Bull. Math. Linguist."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/s10590-018-9220-z","article-title":"Human versus automatic quality evaluation of NMT and PBSMT","volume":"32","author":"Shterionov","year":"2018","journal-title":"Mach. Transl."},{"key":"ref_6","unstructured":"Vanmassenhove, E., Shterionov, D., and Way, A. (2019, January 19\u201323). Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation. Proceedings of Machine Translation Summit XVII Volume 1: Research Track, Dublin, Ireland."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Vanmassenhove, E., Shterionov, D.S., and Gwilliam, M. (2021). Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation. arXiv.","DOI":"10.18653\/v1\/2021.eacl-main.188"},{"key":"ref_8","unstructured":"Roberts, N., Liang, D., Neubig, G., and Lipton, Z.C. (2020). Decoding and Diversity in Machine Translation. arXiv."},{"key":"ref_9","unstructured":"Farrell, M. (2018, January 15\u201316). Machine Translation Markers in Post-Edited Machine Translation Output. Proceedings of the 40th Conference Translating and the Computer, London, UK."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Baker, M., Francis, G., and Tognini-Bonelli, E. (1993). Corpus Linguistics and Translation Studies: Implications and Applications. Text and Technology: In Honour of John Sinclair, John Benjamins Publishing Company.","DOI":"10.1075\/z.64"},{"key":"ref_11","unstructured":"Toral, A. (2019, January 19\u201323). Post-editese: An Exacerbated Translationese. Proceedings of the Machine Translation Summit XVII, Dublin, Ireland."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Castilho, S., Resende, N., and Mitkov, R. (2019, January 5\u20136). What Influences the Features of Post-Editese? A Preliminary Study. Proceedings of the Human-Informed Translation and Interpreting Technology Workshop (HiT-IT 2019), Varna, Bulgaria.","DOI":"10.26615\/issn.2683-0078.2019_003"},{"key":"ref_13","first-page":"21","article-title":"Uncovering Machine Translationese Using Corpus Analysis Techniques to Distinguish between Original and Machine-Translated French","volume":"101","author":"Loock","year":"2021","journal-title":"Transl. Q."},{"key":"ref_14","unstructured":"\u010culo, O., and Nitzke, J. (June, January 30). Patterns of Terminological Variation in Post-editing and of Cognate Use in Machine Translation in Contrast to Human Translation. Proceedings of the 19th Annual Conference of the European Association for Machine Translation, Riga, Latvia."},{"key":"ref_15","unstructured":"Arcan, M. (2018, January 20\u201321). A Comparison of Statistical and Neural Machine Translation for Slovene, Serbian and Croatian. Proceedings of the Conference on Language Technologies & Digital Humanities, Ljubljana, Slovenia."},{"key":"ref_16","unstructured":"Gregor Donaj, M.S.M. (2018, January 20\u201321). Prehod iz statisti\u010dnega strojnega prevajanja na prevajanje z nevronskimi omre\u017eji za jezikovni par sloven\u0161\u010dina-angle\u0161\u010dina. Proceedings of the Conference on Language Technologies & Digital Humanities, Ljubljana, Slovenia."},{"key":"ref_17","unstructured":"Du, J., Arcan, M., Liu, Q., and Isahara, H. (2018, January 7\u201312). Terminology Translation Accuracy in Statistical versus Neural MT: An Evaluation for the English-Slovene Language Pair. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan."},{"key":"ref_18","unstructured":"(2020, March 03). LASERJET PRO 300 COLOR MFP\/LASERJET PRO 400 COLOR MFP User Guide. Available online: http:\/\/h10032.www1.hp.com\/ctg\/Manual\/c02666267.pdf."},{"key":"ref_19","unstructured":"(2020, March 03). LASERJET PRO 300 COLOR MFP\/LASERJET PRO 400 COLOR MFP Uporabni\u0161ki Priro\u010dnik. Available online: http:\/\/h10032.www1.hp.com\/ctg\/Manual\/c02666724.pdf."},{"key":"ref_20","unstructured":"(2020, March 03). LASERJET PRO 300 COLOR MFP\/LASERJET PRO 400 COLOR MFP Installation Guide. Available online: http:\/\/h10032.www1.hp.com\/ctg\/Manual\/c02843075.pdf."},{"key":"ref_21","unstructured":"(2020, March 03). LASERJET PRO 300 COLOR MFP\/LASERJET PRO 400 COLOR MFP Priro\u010dnik za Namestitev. Available online: http:\/\/h10032.www1.hp.com\/ctg\/Manual\/c02843079.pdf."},{"key":"ref_22","unstructured":"Blashford-Snell, V. (2008). The Cooking Book, Dorling Kindersley."},{"key":"ref_23","unstructured":"Blashford-Snell, V. (2012). Dobra Kuha, Mladinska Knjiga."},{"key":"ref_24","unstructured":"James, J. (2009). Practice Makes Perfect, Berkley Sensation."},{"key":"ref_25","unstructured":"James, J. (2014). Osem let skomin, Hi\u0161a knjig, Zalo\u017eba KM\u0160."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s40607-014-0009-9","article-title":"The Sketch Engine: Ten years on","volume":"1","author":"Kilgarriff","year":"2014","journal-title":"Lexicography"},{"key":"ref_27","unstructured":"McCarthy, P.M. (2005). An Assessment of the Range and Usefulness of Lexical Diversity Measures and the Potential of the Measure of Textual, Lexical Diversity (MTLD). [Ph.D. Thesis, The University of Memphis]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"381","DOI":"10.3758\/BRM.42.2.381","article-title":"MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment","volume":"42","author":"Mccarthy","year":"2010","journal-title":"Behav. Res. Methods"},{"key":"ref_29","first-page":"S397","article-title":"Measuring Lexical Diversity in Narrative Discourse of People with Aphasia","volume":"22","author":"Fergadiotis","year":"2013","journal-title":"Am. J. Speech-Lang. Pathol.\/Am. Speech-Lang. Assoc."},{"key":"ref_30","unstructured":"Kyle, K. (2020, June 01). Lexical Diversity. Available online: https:\/\/github.com\/kristopherkyle\/lexical_diversity."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/93\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:20:23Z","timestamp":1760134823000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/93"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,15]]},"references-count":30,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020093"],"URL":"https:\/\/doi.org\/10.3390\/info13020093","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,15]]}}}