{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T13:08:33Z","timestamp":1792242513119,"version":"4.4.0"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2021,10,20]],"date-time":"2021-10-20T00:00:00Z","timestamp":1634688000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Ministry of Education of the Republic of Korea and the National Research Foundation of Korea","award":["NRF-2018S1A5A2A01030227"],"award-info":[{"award-number":["NRF-2018S1A5A2A01030227"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,8,23]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>This study aims to test the classification of literary translations by human and machine translators in a bid to explore stylistic distinctions between the two groups and among machine translators and their evolution over 1 year between 2019 and 2020. For this, the study takes a stylometric approach by employing two statistical methods popularly used in authorship attribution\u2014support vector machine (SVM) and principal component analysis (PCA), which are applied to analyzing three types of features\u20141-grams, 2-grams, and 3 grams. The results show that over the researched period, the three machine translators investigated (Google, Bing, and Papago) moved in the direction of converging in style while advancing toward human translators. The distance gained by them on human translators, however, was not significant enough to challenge the clear-cut divide between the two groups. The PCA tests additionally revealed some characteristics of the machine translator that might be responsible for the stylistic distinction between them and their human counterparts including their tendency of overusing basic standard language.<\/jats:p>","DOI":"10.1093\/llc\/fqab091","type":"journal-article","created":{"date-parts":[[2021,9,30]],"date-time":"2021-09-30T13:32:45Z","timestamp":1633008765000},"page":"813-829","source":"Crossref","is-referenced-by-count":6,"title":["How do machine translators measure up to human literary translators in stylometric tests?"],"prefix":"10.1093","volume":"37","author":[{"given":"Changsoo","family":"Lee","sequence":"first","affiliation":[{"name":"Graduate School of Interpretation and Translation, Hankuk University of Foreign Studies , Seoul, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,10,20]]},"reference":[{"issue":"2","key":"2022082315161152500_fqab091-B1","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1075\/target.12.2.04bak","article-title":"Towards a methodology for investigating the style of a literary translator","volume":"12","author":"Baker","year":"2000","journal-title":"Target"},{"key":"2022082315161152500_fqab091-B3","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.esp.2006.08.003","article-title":"Lexical bundles in university spoken and written registers","volume":"26","author":"Biber","year":"2007","journal-title":"English for Specific Purposes"},{"key":"2022082315161152500_fqab091-B151","author":"Brett","year":"2013","journal-title":"Machine Learning with R."},{"issue":"4","key":"2022082315161152500_fqab091-B4","first-page":"677","article-title":"The Englishing of Juvenal: computational stylistics and translated texts","volume":"36","author":"Burrows","year":"2002","journal-title":"Style"},{"issue":"3","key":"2022082315161152500_fqab091-B5","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1093\/llc\/17.3.267","article-title":"\u2018Delta\u2019: a measure of stylistic difference and a guide to likely authorship","volume":"17","author":"Burrows","year":"2002","journal-title":"Literary and Linguistic Computing"},{"issue":"1","key":"2022082315161152500_fqab091-B6","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1021814530952","article-title":"Questions of authorship: attribution and beyond; a lecture delivered on the occasion of the Roberto Busa Award ACH-ALLC 2001, New York","volume":"37","author":"Burrows","year":"2003","journal-title":"Computers and the Humanities"},{"issue":"1","key":"2022082315161152500_fqab091-B7","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1515\/pralin-2017-0013","article-title":"Is neural machine translation the new state of the art?","volume":"108","author":"Castilho","year":"2017","journal-title":"Prague Bulletin of Mathematical Linguistics"},{"key":"2022082315161152500_fqab091-B8","first-page":"11684v1","article-title":"On measuring gender bias in translation of gender-neutral pronouns","volume":"1905","author":"Cho","year":"2019","journal-title":"arXiv"},{"issue":"1","key":"2022082315161152500_fqab091-B9","first-page":"1","article-title":"On the need for a new research agenda for corpus-based translation studies: a multi-methodological, multifactorial and interdisciplinary approach","volume":"28","author":"De Sutter","year":"2020","journal-title":"Perspectives-Studies in Translation Theory and Practice"},{"issue":"2","key":"2022082315161152500_fqab091-B11","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1093\/llc\/fqt066","article-title":"Does size matter? Authorship attribution, small samples, big problem","volume":"30","author":"Eder","year":"2015","journal-title":"Digital Scholarship in the Humanities"},{"issue":"1","key":"2022082315161152500_fqab091-B12","doi-asserted-by":"crossref","first-page":"107","DOI":"10.32614\/RJ-2016-007","article-title":"Stylometry with R: a package for computational text analysis","volume":"8","author":"Eder","year":"2016","journal-title":"R Journal"},{"key":"2022082315161152500_fqab091-B13","doi-asserted-by":"crossref","first-page":"2039","DOI":"10.1109\/FUZZY.2011.6007535","volume-title":"Proceedings of 2011 IEEE International Conference on Fuzzy Systems","author":"El-Fiqi","year":"2011"},{"key":"2022082315161152500_fqab091-B14","volume-title":"Corpus Linguistics in Literary Analysis: Jane Austen and her Contemporaries","author":"Fischer-Starcke","year":"2010"},{"issue":"2","key":"2022082315161152500_fqab091-B15","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1093\/llc\/fqt018","article-title":"Found in translation: to what extent is authorial discriminability preserved by translators?","volume":"29","author":"Forsyth","year":"2014","journal-title":"Literary and Linguistic Computing"},{"key":"2022082315161152500_fqab091-B16","first-page":"01122v1","article-title":"Machine translation: a literature review","volume":"1901","author":"Garg","year":"2018","journal-title":"arXiv"},{"key":"2022082315161152500_fqab091-B17","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1075\/cilt.341.09gra","volume-title":"Multiword Units in Machine Translation and Translation Technology","author":"Grabowski","year":"2018"},{"key":"2022082315161152500_fqab091-B18","article-title":"Achieving human parity on automatic Chinese to English news translation","author":"Hassan","year":"2018","journal-title":"arXiv"},{"issue":"1","key":"2022082315161152500_fqab091-B19","first-page":"25","article-title":"Revisiting style, a key concept in literary studies","volume":"9","author":"Herrmann","year":"2015","journal-title":"Journal of Literary Theory"},{"issue":"4","key":"2022082315161152500_fqab091-B20","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1093\/llc\/18.4.341","article-title":"Multivariate analysis and the study of style variation","volume":"18","author":"Hoover","year":"2003","journal-title":"Literary and Linguistic Computing"},{"key":"2022082315161152500_fqab091-B21","first-page":"517","volume-title":"A Companion to Digital Literary Studies","author":"Hoover","year":"2007"},{"key":"2022082315161152500_fqab091-B22","volume-title":"Exploratory Multivariate Analysis by Example Using R","author":"Husson","year":"2011"},{"key":"2022082315161152500_fqab091-B23","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4615-0907-3","volume-title":"Learning to Classify Text Using Support Vector Machines","author":"Joachims","year":"2002"},{"issue":"3","key":"2022082315161152500_fqab091-B24","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1561\/1500000005","article-title":"Authorship attribution","volume":"1","author":"Juola","year":"2006","journal-title":"Foundations and Trends in Information Retrieval"},{"issue":"2","key":"2022082315161152500_fqab091-B25","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1093\/llc\/fqt063","article-title":"Collaborative authorship in the twelfth century: a stylometric study of Hildegard of Bingen and Guibert of Gembloux","volume":"30","author":"Kestemont","year":"2015","journal-title":"Digital Scholarship in the Humanities"},{"key":"2022082315161152500_fqab091-B26","doi-asserted-by":"crossref","DOI":"10.21832\/9781853595639","volume-title":"Literary Translation, A Practical Guide","author":"Landers","year":"2001"},{"key":"2022082315161152500_fqab091-B27","article-title":"Has machine translation achieved human parity? A case for document-level evaluation","author":"L\u00e4ubli","year":"2018","journal-title":"arXiv"},{"issue":"3","key":"2022082315161152500_fqab091-B28","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1093\/llc\/fqx056","article-title":"Do language combinations affect translators\u2019 stylistic visibility in translated texts?","volume":"33","author":"Lee","year":"2018","journal-title":"Digital Scholarship in the Humanities"},{"key":"2022082315161152500_fqab091-B29","volume-title":"Corpus Stylistics in Heart of Darkness and its Italian Translations","author":"Mastropierro","year":"2018"},{"issue":"2","key":"2022082315161152500_fqab091-B30","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1075\/ts.18014.moo","article-title":"Translators\u2019 perceptions of literary post-editing using statistical and neural machine translation","volume":"7","author":"Moorkens","year":"2018","journal-title":"Translation Spaces"},{"key":"2022082315161152500_fqab091-B31","first-page":"221","author":"Oakes","year":"2013"},{"key":"2022082315161152500_fqab091-B32","first-page":"245","volume-title":"Proceedings of the Digital Humanities 2009 Conference","author":"Rybicki","year":"2009"},{"key":"2022082315161152500_fqab091-B33","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1075\/scl.51.09ryb","volume-title":"Quantitative Methods in Corpus-Based Translation Studies. Amsterdam: John Benjamins","author":"Rybicki","year":"2012"},{"issue":"1","key":"2022082315161152500_fqab091-B34","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1080\/13556509.2011.10799478","article-title":"Translator style: methodological considerations","volume":"17","author":"Saldanha","year":"2011","journal-title":"The Translator"},{"key":"2022082315161152500_fqab091-B35","volume-title":"WordSmith Tools Manual","author":"Scott","year":"2017"},{"key":"2022082315161152500_fqab091-B37","article-title":"Ethical issues regarding machine(-assisted) translation of literary texts","author":"Taivalkoski-Shilov","year":"2018","journal-title":"Perspectives: Studies in Translation Theory and Practice, 27: 689\u2013703"},{"key":"2022082315161152500_fqab091-B38","first-page":"174","volume-title":"Proceedings of Translating and the Computer 36","author":"Toral","year":"2014"},{"key":"2022082315161152500_fqab091-B39","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1075\/ts.4.2.04tor","article-title":"Machine-assisted translation of literary text: a case study","volume":"4","author":"Toral","year":"2015","journal-title":"Translation Spaces"},{"key":"2022082315161152500_fqab091-B40","first-page":"04962v1","article-title":"What level of quality can neural machine translation attain on literary text?","volume":"1801","author":"Toral","year":"2018","journal-title":"arXiv"},{"key":"2022082315161152500_fqab091-B41","doi-asserted-by":"crossref","DOI":"10.3389\/fdigh.2018.00009","article-title":"Post-editing effort of a novel with statistical and neural machine translation","author":"Toral","year":"2018","journal-title":"Frontiers in Digital Humanities."},{"key":"2022082315161152500_fqab091-B42","first-page":"3003","author":"Vanmassenhove","year":"2018"},{"key":"2022082315161152500_fqab091-B43","first-page":"1264","author":"Voita","year":"2018"},{"key":"2022082315161152500_fqab091-B44","first-page":"12","volume-title":"Proceedings of the Workshop on Multi-word Units in Machine Translation and Translation Technology","author":"Wehrli","year":"2013"},{"key":"2022082315161152500_fqab091-B45","article-title":"Google's neural machine translation system: bridging the gap between human and machine translation","author":"Wu","year":"2016","journal-title":"arXiv"}],"container-title":["Digital Scholarship in the Humanities"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/dsh\/article-pdf\/37\/3\/813\/45501868\/fqab091.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/dsh\/article-pdf\/37\/3\/813\/45501868\/fqab091.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T15:32:33Z","timestamp":1661268753000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/dsh\/article\/37\/3\/813\/6422199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,20]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,10,20]]},"published-print":{"date-parts":[[2022,8,23]]}},"URL":"https:\/\/doi.org\/10.1093\/llc\/fqab091","relation":{},"ISSN":["2055-7671","2055-768X"],"issn-type":[{"value":"2055-7671","type":"print"},{"value":"2055-768X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,9,1]]},"published":{"date-parts":[[2021,10,20]]}}}