{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:08:00Z","timestamp":1773929280781,"version":"3.50.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T00:00:00Z","timestamp":1622505600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,22]],"date-time":"2021-06-22T00:00:00Z","timestamp":1624320000000},"content-version":"vor","delay-in-days":21,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001602","name":"Science Foundation Ireland","doi-asserted-by":"crossref","award":["13\/RC\/2106"],"award-info":[{"award-number":["13\/RC\/2106"]}],"id":[{"id":"10.13039\/501100001602","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Edge Marie Curie Co-fund Programme","award":["713567"],"award-info":[{"award-number":["713567"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Machine Translation"],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper presents results of the effect of different translation modalities on users when working with the Microsoft Word user interface. An experimental study was set up with 84 Japanese, German, Spanish, and English native speakers working with Microsoft Word in three modalities: the published translated version, a machine translated (MT) version (with unedited MT strings incorporated into the MS Word interface) and the published English version. An eye-tracker measured the cognitive load and usability according to the ISO\/TR 16982 guidelines: i.e., effectiveness, efficiency, and satisfaction followed by retrospective think-aloud protocol. The results show that the users\u2019 effectiveness (number of tasks completed) does not significantly differ due to the translation modality. However, their efficiency (time for task completion) and self-reported satisfaction are significantly higher when working with the released product as opposed to the unedited MT version, especially when participants are less experienced. The eye-tracking results show that users experience a higher cognitive load when working with MT and with the human-translated versions as opposed to the English original. The results suggest that language and translation modality play a significant role in the usability of software products whether users complete the given tasks or not and even if they are unaware that MT was used to translate the interface.<\/jats:p>","DOI":"10.1007\/s10590-021-09267-z","type":"journal-article","created":{"date-parts":[[2021,6,22]],"date-time":"2021-06-22T19:02:27Z","timestamp":1624388547000},"page":"205-237","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["The impact of translation modality on user experience: an eye-tracking study of the Microsoft Word user interface"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9820-7074","authenticated-orcid":false,"given":"Ana","family":"Guerberof Arenas","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joss","family":"Moorkens","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sharon","family":"O\u2019Brien","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,6,22]]},"reference":[{"key":"9267_CR1","doi-asserted-by":"crossref","unstructured":"Bentivogli L, Bisazza A, Cettolo M, Federico M (2016) Neural versus phrase-based machine translation quality: a case study. arXiv preprint arXiv:1608.04631","DOI":"10.18653\/v1\/D16-1025"},{"issue":"2","key":"9267_CR2","first-page":"13","volume":"14","author":"L Bowker","year":"2015","unstructured":"Bowker L (2015) Translatability and user experience: compatible or in conflict? Localis Focus Int J Localis 14(2):13\u201327","journal-title":"Localis Focus Int J Localis"},{"key":"9267_CR3","first-page":"8","volume-title":"Routledge studies in translation","author":"L Bowker","year":"2018","unstructured":"Bowker L, Buitrago Ciro J (2018) Localizing websites using machine translation: the human factor in machine translation. In: Sin-wai C (ed) Routledge studies in translation. Routledge, London, New York, pp 8\u201329"},{"issue":"194","key":"9267_CR4","first-page":"4","volume":"189","author":"J Brooke","year":"1996","unstructured":"Brooke J (1996) SUS\u2014a quick and dirty usability scale. Usabil Eval Ind 189(194):4\u20137","journal-title":"Usabil Eval Ind"},{"key":"9267_CR5","unstructured":"Castilho S, Guerberof Arenas A (2018) Reading comprehension of machine translation output: what makes for a better read? In Proceedings 21st Annual Conference of the European Association for Machine Translation, Alacant, pp 79\u201388"},{"issue":"1","key":"9267_CR6","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1515\/pralin-2017-0013","volume":"108","author":"S Castilho","year":"2017","unstructured":"Castilho S, Moorkens J, Gaspari F, Calixto I, Tinsley J, Way A (2017) Is neural machine translation the new state of the art? Prague Bull Math Linguist 108(1):109\u2013120","journal-title":"Prague Bull Math Linguist"},{"key":"9267_CR7","unstructured":"Castilho S, O\u2019Brien S, Alves F, O\u2019Brien M (2014) Does post-editing increase usability? A study with Brazilian Portuguese as Target Language. In Proceedings of the Seventeenth Annual Conference of the European Association for Machine Translation, Dubrovnik, pp 183\u2013190"},{"key":"9267_CR8","unstructured":"Castilho S (2016) Measuring acceptability of machine translated enterprise content. PhD Thesis. Dublin City University, Dublin, Ireland"},{"issue":"1","key":"9267_CR9","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s10590-019-09230-z","volume":"33","author":"J Daems","year":"2019","unstructured":"Daems J, Macken L (2019) Interactive adaptive SMT versus interactive adaptive NMT: a user experience evaluation. Mach Transl 33(1):117\u2013134. https:\/\/doi.org\/10.1007\/s10590-019-09230-z","journal-title":"Mach Transl"},{"key":"9267_CR10","unstructured":"De Almeida G, O\u2019Brien S (2010) Analysing post-editing performance: correlations with years of translation experience. In Proceedings of the 14th Annual Conference of the European Association for Machine Translation, S. Rafael, France"},{"key":"9267_CR11","doi-asserted-by":"crossref","unstructured":"Doddington G (2002) Automatic evaluation of machine translation quality using N-gram co-occurrence statistics. In Proceedings of the Second International Conference on Human Language Technology Research, 138\u2013145. HLT \u201902. Morgan Kaufmann Publishers Inc, San Diego","DOI":"10.3115\/1289189.1289273"},{"key":"9267_CR12","unstructured":"Doherty S, O\u2019Brien S (2009) Can MT output be evaluated through eye tracking?. In Proceedings of the Machine Translation Summit XII, Ottawa, Ontario, Canada, pp 214\u2013221"},{"key":"9267_CR13","unstructured":"Doherty S, O\u2019Brien S (2012) A user-based usability assessment of raw machine translated technical instructions. In Proceedings of the Tenth Biennial Conference of the Association for Machine Translation in the Americas. Proceedings, San Diego, 6 pp"},{"issue":"1","key":"9267_CR14","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1080\/10447318.2013.802199","volume":"30","author":"S Doherty","year":"2014","unstructured":"Doherty S, O\u2019Brien S (2014) Assessing the usability of raw machine translated output: a user-centered study using eye tracking. Int J Hum Comput Interact 30(1):40\u201351","journal-title":"Int J Hum Comput Interact"},{"issue":"1","key":"9267_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10590-010-9070-9","volume":"24","author":"S Doherty","year":"2010","unstructured":"Doherty S, O\u2019Brien S, Carl M (2010) Eye tracking as an MT evaluation technique. Mach Transl 24(1):1\u201313","journal-title":"Mach Transl"},{"key":"9267_CR16","doi-asserted-by":"publisher","unstructured":"Gao G, Wang H-C, Cosley D, Fussell SR (2013) Same translation but different experience: The effects of highlighting on machine-translated conversations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp 449\u2013458. https:\/\/doi.org\/10.1145\/2470654.2470719","DOI":"10.1145\/2470654.2470719"},{"key":"9267_CR17","unstructured":"Guerberof A (2012) Productivity and quality in the post-editing of outputs from translation memories and machine translation. PhD thesis. Universitat Rovira i Virgili, Tarragona, Spain"},{"key":"9267_CR18","unstructured":"Guerberof A, Moorkens J, O\u2019Brien S (2019) What is the impact of raw MT on Japanese users of Word preliminary results of a usability study using eye-tracking. In Proceedings of the Seventh Machine Translation Summit (MT Summit 2019), Dublin, Ireland"},{"key":"9267_CR19","unstructured":"L\u00e4ubli S, Amrhein C, D\u00fcggelin P, Gonzalez B, Zwahlen A, Volk M (2019) Post-editing productivity with neural machine translation: an empirical assessment of speed and quality in the banking and finance domain. ArXiv:1906.01685 [Cs]. http:\/\/arxiv.org\/abs\/1906.01685"},{"key":"9267_CR20","doi-asserted-by":"publisher","first-page":"325","DOI":"10.3758\/s13428-011-0146-0","volume":"44","author":"K Lemh\u00f6fer","year":"2012","unstructured":"Lemh\u00f6fer K, Broersma M (2012) Introducing LexTALE: a quick and valid lexical test for advanced learners of English. Behav Res Methods 44:325\u2013343","journal-title":"Behav Res Methods"},{"key":"9267_CR21","doi-asserted-by":"crossref","unstructured":"Lewis JR (1992) Psychometric evaluation of the post-study system usability questionnaire: the PSSUQ. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (vol 36(16), pp 1259\u20131260). SAGE Publications, Los Angeles","DOI":"10.1177\/154193129203601617"},{"issue":"1","key":"9267_CR22","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1080\/10447319509526110","volume":"7","author":"JR Lewis","year":"1995","unstructured":"Lewis JR (1995) IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 7(1):57\u201378","journal-title":"Int J Hum Comput Interact"},{"key":"9267_CR23","doi-asserted-by":"publisher","unstructured":"Lim H, Fussell SR (2017) Making sense of foreign language posts in social media. In Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), pp 69:1\u201369:16. https:\/\/doi.org\/10.1145\/3134704","DOI":"10.1145\/3134704"},{"issue":"3","key":"9267_CR24","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/s10590-015-9175-2","volume":"29","author":"J Moorkens","year":"2015","unstructured":"Moorkens J, O\u2019Brien S, da Silva I, de Lima Fonseca N, Alves F (2015) Correlations of perceived post-editing effort with measurements of actual effort. Mach Transl 29(3):267\u2013284","journal-title":"Mach Transl"},{"key":"9267_CR25","unstructured":"O\u2019Brien S, Castilho S (2016) Evaluating the impact of light post-editing on usability. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portoro\u017e, Slovenia, pp 310\u2013316"},{"issue":"3","key":"9267_CR26","first-page":"185","volume":"14","author":"S O\u2019Brien","year":"2006","unstructured":"O\u2019Brien S (2006) Eye-tracking and translation memory matches. Perspect Stud Translatol 14(3):185\u2013205","journal-title":"Perspect Stud Translatol"},{"key":"9267_CR27","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1007\/978-3-319-98743-9_5","volume-title":"Collaboration technologies and social computing","author":"M Pituxcoosuvarn","year":"2018","unstructured":"Pituxcoosuvarn M, Ishida T, Yamashita N, Takasaki T, Mori Y (2018) Machine translation usage in a children\u2019s workshop. In: Egi H, Yuizono T, Baloian N, Yoshino T, Ichimura S, Rodrigues A (eds) Collaboration technologies and social computing. Springer, New York, pp 59\u201373. https:\/\/doi.org\/10.1007\/978-3-319-98743-9_5"},{"key":"9267_CR28","first-page":"7","volume":"23","author":"M Plitt","year":"2010","unstructured":"Plitt M, Masselot F (2010) A productivity test of statistical machine translation post-editing in a typical localization context. Prague Bull Math Linguist 23:7\u201316","journal-title":"Prague Bull Math Linguist"},{"key":"9267_CR29","doi-asserted-by":"publisher","unstructured":"Quirk C, Menezes A, Cherry C (2005) Dependency Treelet Translation: syntactically informed phrasal SMT. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL\u201905), 271\u2013279. https:\/\/doi.org\/10.3115\/1219840.1219874","DOI":"10.3115\/1219840.1219874"},{"key":"9267_CR30","unstructured":"Schmidtke D (2016) Large scale Machine Translation publishing, with acceptable quality, for Microsoft Support content. In AMTA 2016 Workshop on Interacting with Machine Translation (MT 2016). Austin."},{"key":"9267_CR31","unstructured":"Schmidtke D, Groves D (2019). Automatic translation for software with safe velocity. In Proceedings of Machine Translation Summit XVII Volume 2: Translator, Project and User Tracks, 159\u2013166. https:\/\/www.aclweb.org\/anthology\/W19-6729"},{"key":"9267_CR32","first-page":"135","volume":"31","author":"B Screen","year":"2019","unstructured":"Screen B (2019) What effect does post-editing have on the translation product from an end-user\u2019s perspective? J Special Transl 31:135\u2013157","journal-title":"J Special Transl"},{"key":"9267_CR33","doi-asserted-by":"publisher","first-page":"9","DOI":"10.3389\/fdigh.2018.00009","volume":"5","author":"A Toral","year":"2018","unstructured":"Toral A, Wieling M, Way A (2018) Post-editing effort of a novel with statistical and neural machine translation. Front Digital Human 5:9","journal-title":"Front Digital Human"},{"key":"9267_CR34","first-page":"158","volume":"31","author":"GW van Egdom","year":"2019","unstructured":"van Egdom GW, Pluymaekers M (2019) Why go the extra mile? How different degrees of post-editing affect perceptions of texts, senders and products among end users. J Special Transl 31:158\u2013176","journal-title":"J Special Transl"},{"key":"9267_CR35","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. In Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds.). Advances in neural information processing systems, vol 30, pp 5998\u20136008. Curran Associates, Inc. http:\/\/papers.nips.cc\/paper\/7181-attention-is-all-you-need.pdf"},{"key":"9267_CR36","doi-asserted-by":"publisher","unstructured":"Wang H-C, Fussell S, Cosley D (2013) Machine translation vs. common language: effects on idea exchange in cross-lingual groups. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, pp 935\u2013944. https:\/\/doi.org\/10.1145\/2441776.2441882","DOI":"10.1145\/2441776.2441882"}],"container-title":["Machine Translation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10590-021-09267-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10590-021-09267-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10590-021-09267-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,22]],"date-time":"2021-07-22T16:26:26Z","timestamp":1626971186000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10590-021-09267-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["9267"],"URL":"https:\/\/doi.org\/10.1007\/s10590-021-09267-z","relation":{},"ISSN":["0922-6567","1573-0573"],"issn-type":[{"value":"0922-6567","type":"print"},{"value":"1573-0573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6]]},"assertion":[{"value":"13 June 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"First author received an Edge Marie Sk\u0142odowska Curie Fellowship.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}