{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:15:21Z","timestamp":1772118921028,"version":"3.50.1"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,2,5]],"date-time":"2024-02-05T00:00:00Z","timestamp":1707091200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,5]],"date-time":"2024-02-05T00:00:00Z","timestamp":1707091200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Hum-Cent Intell Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Advances in neural machine translation utilizing pretrained language models (PLMs) have shown promise in improving the translation quality between diverse languages. However, translation from English to languages with complex morphology, such as Arabic, remains challenging. This study investigated the prevailing error patterns of state-of-the-art PLMs when translating from English to Arabic across different text domains. Through empirical analysis using automatic metrics (chrF, BERTScore, COMET) and manual evaluation with the Multidimensional Quality Metrics (MQM) framework, we compared Google Translate and five PLMs (Helsinki, Marefa, Facebook, GPT-3.5-turbo, and GPT-4). Key findings provide valuable insights into current PLM limitations in handling aspects of Arabic grammar and vocabulary while also informing future improvements for advancing English\u2013Arabic machine translation capabilities and accessibility.<\/jats:p>","DOI":"10.1007\/s44230-024-00061-7","type":"journal-article","created":{"date-parts":[[2024,2,5]],"date-time":"2024-02-05T11:03:40Z","timestamp":1707131020000},"page":"206-219","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Error Analysis of Pretrained Language Models (PLMs) in English-to-Arabic Machine Translation"],"prefix":"10.1007","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7328-4935","authenticated-orcid":false,"given":"Hend","family":"Al-Khalifa","sequence":"first","affiliation":[]},{"given":"Khaloud","family":"Al-Khalefah","sequence":"additional","affiliation":[]},{"given":"Hesham","family":"Haroon","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,2,5]]},"reference":[{"issue":"2","key":"61_CR1","first-page":"1","volume":"11","author":"IA Zaugg","year":"2022","unstructured":"Zaugg IA, Hossain A, Molloy B. Digitally-disadvantaged languages. Internet Policy Rev J Internet Regul. 2022;11(2):1\u201311.","journal-title":"Internet Policy Rev J Internet Regul"},{"key":"61_CR2","unstructured":"Patil A, Joshi I, Kadam D. PICT@WAT 2022: neural machine translation systems for indic languages. In: Proceedings of the 9th workshop on Asian Translation, Gyeongju, Republic of Korea: international conference on computational linguistics. 2022. pp. 106\u2013110. https:\/\/aclanthology.org\/2022.wat-1.13. Accessed 20 Dec 2023."},{"key":"61_CR3","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1109\/TASLP.2021.3138714","volume":"30","author":"K Chen","year":"2022","unstructured":"Chen K, Wang R, Utiyama M, Sumita E. Integrating prior translation knowledge into neural machine translation. IEEEACM Trans Audio Speech Lang Process. 2022;30:330\u20139. https:\/\/doi.org\/10.1109\/TASLP.2021.3138714.","journal-title":"IEEEACM Trans Audio Speech Lang Process"},{"issue":"1","key":"61_CR4","doi-asserted-by":"publisher","first-page":"58","DOI":"10.7575\/aiac.alls.v.10n.1p.58","volume":"10","author":"MF Akan","year":"2019","unstructured":"Akan MF, Karim MR, Chowdhury AMK. An analysis of Arabic\u2013English translation: problems and prospects. Adv Lang Lit Stud. 2019;10(1):58\u201365. https:\/\/doi.org\/10.7575\/aiac.alls.v.10n.1p.58.","journal-title":"Adv Lang Lit Stud"},{"key":"61_CR5","doi-asserted-by":"publisher","DOI":"10.53730\/ijhs.v6nS5.10039","author":"MMA Mamoori","year":"2022","unstructured":"Mamoori MMA, Tarish AH, Hasani SA. Difficulties of translation and evaluative idioms in English and Arabic. Int J Health Sci. 2022. https:\/\/doi.org\/10.53730\/ijhs.v6nS5.10039.","journal-title":"Int J Health Sci"},{"issue":"17","key":"61_CR6","doi-asserted-by":"publisher","first-page":"art no. 17","DOI":"10.3390\/app12178805","volume":"12","author":"M Mars","year":"2022","unstructured":"Mars M. From word embeddings to pre-trained language models: a state-of-the-art walkthrough. Appl Sci. 2022;12(17):art no. 17. https:\/\/doi.org\/10.3390\/app12178805.","journal-title":"Appl Sci"},{"key":"61_CR7","doi-asserted-by":"publisher","unstructured":"Zakraoui J, Saleh M, Al-Maadeed S, AlJa\u2019am JM. Evaluation of Arabic to English machine translation systems. In: 2020 11th International conference on information and communication systems (ICICS). 2020. pp. 185\u2013190. https:\/\/doi.org\/10.1109\/ICICS49469.2020.239518.","DOI":"10.1109\/ICICS49469.2020.239518"},{"key":"61_CR8","doi-asserted-by":"publisher","unstructured":"Bar-Hillel Y. The Present status of automatic translation of languages. In: Alt FL, editors. Advances in computers, vol. 1. Elsevier; 1960. pp. 91\u2013163. https:\/\/doi.org\/10.1016\/S0065-2458(08)60607-5.","DOI":"10.1016\/S0065-2458(08)60607-5"},{"key":"61_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.cosrev.2020.100305","volume":"38","author":"MSH Ameur","year":"2020","unstructured":"Ameur MSH, Meziane F, Guessoum A. Arabic machine translation: a survey of the latest trends and challenges. Comput Sci Rev. 2020;38: 100305. https:\/\/doi.org\/10.1016\/j.cosrev.2020.100305.","journal-title":"Comput Sci Rev"},{"key":"61_CR10","doi-asserted-by":"publisher","first-page":"161445","DOI":"10.1109\/ACCESS.2021.3132488","volume":"9","author":"J Zakraoui","year":"2021","unstructured":"Zakraoui J, Saleh M, Al-Maadeed S, Alja\u2019am JM. Arabic machine translation: a survey with challenges and future directions. IEEE Access. 2021;9:161445\u201368. https:\/\/doi.org\/10.1109\/ACCESS.2021.3132488.","journal-title":"IEEE Access"},{"key":"61_CR11","unstructured":"Farhat A, Al-Taani AT. A rule-based English to Arabic machine translation approach. In: Presented at the international Arab conference on information technology (ACIT\u20192015). 2015. https:\/\/www.semanticscholar.org\/paper\/A-Rule-based-English-to-Arabic-Machine-Translation-Farhat-Al-Taani\/4e7f555a0221eb7f980c597b15bdb8f6a1089e7f. Accessed 16 Jul 2023."},{"key":"61_CR12","doi-asserted-by":"publisher","unstructured":"Fadiel Alawneh M, Sembok TM, Mohd M. Grammar-based and example-based techniques in machine translation from English to Arabic. In: 2013 5th international conference on information and communication technology for the Muslim World (ICT4M). 2013. pp. 1\u20136. https:\/\/doi.org\/10.1109\/ICT4M.2013.6518910.","DOI":"10.1109\/ICT4M.2013.6518910"},{"key":"61_CR13","doi-asserted-by":"crossref","unstructured":"Al-Rukban A, Saudagar AKJ. Evaluation of English to Arabic machine translation systems using BLEU and GTM. In: Proceedings of the 2017 9th international conference on education technology and computers. ACM; 2017.","DOI":"10.1145\/3175536.3175570"},{"issue":"4","key":"61_CR14","first-page":"396","volume":"11","author":"M Akeel","year":"2014","unstructured":"Akeel M, Mishra R. ANN and rule based method for english to arabic machine translation. Int Arab J Inf Technol. 2014;11(4):396\u2013405.","journal-title":"Int Arab J Inf Technol"},{"key":"61_CR15","doi-asserted-by":"publisher","unstructured":"Aljohany DA, Al-Barhamtoshy HM, Abukhodair FA. Arabic machine translation (ArMT) based on LSTM with attention mechanism architecture. In: 2022 20th International conference on language engineering (ESOLEC). 2022. pp. 78\u201383. https:\/\/doi.org\/10.1109\/ESOLEC54569.2022.10009530.","DOI":"10.1109\/ESOLEC54569.2022.10009530"},{"key":"61_CR16","unstructured":"Aref M, Al-Mulhem M, Al-Muhtaseb H. English to Arabic machine translation: a critical review and suggestions for development. King Fahd Univ. Pet. Miner. Dhahran Saudi Arab. 1992."},{"key":"61_CR17","unstructured":"Nagoudi EMB, Elmadany A, Abdul-Mageed M. TURJUMAN: a public toolkit for neural Arabic machine translation. In: Proceedings of the 5th workshop on open-source Arabic corpora and processing tools with shared tasks on Qur\u2019an QA and fine-grained hate speech detection. Marseille, France: European Language Resources Association; 2022. pp. 1\u201311. https:\/\/aclanthology.org\/2022.osact-1.1. Accessed 16 Jul 2023."},{"key":"61_CR18","doi-asserted-by":"publisher","unstructured":"Lewis M, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th annual meeting of the association for computational linguistics. Association for Computational Linguistics; 2020. pp. 7871\u20137880. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.703.","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"61_CR19","doi-asserted-by":"publisher","unstructured":"Chronopoulou A. Stojanovski D, Fraser A. Reusing a pretrained language model on languages with limited corpora for unsupervised NMT. In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP). Association for Computational Linguistics; 2020. pp. 2703\u20132711. https:\/\/doi.org\/10.18653\/v1\/2020.emnlp-main.214.","DOI":"10.18653\/v1\/2020.emnlp-main.214"},{"key":"61_CR20","doi-asserted-by":"publisher","unstructured":"Edunov S, Baevski A, Auli M. Pre-trained language model representations for language generation. In: Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers). Minneapolis, Minnesota: Association for Computational Linguistics; 2019. pp. 4052\u20134059. https:\/\/doi.org\/10.18653\/v1\/N19-1409.","DOI":"10.18653\/v1\/N19-1409"},{"key":"61_CR21","doi-asserted-by":"publisher","unstructured":"Zheng F, Reid M, Marrese-Taylor E, Matsuo Y. Low-resource machine translation using cross-lingual language model pretraining. In: Proceedings of the first workshop on natural language processing for indigenous languages of the Americas. Association for Computational Linguistics; 2021. pp. 234\u2013240. https:\/\/doi.org\/10.18653\/v1\/2021.americasnlp-1.26.","DOI":"10.18653\/v1\/2021.americasnlp-1.26"},{"issue":"Art. no. 5","key":"61_CR22","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3390\/info13050220","volume":"13","author":"M De Coster","year":"2022","unstructured":"De Coster M, Dambre J. Leveraging frozen pretrained written language models for neural sign language translation. Information. 2022;13(Art. no. 5):5. https:\/\/doi.org\/10.3390\/info13050220.","journal-title":"Information"},{"key":"61_CR23","doi-asserted-by":"crossref","unstructured":"Agarwal V, Rao P, Jayagopi DB (2023) Hinglish to English machine translation using multilingual transformers. In: Proceedings of the student research workshop associated with RANLP 2021, INCOMA Ltd., Sep. 2021, pp. 16\u201321. https:\/\/aclanthology.org\/2021.ranlp-srw.3. Accessed 16 Jul 2023.","DOI":"10.26615\/issn.2603-2821.2021_003"},{"key":"61_CR24","doi-asserted-by":"publisher","unstructured":"Jude Ogundepo O, Oladipo A, Adeyemi M, Ogueji K, Lin J. AfriTeVA: Extending? Small data? Pretraining approaches to sequence-to-sequence models. In: Proceedings of the third workshop on deep learning for low-resource natural language processing, hybrid. Association for Computational Linguistics; 2022. pp. 126\u2013135. https:\/\/doi.org\/10.18653\/v1\/2022.deeplo-1.14.","DOI":"10.18653\/v1\/2022.deeplo-1.14"},{"key":"61_CR25","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1016\/B978-0-08-042580-1.50066-0","volume-title":"Concise history of the language sciences","author":"WJ Hutchins","year":"1995","unstructured":"Hutchins WJ. Machine translation: a brief history. In: Koerner EFK, Asher RE, editors. Concise history of the language sciences. Amsterdam: Pergamon; 1995. p. 431\u201345. https:\/\/doi.org\/10.1016\/B978-0-08-042580-1.50066-0."},{"key":"61_CR26","doi-asserted-by":"publisher","unstructured":"Popovi\u0107 M. Error classification and analysis for machine translation quality assessment. In: Moorkens J, Castilho S, Gaspari F, Doherty S, editors. Translation quality assessment: from principles to practice. Machine translation: technologies and applications. Cham: Springer International Publishing; 2018. pp. 129\u2013158. https:\/\/doi.org\/10.1007\/978-3-319-91241-7_7.","DOI":"10.1007\/978-3-319-91241-7_7"},{"issue":"2","key":"61_CR27","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1017\/S1351324919000469","volume":"26","author":"E Chatzikoumi","year":"2020","unstructured":"Chatzikoumi E. How to evaluate machine translation: a review of automated and human metrics. Nat Lang Eng. 2020;26(2):137\u201361. https:\/\/doi.org\/10.1017\/S1351324919000469.","journal-title":"Nat Lang Eng"},{"key":"61_CR28","doi-asserted-by":"crossref","unstructured":"Alotaibi H (2023) Arabic-English parallel corpus: a new resource for translation training and language teaching. Arab World Engl J AWEJ 2017;8(3). https:\/\/awej.org\/arabic-english-parallel-corpus-a-new-resource-for-translation-training-and-language-teaching\/. Accessed 26 Jul 2023.","DOI":"10.24093\/awej\/vol8no3.21"},{"key":"61_CR29","unstructured":"marefa-nlp\/marefa-mt-en-ar \u00b7 Hugging Face. https:\/\/huggingface.co\/marefa-nlp\/marefa-mt-en-ar. Accessed 19 Jul 2023."},{"key":"61_CR30","unstructured":"Helsinki-NLP\/opus-mt-tc-big-ar-en \u00b7 Hugging Face. https:\/\/huggingface.co\/Helsinki-NLP\/opus-mt-tc-big-ar-en. Accessed 19 Jul 2023."},{"key":"61_CR31","unstructured":"facebook\/m2m100_1.2B \u00b7 Hugging Face. https:\/\/huggingface.co\/facebook\/m2m100_1.2B. Accessed 19 Jul 2023."},{"issue":"1","key":"61_CR32","first-page":"107:4839","volume":"22","author":"A Fan","year":"2021","unstructured":"Fan A, et al. Beyond English-centric multilingual machine translation. J Mach Learn Res. 2021;22(1):107:4839-107:4886.","journal-title":"J Mach Learn Res"},{"key":"61_CR33","unstructured":"OpenAI Platform. https:\/\/platform.openai.com. Accessed 26 Jul 2023."},{"issue":"9","key":"61_CR34","doi-asserted-by":"publisher","first-page":"10137","DOI":"10.1007\/s10462-023-10423-5","volume":"56","author":"SK Mondal","year":"2023","unstructured":"Mondal SK, Zhang H, Kabir HMD, Ni K, Dai H-N. Machine translation and its evaluation: a study. Artif Intell Rev. 2023;56(9):10137\u2013226. https:\/\/doi.org\/10.1007\/s10462-023-10423-5.","journal-title":"Artif Intell Rev"},{"key":"61_CR35","doi-asserted-by":"publisher","unstructured":"Lommel A. Metrics for translation quality assessment: a case for standardising error typologies. In: Moorkens J, Castilho S, Gaspari F, Doherty S, editors. Translation quality assessment: from principles to practice. Machine translation: technologies and applications. Cham: Springer International Publishing; 2018, pp. 109\u2013127. https:\/\/doi.org\/10.1007\/978-3-319-91241-7_6.","DOI":"10.1007\/978-3-319-91241-7_6"},{"key":"61_CR36","doi-asserted-by":"publisher","unstructured":"Popovi\u0107 M. chrF: character n-gram F-score for automatic MT evaluation. In: Bojar O, Chatterjee R, Federmann C, Haddow B, Hokamp C, Huck M, Logacheva V, Pecina P, editors. Proceedings of the tenth workshop on statistical machine translation. Lisbon, Portugal: Association for Computational Linguistics; 2015. pp. 392\u2013395. https:\/\/doi.org\/10.18653\/v1\/W15-3049.","DOI":"10.18653\/v1\/W15-3049"},{"key":"61_CR37","doi-asserted-by":"publisher","unstructured":"Zhang T, Kishore V, Wu F, Weinberger KQ, Artzi Y. BERTScore: evaluating text generation with BERT. 2020. https:\/\/doi.org\/10.48550\/arXiv.1904.09675.","DOI":"10.48550\/arXiv.1904.09675"},{"key":"61_CR38","doi-asserted-by":"publisher","unstructured":"Rei R, Stewart C, Farinha AC, Lavie A. COMET: a neural framework for MT evaluation. 2020. https:\/\/doi.org\/10.48550\/arXiv.2009.09025.","DOI":"10.48550\/arXiv.2009.09025"},{"key":"61_CR39","doi-asserted-by":"publisher","unstructured":"Lyu C, Xu J, Wang L. New trends in machine translation using large language models: case examples with ChatGPT. 2023. https:\/\/doi.org\/10.48550\/arXiv.2305.01181.","DOI":"10.48550\/arXiv.2305.01181"}],"container-title":["Human-Centric Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44230-024-00061-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44230-024-00061-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44230-024-00061-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,5]],"date-time":"2024-06-05T07:31:10Z","timestamp":1717572670000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44230-024-00061-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,5]]},"references-count":39,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["61"],"URL":"https:\/\/doi.org\/10.1007\/s44230-024-00061-7","relation":{},"ISSN":["2667-1336"],"issn-type":[{"value":"2667-1336","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,5]]},"assertion":[{"value":"3 October 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not Applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not Applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"The authors hereby grant full consent for the publication of the manuscript in the HCIN journal.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}