{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T13:34:46Z","timestamp":1773840886757,"version":"3.50.1"},"reference-count":65,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T00:00:00Z","timestamp":1758153600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Exploring Arabic dialects in Natural Language Processing (NLP) is essential to understand linguistic variation and meet regional communication demands. Recent advances in Large Language Models (LLMs) have opened up new vistas for multilingual communication and text generation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>This paper investigates the performance of GPT-3.5, GPT-4, and Bard (Gemini) on the QADI and MADAR datasets, while GPT-5 was evaluated exclusively on MADAR encompassing over 15 different countries. Several metrics have been used in the evaluation, such as cosine similarity, universal similarity encoder, sentence BERT, TER, ROUGE, and BLEU. In this study, different prompting techniques were used: zero-shot and few-shot. Zero-shot was employed for all dialects, and few-shot was employed only for the least translation performance dialect, Tunisian.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Analysis revealed that in the QADI dataset, GPT-4 significantly outperformed others in translating MSA to DA, with ANOVA tests showing strong significance (<jats:italic>p<\/jats:italic> &amp;lt; 0.05) in most metrics, except for BLEU and TER where it does not show significance, indicating comparable translation performance among models. Furthermore, GPT-4 was highest in semantic similarity compared to GPT-3.5 and Bard (Gemini), 0.66, 0.61, and 0.63, respectively. GPT-4 was the best in identifying overlapping sentences (i.e., those where the source and target are identical) with a combined average of 0.41 in BLEU and ROUGE-L. All LLMs scored TER values between 6% and 25%, indicating generally good translation quality. However, GPT models, especially GPT-5, responded better to prompting and translation to Levant countries compared to Bard (Gemini). For the MADAR dataset, no significant translation differences were observed in sentence-BERT, ROUGE-L, and TER, while differences are identified in cosine similarity, BLEU, and universal similarity encoder metrics. Therefore, GPT-5 is the top performer in identifying sentence overlaps measured by BLEU and ROUGE-L (combined average 0.37).<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The few-shot approach did not show a significant improvement in translation performance, especially for GPT-4 and Bard (Gemini), while GPT-3.5 performed consistently. Zero-shot prompts were effective across dialects, while few-shot prompting, applied to the weakest-performing dialect (Tunisian), did not yield improvement. GPT-4 and Bard performed worse under this set-up, while GPT-3.5 remained consistent.<\/jats:p><\/jats:sec>","DOI":"10.3389\/frai.2025.1661789","type":"journal-article","created":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T05:31:22Z","timestamp":1758173482000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Cross-dialectal Arabic translation: comparative analysis on large language models"],"prefix":"10.3389","volume":"8","author":[{"given":"Ayah","family":"Beidas","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kousar","family":"Mohi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fatme","family":"Ghaddar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Imtiaz","family":"Ahmad","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sa'Ed","family":"Abed","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,9,18]]},"reference":[{"key":"B1","article-title":"Arabic dialect identification in the wild","author":"Abdelali","year":"2020","journal-title":"arXiv preprint arXiv:2005.06557"},{"key":"B2","first-page":"112","article-title":"\u201cLLM-based MT data creation: Dialectal to MSA translation shared task,\u201d","author":"Abdelaziz","year":"2024","journal-title":"Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation@ LREC-COLING 2024"},{"key":"B3","first-page":"91","article-title":"\u201cShifting boundaries: the effect of msa on dialect convergence in Baghdad,\u201d","volume-title":"Perspectives on Arabic Linguistics: Papers from the Annual Symposium on Arabic Linguistics. Volume IV: Detroit, Michigan 1990","author":"Abu-Haidar","year":"2011"},{"key":"B4","article-title":"Advancing dialectal Arabic to modern standard Arabic machine translation","author":"Alabdullah","year":"2025","journal-title":"arXiv preprint arXiv:2507.20301"},{"key":"B5","first-page":"117","article-title":"\u201cSirius_Translators at OSACT6 2024 shared task: fin-tuning ara-T5 models for translating arabic dialectal text to modern standard Arabic,\u201d","volume-title":"Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation@ LREC-COLING 2024","author":"Alahmari","year":"2024"},{"key":"B6","first-page":"39","article-title":"A method to convert Sana'ani accent to modern standard Arabic","volume":"8","author":"Al-Gaphari","year":"2010","journal-title":"Int. J. Inf. Sci. Manag"},{"key":"B7","first-page":"679","article-title":"Fine-tuned transformers for translating multi-dialect texts to modern standard Arabic","volume":"18","author":"Alimi","year":"2024","journal-title":"Int. J. Cogn. Lang. Sci"},{"key":"B8","first-page":"547","article-title":"\u201cA simple zero-shot prompt weighting technique to improve prompt ensembling in text-image models,\u201d","volume-title":"International Conference on Machine Learning","author":"Allingham","year":"2023"},{"key":"B9","article-title":"Taqyim: evaluating Arabic NLP tasks using ChatGPT models","author":"Alyafeai","year":"2023","journal-title":"arXiv preprint arXiv:2306.16322"},{"key":"B10","first-page":"98","article-title":"\u201cOSACT 2024 task 2: Arabic dialect to MSA translation,\u201d","author":"Atwany","year":"2024","journal-title":"Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation@ LREC-COLING 2024"},{"key":"B11","doi-asserted-by":"publisher","first-page":"592","DOI":"10.18653\/v1\/2020.coling-main.51","article-title":"\u201cArabizi language models for sentiment analysis,\u201d","author":"Baert","year":"2020","journal-title":"Proceedings of the 28th International Conference on Computational Linguistics"},{"key":"B12","article-title":"\u201cA hybrid approach for converting written egyptian colloquial dialect into diacritized Arabic,\u201d","volume-title":"The 6th International Conference on Informatics and Systems, infos2008. Cairo University","author":"Bakr","year":"2008"},{"key":"B13","doi-asserted-by":"publisher","first-page":"3666","DOI":"10.3390\/math10193666","article-title":"A reverse positional encoding multi-head attention-based neural machine translation model for Arabic dialects","volume":"10","author":"Baniata","year":"2022","journal-title":"Mathematics"},{"key":"B14","doi-asserted-by":"publisher","first-page":"7534712","DOI":"10.1155\/2018\/7534712","article-title":"A neural machine translation model for Arabic dialects that utilizes multitask learning (MTL)","volume":"2018","author":"Baniata","year":"2018","journal-title":"Comput. Intell. Neurosci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1080\/13645579.2016.1252188","article-title":"Assessing the use of back translation: the shortcomings of back translation as a quality testing method","volume":"20","author":"Behr","year":"2017","journal-title":"Int. J. Soc. Res. Methodol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"187","DOI":"10.26615\/978-954-452-092-2_021","article-title":"\u201cGenerative models for indic languages: evaluating content generation capabilities,\u201d","author":"Bhat","year":"2023","journal-title":"Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing"},{"key":"B17","first-page":"1240","article-title":"\u201cA multidialectal parallel corpus of Arabic,\u201d","volume-title":"Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14), Reykjavik, Iceland","author":"Bouamor","year":"2014"},{"key":"B18","doi-asserted-by":"publisher","first-page":"199","DOI":"10.18653\/v1\/W19-4622","article-title":"\u201cThe MADAR shared task on Arabic fine-grained dialect identification,\u201d","author":"Bouamor","year":"2019","journal-title":"Proceedings of the Fourth Arabic Natural Language Processing Workshop"},{"key":"B19","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1109\/ISET61814.2024.00021","article-title":"\u201cGPT and translation: a systematic review,\u201d","volume-title":"2024 International Symposium on Educational Technology (ISET)","author":"Chan","year":"2024"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3641289","article-title":"A survey on evaluation of large language models","volume":"15","author":"Chang","year":"2023","journal-title":"ACM Trans. Intell. Syst. Technol"},{"key":"B21","doi-asserted-by":"publisher","first-page":"139","DOI":"10.18653\/v1\/2023.acl-short.14","article-title":"\u201cScaling in cognitive modelling: a multilingual approach to human reading times,\u201d","author":"De Varda","year":"2023","journal-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.arabicnlp-1.80","article-title":"\u201cArabic train at NADI 2024 shared task: LLMs' ability to translate Arabic dialects into Modern Standard Arabic,\u201d","volume-title":"Proceedings of the Second Arabic Natural Language Processing Conference","author":"Demidova","year":"2024"},{"key":"B23","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2018","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"B24","author":"Eberhard","year":"2019","journal-title":"Ethnologue: Languages of Asia, Twenty-Second Edition"},{"key":"B25","article-title":"\u201cNeural vs. statistical translation of Algerian Arabic dialect written with Arabizi and Arabic letter,\u201d","author":"Guellil","year":"2017","journal-title":"The 31st Pacific Asia Conference on Language, Information and Computation Paclic"},{"key":"B26","doi-asserted-by":"publisher","first-page":"497","DOI":"10.1016\/j.jksuci.2019.02.006","article-title":"Arabic natural language processing: an overview","volume":"33","author":"Guellil","year":"2021","journal-title":"J. King Saud Univ. Comput. Inf. Sci"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.36227\/techrxiv.23589741.v2","article-title":"Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects","volume":"1","author":"Hadi","year":"2023","journal-title":"Authorea Preprints"},{"key":"B28","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1007\/978-3-319-67056-0_7","author":"Hamada","year":"2018","journal-title":"Developing a Transfer-Based System for Arabic Dialects Translation"},{"key":"B29","first-page":"12","article-title":"\u201cLahjawi: Arabic cross-dialect translator,\u201d","author":"Hamed","year":"2025","journal-title":"Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4)"},{"key":"B30","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1016\/j.ipm.2017.08.003","article-title":"Machine translation for Arabic dialects (survey)","volume":"56","author":"Harrat","year":"2019","journal-title":"Inf. Proc. Manag"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3491101.3503564","article-title":"\u201cPromptmaker: prompt-based prototyping with large language models,\u201d","author":"Jiang","year":"2022","journal-title":"CHI Conference on Human Factors in Computing Systems Extended Abstracts"},{"key":"B32","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1162\/tacl_a_00324","article-title":"How can we know what language models know?","volume":"8","author":"Jiang","year":"2020","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B33","first-page":"1","article-title":"An empirical evaluation of Arabic text formality transfer: a comparative study","volume":"2024","author":"Jibrin","year":"2025","journal-title":"Lang. Resour. Evaluat"},{"key":"B34","article-title":"Natural language processing for dialects of a language: a survey","author":"Joshi","year":"2024","journal-title":"arXiv preprint arXiv:2401.05632"},{"key":"B35","article-title":"TARJAMAT: evaluation of bard and chatGPT on machine translation of ten Arabic varieties","author":"Kadaoui","year":"2023","journal-title":"arXiv preprint arXiv:2308.03051"},{"key":"B36","doi-asserted-by":"publisher","first-page":"102274","DOI":"10.1016\/j.lindif.2023.102274","article-title":"ChatGPT for good? On opportunities and challenges of large language models for education","volume":"103","author":"Kasneci","year":"2023","journal-title":"Learn. Indiv. Differ"},{"key":"B37","doi-asserted-by":"publisher","first-page":"350","DOI":"10.3102\/00346543068003350","article-title":"Statistical practices of educational researchers: an analysis of their ANOVA, MANOVA, and ANCOVA analyses","volume":"68","author":"Keselman","year":"1998","journal-title":"Rev. Educ. Res"},{"key":"B38","article-title":"SentimentGPT: exploiting GPT for advanced sentiment analysis and its departure from current machine learning","author":"Kheiri","year":"2023","journal-title":"arXiv preprint arXiv:2307.10234"},{"key":"B39","first-page":"50","article-title":"\u201cDial2MSA-verified: a multi-dialect arabic social media dataset for neural machine translation to modern standard Arabic,\u201d","author":"Khered","year":"2025","journal-title":"Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4)"},{"key":"B40","article-title":"GPTAraEval: a comprehensive evaluation of ChatGPT on Arabic NLP","author":"Khondaker","year":"2023","journal-title":"arXiv preprint arXiv:2305.14976"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-2814154\/v1","author":"Khoshafah","year":"2023","journal-title":"ChatGPT for Arabic-English translation: Evaluating the accuracy"},{"key":"B42","article-title":"ArabianGPT: Native Arabic GPT-based Large Language","author":"Koubaa","year":"2024","journal-title":"arXiv preprint arXiv:2402.15313"},{"key":"B43","first-page":"235","article-title":"ChatGPT-4 and Italian dialects: assessing linguistic competence","volume":"16","author":"Lilli","year":"2023","journal-title":"Umanistica Digitale"},{"key":"B44","doi-asserted-by":"publisher","first-page":"150","DOI":"10.3115\/1073445.1073465","article-title":"\u201cAutomatic evaluation of summaries using N-gram co-occurrence statistics,\u201d","author":"Lin","year":"2003","journal-title":"Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics"},{"key":"B45","doi-asserted-by":"publisher","first-page":"100032","DOI":"10.1016\/j.nlp.2023.100032","article-title":"GPT-3.5, GPT-4, or BARD? Evaluating LLMs reasoning ability in zero-shot setting and performance boosting through prompts","volume":"5","author":"L\u00f3pez Espejel","year":"2023","journal-title":"Nat. Lang. Proc. J"},{"key":"B46","article-title":"AraFinNlp 2024: The first Arabic financial NLP shared task","author":"Malaysha","year":"2024","journal-title":"arXiv preprint arXiv:2407.09818"},{"key":"B47","article-title":"A survey of large language models for Arabic language and its dialectss","author":"Mashaabi","year":"2024","journal-title":"arXiv preprint arXiv:2410.20238"},{"key":"B48","first-page":"176","article-title":"\u201cTransforming standard arabic to colloquial Arabic,\u201d","author":"Mohamed","year":"2012","journal-title":"Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"},{"key":"B49","doi-asserted-by":"publisher","first-page":"1396","DOI":"10.1109\/TASLP.2022.3161160","article-title":"SBSim: a sentence-BERT similarity-based evaluation metric for indian language neural machine translation systems","volume":"30","author":"Mrinalini","year":"2022","journal-title":"IEEE\/ACM Trans. Audio, Speech, Lang. Proc"},{"key":"B50","first-page":"159","article-title":"Machine translation approaches: issues and challenges","volume":"11","author":"Okpor","year":"2014","journal-title":"Int. J. Comput. Sci. Issues"},{"key":"B51","doi-asserted-by":"publisher","first-page":"311","DOI":"10.3115\/1073083.1073135","article-title":"\u201cBLEU: a method for automatic evaluation of machine translation,\u201d","author":"Papineni","year":"2002","journal-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics"},{"key":"B52","article-title":"The RefinedWeb dataset for falcon LLM: outperforming curated corpora with web data only","author":"Penedo","year":"2023","journal-title":"arXiv preprint arXiv:2306.01116"},{"key":"B53","article-title":"Towards making the most of ChatGPT for machine translation","author":"Peng","year":"2023","journal-title":"arXiv preprint arXiv:2303.13780"},{"key":"B54","first-page":"1","article-title":"\u201cA hybrid approach to translate Moroccan Arabic dialect,\u201d","volume-title":"2014 9th International Conference on Intelligent Systems: Theories and Applications (SITA-14)","author":"Ridouane","year":"2014"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.58496\/MJAIH\/2024\/001","article-title":"Evaluating ChatGPT performance in Arabic dialects: a comparative study showing defects in responding to Jordanian and Tunisian general health prompts","volume":"2024","author":"Sallam","year":"2024","journal-title":"Mesopot. J. Artif. Intell. Healthcare"},{"key":"B56","first-page":"385","article-title":"\u201cElissa: a dialectal to standard Arabic machine translation system,\u201d","author":"Salloum","year":"2012","journal-title":"Proceedings of COLING 2012: Demonstration Papers"},{"key":"B57","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1016\/j.procs.2020.08.033","article-title":"Rule-based machine translation from Tunisian dialect to modern standard Arabic","volume":"176","author":"Sghaier","year":"2020","journal-title":"Procedia Comput. Sci"},{"key":"B58","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1109\/FIT60620.2023.00047","article-title":"\u201cExploring the potential of large-language models (LLMs) for student feedback sentiment analysis,\u201d","volume-title":"2023 International Conference on Frontiers of Information Technology (FIT)","author":"Shaikh","year":"2023"},{"key":"B59","article-title":"SHAMI-MT: a Syrian Arabic dialect to modern standard Arabic bidirectional machine translation system","author":"Sibaee","year":"2025","journal-title":"arXiv preprint arXiv:2508.02268"},{"key":"B60","doi-asserted-by":"publisher","first-page":"100160","DOI":"10.1016\/j.caeai.2023.100160","article-title":"To GPT or not GPT? Empowering our students to learn with AI","volume":"5","author":"Steele","year":"2023","journal-title":"Comput. Educ"},{"key":"B61","author":"Wright","year":"2011","journal-title":"A Grammar of the Arabic Language"},{"key":"B62","article-title":"BLOOM+1: adding language support to BLOOM for zero-shot prompting","author":"Yong","year":"2022","journal-title":"arXiv preprint arXiv:2212.09535"},{"key":"B63","author":"Yu","year":"2023"},{"key":"B64","first-page":"307","article-title":"\u201cSentiment analysis methods for hpv vaccines related tweets based on transfer learning,\u201d","volume-title":"Healthcare","author":"Zhang","year":"2020"},{"key":"B65","article-title":"Multilingual machine translation with large language models: empirical results and analysis","author":"Zhu","year":"2023","journal-title":"arXiv preprint arXiv:2304.04675."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1661789\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T05:31:28Z","timestamp":1758173488000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1661789\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,18]]},"references-count":65,"alternative-id":["10.3389\/frai.2025.1661789"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1661789","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,18]]},"article-number":"1661789"}}