{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T10:22:52Z","timestamp":1774952572760,"version":"3.50.1"},"reference-count":48,"publisher":"Emerald","issue":"3","license":[{"start":{"date-parts":[[2024,7,5]],"date-time":"2024-07-05T00:00:00Z","timestamp":1720137600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJICC"],"published-print":{"date-parts":[[2024,7,17]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-subheading\">Purpose<\/jats:title><jats:p>The paper aims to enhance Arabic machine translation (MT) by proposing novel approaches: (1) a dimensionality reduction technique for word embeddings tailored for Arabic text, optimizing efficiency while retaining semantic information; (2) a comprehensive comparison of meta-embedding techniques to improve translation quality; and (3) a method leveraging self-attention and Gated CNNs to capture token dependencies, including temporal and hierarchical features within sentences, and interactions between different embedding types. These approaches collectively aim to enhance translation quality by combining different embedding schemes and leveraging advanced modeling techniques.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Design\/methodology\/approach<\/jats:title><jats:p>Recent works on MT in general and Arabic MT in particular often pick one type of word embedding model. In this paper, we present a novel approach to enhance Arabic MT by addressing three key aspects. Firstly, we propose a new dimensionality reduction technique for word embeddings, specifically tailored for Arabic text. This technique optimizes the efficiency of embeddings while retaining their semantic information. Secondly, we conduct an extensive comparison of different meta-embedding techniques, exploring the combination of static and contextual embeddings. Through this analysis, we identify the most effective approach to improve translation quality. Lastly, we introduce a novel method that leverages self-attention and Gated convolutional neural networks (CNNs) to capture token dependencies, including temporal and hierarchical features within sentences, as well as interactions between different types of embeddings. Our experimental results demonstrate the effectiveness of our proposed approach in significantly enhancing Arabic MT performance. It outperforms baseline models with a BLEU score increase of 2 points and achieves superior results compared to state-of-the-art approaches, with an average improvement of 4.6 points across all evaluation metrics.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Findings<\/jats:title><jats:p>The proposed approaches significantly enhance Arabic MT performance. The dimensionality reduction technique improves the efficiency of word embeddings while preserving semantic information. Comprehensive comparison identifies effective meta-embedding techniques, with the contextualized dynamic meta-embeddings (CDME) model showcasing competitive results. Integration of Gated CNNs with the transformer model surpasses baseline performance, leveraging both architectures' strengths. Overall, these findings demonstrate substantial improvements in translation quality, with a BLEU score increase of 2 points and an average improvement of 4.6 points across all evaluation metrics, outperforming state-of-the-art approaches.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-subheading\">Originality\/value<\/jats:title><jats:p>The paper\u2019s originality lies in its departure from simply fine-tuning the transformer model for a specific task. Instead, it introduces modifications to the internal architecture of the transformer, integrating Gated CNNs to enhance translation performance. This departure from traditional fine-tuning approaches demonstrates a novel perspective on model enhancement, offering unique insights into improving translation quality without solely relying on pre-existing architectures. The originality in dimensionality reduction lies in the tailored approach for Arabic text. While dimensionality reduction techniques are not new, the paper introduces a specific method optimized for Arabic word embeddings. By employing independent component analysis (ICA) and a post-processing method, the paper effectively reduces the dimensionality of word embeddings while preserving semantic information which has not been investigated before especially for MT task.<\/jats:p><\/jats:sec>","DOI":"10.1108\/ijicc-03-2024-0106","type":"journal-article","created":{"date-parts":[[2024,7,4]],"date-time":"2024-07-04T11:58:25Z","timestamp":1720094305000},"page":"605-631","source":"Crossref","is-referenced-by-count":3,"title":["Contextualized dynamic meta embeddings based on Gated CNNs and self-attention for Arabic machine translation"],"prefix":"10.1108","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8747-4409","authenticated-orcid":false,"given":"Nouhaila","family":"Bensalah","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Habib","family":"Ayad","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdellah","family":"Adib","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdelhamid","family":"Ibn El Farouk","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","published-online":{"date-parts":[[2024,7,5]]},"reference":[{"key":"key2024071612512418700_ref001","doi-asserted-by":"publisher","first-page":"6030","DOI":"10.18653\/v1\/2020.coling-main.530","article-title":"The SADID evaluation datasets for low-resource spoken language machine translation of Arabic dialects","year":"2020"},{"key":"key2024071612512418700_ref002","article-title":"A hybrid neural machine translation technique for translating low resource languages","year":"2018"},{"key":"key2024071612512418700_ref003","article-title":"A recipe for Arabic-English neural machine translation","year":"2018","journal-title":"ArXiv abs\/1808.06116"},{"key":"key2024071612512418700_ref004","article-title":"Layer normalization","year":"2016","journal-title":"CoRR abs\/1607.06450"},{"key":"key2024071612512418700_ref005","article-title":"Neural machine translation by jointly learning to align and translate","year":"2015"},{"key":"key2024071612512418700_ref006","first-page":"65","article-title":"METEOR: an automatic metric for MT evaluation with improved correlation with human judgments","year":"2005"},{"key":"key2024071612512418700_ref007","doi-asserted-by":"crossref","unstructured":"Bensalah, N., Ayad, H., Adib, A. and Farouk, A.I.E. (2020), \u201cArabic Machine Translation based on the combination of word embedding techniques\u201d, in Intelligent Systems in Big Data, Semantic Web and Machine Learning.","DOI":"10.1007\/978-3-030-72588-4_17"},{"key":"key2024071612512418700_ref008","article-title":"LSTM vs GRU for Arabic machine translation","year":"2021"},{"key":"key2024071612512418700_ref009","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1007\/978-3-031-07969-6_30","article-title":"Transformer model and convolutional neural networks (CNNs) for Arabic to English machine translation","year":"2022"},{"key":"key2024071612512418700_ref010","doi-asserted-by":"crossref","unstructured":"Bensalah, N., Ayad, H., Adib, A. and Ibn El Farouk, A. (2022b), \u201cCRAN: an hybrid CNN-RNN attention-based model for Arabic machine translation\u201d, in Networking, Intelligent Systems and Security.","DOI":"10.1007\/978-981-16-3637-0_7"},{"key":"key2024071612512418700_ref011","doi-asserted-by":"publisher","first-page":"778","DOI":"10.1007\/978-3-031-26384-2_69","article-title":"Improving Arabic to English machine translation","year":"2023"},{"key":"key2024071612512418700_ref012","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1007\/978-3-031-27524-1_7","article-title":"Arabic machine translation based on the combination of word embedding techniques","year":"2023"},{"issue":"12","key":"key2024071612512418700_ref013","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3634681","article-title":"A comparative study of different dimensionality reduction techniques for Arabic machine translation","volume":"22","year":"2023","journal-title":"ACM Transactions on Asian and Low-Resource Language Information Processing"},{"key":"key2024071612512418700_ref014","first-page":"993","article-title":"Latent dirichlet\u00a0allocation","volume":"3","year":"2003","journal-title":"Journal of Machine Learning Research"},{"key":"key2024071612512418700_ref015","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","year":"2017","journal-title":"Transactions of the Association for Computational Linguistics"},{"issue":"3","key":"key2024071612512418700_ref016","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1016\/0165-1684(94)90029-9","article-title":"Independent component analysis, A new concept?","volume":"36","year":"1994","journal-title":"Signal Processing"},{"key":"key2024071612512418700_ref017","first-page":"933","article-title":"Language modeling with gated convolutional networks","year":"2017"},{"key":"key2024071612512418700_ref018","first-page":"4171","article-title":"BERT: pre-training of deep bidirectional transformers for language understanding","year":"2019"},{"key":"key2024071612512418700_ref019","first-page":"1243","article-title":"Convolutional sequence to sequence learning","year":"2017"},{"key":"key2024071612512418700_ref020","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/cvpr.2016.90","article-title":"Deep residual learning for image recognition","year":"2016"},{"key":"key2024071612512418700_ref021","volume-title":"Independent Component Analysis","year":"2001"},{"key":"key2024071612512418700_ref022","first-page":"1700","article-title":"Recurrent continuous translation models","year":"2013"},{"issue":"2","key":"key2024071612512418700_ref023","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1109\/tac.1980.1102314","article-title":"The singular value decomposition: its computation and some applications","volume":"25","year":"1980","journal-title":"IEEE Transactions on Automatic Control"},{"key":"key2024071612512418700_ref024","first-page":"1106","article-title":"Imagenet classification with deep convolutional neural networks","year":"2012"},{"key":"key2024071612512418700_ref025","doi-asserted-by":"publisher","article-title":"Word embeddings with limited memory","year":"2016","DOI":"10.18653\/v1\/p16-2063"},{"key":"key2024071612512418700_ref026","doi-asserted-by":"publisher","first-page":"726","DOI":"10.1162\/tacl_a_00343","article-title":"Multilingual denoising pre-training for neural machine translation","volume":"8","year":"2020","journal-title":"Transactions of the Association for Computational Linguistics"},{"key":"key2024071612512418700_ref027","doi-asserted-by":"publisher","first-page":"1412","DOI":"10.18653\/v1\/d15-1166","article-title":"Effective approaches to attention-based neural machine translation","year":"2015"},{"key":"key2024071612512418700_ref028","first-page":"142","article-title":"Learning word vectors for sentiment analysis","year":"2011"},{"key":"key2024071612512418700_ref029","first-page":"6294","article-title":"Learned in translation: contextualized word vectors","year":"2017"},{"key":"key2024071612512418700_ref030","doi-asserted-by":"crossref","unstructured":"Mesleh, A.M. (2008), \u201cSupport vector machines based Arabic language text classification system: feature selection comparative study\u201d, in Advances in Computer and Information Sciences and Engineering, pp.\u00a011-16.","DOI":"10.1007\/978-1-4020-8741-7_3"},{"key":"key2024071612512418700_ref031","article-title":"Efficient estimation of word representations in vector space","year":"2013"},{"key":"key2024071612512418700_ref032","article-title":"All-but-the-top: simple and effective postprocessing for word representations","year":"2018"},{"issue":"02","key":"key2024071612512418700_ref033","doi-asserted-by":"publisher","DOI":"10.1142\/s0219649224500096","article-title":"Assessing the impact of static, contextual and character embeddings for Arabic machine translation","volume":"23","year":"2024","journal-title":"Journal of Information and Knowledge Management"},{"key":"key2024071612512418700_ref034","first-page":"214","article-title":"The impact of preprocessing on Arabic-English statistical and neural machine translation","year":"2019"},{"key":"key2024071612512418700_ref035","article-title":"Bleu: a method for automatic evaluation of machine translation","year":"2002"},{"issue":"11","key":"key2024071612512418700_ref036","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1080\/14786440109462720","article-title":"On lines and planes of closest fit to systems of points in space","volume":"2","year":"1901","journal-title":"Philosophical Magazine"},{"key":"key2024071612512418700_ref037","doi-asserted-by":"publisher","first-page":"1532","DOI":"10.3115\/v1\/d14-1162","article-title":"Glove: global vectors for word representation","year":"2014"},{"key":"key2024071612512418700_ref038","doi-asserted-by":"crossref","unstructured":"Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K. and Zettlemoyer, L. (2018), \u201cDeep contextualized word representations\u201d, in NAACL-HLT, Association for Computational Linguistics, pp.\u00a02227-2237.","DOI":"10.18653\/v1\/N18-1202"},{"key":"key2024071612512418700_ref039","article-title":"DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter","year":"2019","journal-title":"CoRR abs\/1910.01108"},{"key":"key2024071612512418700_ref040","article-title":"Very deep convolutional networks for large-scale image recognition","year":"2015"},{"key":"key2024071612512418700_ref041","article-title":"Study of translation edit rate with targeted human annotation","year":"2006"},{"key":"key2024071612512418700_ref042","first-page":"2214","article-title":"Parallel data, tools and interfaces in OPUS","year":"2012"},{"key":"key2024071612512418700_ref043","first-page":"5998","article-title":"Attention is all you need","year":"2017"},{"key":"key2024071612512418700_ref044","unstructured":"Wang, Y.-Y., Acero, A. and Chelba, C. (2003), \u201cIs word error rate a good indicator for spoken language understanding accuracy\u201d, in 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No. 03EX721), pp.\u00a0577-582."},{"key":"key2024071612512418700_ref045","article-title":"Linformer: self-attention with linear complexity","year":"2020","journal-title":"CoRR abs\/2006.04768"},{"key":"key2024071612512418700_ref046","article-title":"Words or characters? Fine-grained gating for reading comprehension","year":"2017"},{"key":"key2024071612512418700_ref047","first-page":"5754","article-title":"XLNet: generalized autoregressive pretraining for language understanding","year":"2019"},{"key":"key2024071612512418700_ref048","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1007\/978-3-319-18111-0_32","article-title":"Word representations in vector space and their applications for Arabic","year":"2015"}],"container-title":["International Journal of Intelligent Computing and Cybernetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJICC-03-2024-0106\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/IJICC-03-2024-0106\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,24]],"date-time":"2025-07-24T22:54:12Z","timestamp":1753397652000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ijicc\/article\/17\/3\/605-631\/1228081"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,5]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,7,5]]},"published-print":{"date-parts":[[2024,7,17]]}},"alternative-id":["10.1108\/IJICC-03-2024-0106"],"URL":"https:\/\/doi.org\/10.1108\/ijicc-03-2024-0106","relation":{},"ISSN":["1756-378X"],"issn-type":[{"value":"1756-378X","type":"print"}],"subject":[],"published":{"date-parts":[[2024,7,5]]}}}