{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T16:01:59Z","timestamp":1784563319891,"version":"3.55.0"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T00:00:00Z","timestamp":1784505600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T00:00:00Z","timestamp":1784505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002322","name":"Coordena\u00e7\u00e3o de Aperfei\u00e7oamento de Pessoal de N\u00edvel Superior","doi-asserted-by":"publisher","award":["Finance Code 001"],"award-info":[{"award-number":["Finance Code 001"]}],"id":[{"id":"10.13039\/501100002322","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Data Sci Anal"],"published-print":{"date-parts":[[2026,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Automatic poetry translation remains a challenging task, as it requires not only semantic accuracy but also the preservation of stylistic and emotional elements. This study investigates the effectiveness of machine translation (MT) systems and large language models (LLMs) in poetry translation. Traditional automatic evaluation metrics often fail to capture the literary quality of such translations; therefore, we propose a three-phase evaluation framework that integrates complementary perspectives: (i) automatic metrics (BLEU, METEOR, and BERTScore) to assess lexical and semantic fidelity, (ii) topic modeling with BERTopic to perform an exploratory analysis of lexical-thematic alignment across translations, and (iii) expert human evaluation to examine poetic structure, style, fluency, and meaning preservation. This framework was applied to compare specialized MT systems (mBART, MarianMT, OpenNMT with RNN, and Google Translate) with LLMs such as ChatGPT and Maritaca AI across six language pairs involving English, French, and Portuguese (English\u2013French, English\u2013Portuguese, French\u2013English, French\u2013Portuguese, Portuguese\u2013English, and Portuguese\u2013French). Additionally, we investigate the impact of fine-tuning strategies using corpora of poems and song lyrics. The results show that LLMs and Google Translate consistently outperform specialized MT systems in terms of fluency, meaning preservation, and lexical-thematic alignment. However, human evaluation reveals that all systems struggle to replicate the poetic structure and stylistic nuances of the originals. The fine-tuning process did not produce improvements for all models; mBART showed notable gains, while the other models did not benefit significantly from domain adaptation.<\/jats:p>","DOI":"10.1007\/s41060-026-01206-6","type":"journal-article","created":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T15:02:28Z","timestamp":1784559748000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating the limits of machine translation for poetry: a multidimensional framework"],"prefix":"10.1007","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-2552-7938","authenticated-orcid":false,"given":"Beatriz Ribeiro","family":"Borges","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5923-4181","authenticated-orcid":false,"given":"Paulo Henrique Ribeiro","family":"Gabriel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5242-9026","authenticated-orcid":false,"given":"Elaine Ribeiro","family":"Faria","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,7,20]]},"reference":[{"issue":"4","key":"1206_CR1","doi-asserted-by":"publisher","first-page":"61","DOI":"10.11648\/j.ijalt.20241004.12","volume":"10","author":"E Ozturk","year":"2024","unstructured":"Ozturk, E.: Preservation of cultural authenticity: analysing cultural transfer in the translation process. Int. J. Appl. Linguist. Transl. 10(4), 61\u201366 (2024)","journal-title":"Int. J. Appl. Linguist. Transl."},{"key":"1206_CR2","unstructured":"Britto, P.H.: A Tradu\u00e7\u00e3o Liter\u00e1ria. Filosofia, Literatura e Artes. Civiliza\u00e7\u00e3o Brasileira, Rio de Janeiro (2012)"},{"key":"1206_CR3","doi-asserted-by":"crossref","unstructured":"Jones, F.R.: The translation of poetry. In: The Oxford Handbook of Translation Studies, pp. 169\u2013182. Oxford University Press, Oxford (2011)","DOI":"10.1093\/oxfordhb\/9780199239306.013.0013"},{"key":"1206_CR4","doi-asserted-by":"publisher","first-page":"1810","DOI":"10.18653\/v1\/P19-1176","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Q Wang","year":"2019","unstructured":"Wang, Q., Li, B., Xiao, T., Zhu, J., Li, C., Wong, D.F., Chao, L.S.: Learning deep transformer models for machine translation. In: Korhonen, A., Traum, D., M\u00e0rquez, L. (eds.) Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1810\u20131822. Association for Computational Linguistics, Florence, Italy (2019)"},{"key":"1206_CR5","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1162\/tacl_a_00642","volume":"12","author":"Z He","year":"2024","unstructured":"He, Z., Liang, T., Jiao, W., Zhang, Z., Yang, Y., Wang, R., Tu, Z., Shi, S., Wang, X.: Exploring human-like translation strategy with large language models. Trans. Assoc. Comput. Linguist. 12, 229\u2013246 (2024)","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"1206_CR6","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., Zhu, W.-J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp. 311\u2013318. Association for Computational Linguistics, Philadelphia, Pennsylvania (2002)","DOI":"10.3115\/1073083.1073135"},{"key":"1206_CR7","unstructured":"Banerjee, S., Lavie, A.: Meteor: an automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation And\/or Summarization, pp. 65\u201372. Association for Computational Linguistics, Ann Arbor, Michigan (2005)"},{"key":"1206_CR8","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: Bertscore: evaluating text generation with bert. In: Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), Addis Ababa, Ethiopia (2020). https:\/\/openreview.net\/forum?id=SkeHuCVFDr"},{"key":"1206_CR9","doi-asserted-by":"publisher","first-page":"2675","DOI":"10.5220\/0014315100004052","volume":"3","author":"BR Borges","year":"2026","unstructured":"Borges, B.R., Gabriel, P.H.R., Faria, E.R.D.: Evaluating translation models and llms in poetic texts. Proceedings of the 18th International Conference on Agents and Artificial Intelligence 3, 2675\u20132682 (2026)","journal-title":"Proceedings of the 18th International Conference on Agents and Artificial Intelligence"},{"key":"1206_CR10","unstructured":"Grootendorst, M.: Bertopic: neural topic modeling with a class-based tf-idf procedure, (2022). arXiv:2203.05794 arXiv preprint"},{"key":"1206_CR11","unstructured":"Tang, Y., Tran, C., Li, X., Chen, P.-J., Goyal, N., Chaudhary, V., Gu, J., Fan, A.: Multilingual translation with extensible multilingual pretraining and finetuning. arXiv preprint abs\/2008.00401 (2020) arXiv:2008.00401"},{"key":"1206_CR12","doi-asserted-by":"crossref","unstructured":"Junczys-Dowmunt, M., Grundkiewicz, R., Dwojak, T., Hoang, H., Heafield, K., Neckermann, T., Seide, F., Germann, U., Aji, A.F., Bogoychev, N., Martins, A.F.T., Birch, A.: Marian: fast neural machine translation in c. In: Proceedings of ACL 2018, System Demonstrations, pp. 116\u2013121. Association for Computational Linguistics, Melbourne, Australia (2018)","DOI":"10.18653\/v1\/P18-4020"},{"key":"1206_CR13","doi-asserted-by":"crossref","unstructured":"Pires, R., Abonizio, H., Almeida, T.S., Nogueira, R.: Sabi\u00e1: Portuguese large language models. In: Proceedings of the 12th Brazilian Conference on Intelligent Systems (BRACIS), pp. 226\u2013240. Sociedade Brasileira de Computa\u00e7\u00e3o, Belo Horizonte\/MG (2023)","DOI":"10.1007\/978-3-031-45392-2_15"},{"key":"1206_CR14","doi-asserted-by":"crossref","unstructured":"Klein, G., Kim, Y., Deng, Y., Senellart, J., Rush, A.: Opennmt: open-source toolkit for neural machine translation. In: Proceedings of ACL 2017, System Demonstrations, pp. 67\u201372. Association for Computational Linguistics, Vancouver, Canada (2017)","DOI":"10.18653\/v1\/P17-4012"},{"key":"1206_CR15","unstructured":"Cespedosa\u00a0V\u00e1zquez, A.I., Mitkov, R.: Machine translation of literary texts: genres, times and systems. In: Proceedings of the 1st Workshop on NLP Tools and Resources for Translation and Interpretation Applications, pp. 48\u201353. INCOMA Ltd., Varna, Bulgaria (2023)"},{"issue":"2","key":"1206_CR16","doi-asserted-by":"publisher","first-page":"41","DOI":"10.5007\/2175-8026.2019v72n2p41","volume":"72","author":"P Humbl\u00e9","year":"2019","unstructured":"Humbl\u00e9, P.: Machine translation and poetry. The case of english and portuguese. Ilha Desterro 72(2), 41\u201356 (2019)","journal-title":"Ilha Desterro"},{"key":"1206_CR17","doi-asserted-by":"publisher","unstructured":"Ghazvininejad, M., Choi, Y., Knight, K.: Neural poetry translation. In: Proceedings of NAACL-HLT, pp. 67\u201371. Association for Computational Linguistics, New Orleans, Louisiana (2018). https:\/\/doi.org\/10.18653\/v1\/N18-2011","DOI":"10.18653\/v1\/N18-2011"},{"key":"1206_CR18","unstructured":"Song, W.L., Xu, H., Wong, D.F., Zhan, R., Chao, L.S., Wang, S.: Towards zero-shot multilingual poetry translation. In: Utiyama, M., Wang, R.: (eds.) Proceedings of the Machine Translation Summit XIX (Vol. 1: Research Track), pp. 324\u2013335. Asia-Pacific Association for Machine Translation, Macau SAR, China (2023)"},{"key":"1206_CR19","first-page":"178","volume-title":"Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Vol. 1: Research Track)","author":"N Resende","year":"2024","unstructured":"Resende, N., Hadley, J.: The translator\u2019s canvas: using llms to enhance poetry translation. In: Knowles, R., Eriguchi, A., Goel, S. (eds.) Proceedings of the 16th Conference of the Association for Machine Translation in the Americas (Vol. 1: Research Track), pp. 178\u2013189. Association for Machine Translation in the Americas, Chicago, USA (2024)"},{"key":"1206_CR20","doi-asserted-by":"crossref","unstructured":"Chakrabarty, T., Saakyan, A., Muresan, S.: Don\u2019t go far off: An empirical study on neural poetry translation. In: Moens, M.F., Huang, X., Specia, L., Yih, S.W.T.: (eds.) Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7253\u20137265. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic (2021)","DOI":"10.18653\/v1\/2021.emnlp-main.577"},{"issue":"2","key":"1206_CR21","doi-asserted-by":"publisher","first-page":"268","DOI":"10.62411\/jcta.11635","volume":"2","author":"O Babalola","year":"2024","unstructured":"Babalola, O., Ojokoh, B., Boyinbode, O.: Comprehensive evaluation of lda, nmf, and bertopic\u2019s performance on news headline topic modeling. J. Comput. Theor. Appl. 2(2), 268\u2013289 (2024)","journal-title":"J. Comput. Theor. Appl."},{"key":"1206_CR22","doi-asserted-by":"publisher","first-page":"189","DOI":"10.18653\/v1\/2026.latechclfl-1.18","volume-title":"Proceedings of the 10th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences,Humanities and Literature 2026","author":"J-F Klumpp","year":"2026","unstructured":"Klumpp, J.-F.: Quantitative analysis of rhyme and metre in LLM-generated translations of poetry. In: Alves, D., Bizzoni, Y., Degaetano-Ortlieb, S., Kazantseva, A., Pagel, J., Szpakowicz, S. (eds.) Proceedings of the 10th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences,Humanities and Literature 2026, pp. 189\u2013201. Association for Computational Linguistics, Rabat, Morocco (2026). https:\/\/doi.org\/10.18653\/v1\/2026.latechclfl-1.18"},{"key":"1206_CR23","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/978-981-33-4909-4_25","volume-title":"International Conference on Communication, Computing and Electronics Systems","author":"K Gangar","year":"2021","unstructured":"Gangar, K., Ruparel, H., Lele, S.: Hindi to english: transformer-based neural machine translation. In: Bindhu, V., Tavares, J.M.R.S., Boulogeorgos, A.-A.A., Vuppalapati, C. (eds.) International Conference on Communication, Computing and Electronics Systems, pp. 337\u2013347. Springer, Singapore (2021)"},{"key":"1206_CR24","doi-asserted-by":"publisher","DOI":"10.17632\/gpmg8hdshh.1","author":"BR Borges","year":"2025","unstructured":"Borges, B.R.: Multilingual corpus of poems with human reference translations for literary machine translation research. Mendeley Data (2025). https:\/\/doi.org\/10.17632\/gpmg8hdshh.1","journal-title":"Mendeley Data"},{"key":"1206_CR25","doi-asserted-by":"crossref","unstructured":"Bichri, H., Chergui, A., Hain, M.: Investigating the impact of train \/ test split ratio on the performance of pre-trained models with custom datasets. Int. J. Adv. Comput. Sci. Appl. 15(2), (2024)","DOI":"10.14569\/IJACSA.2024.0150235"},{"key":"1206_CR26","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 6000\u20136010. MIT Press, Cambridge, MA (2017)"},{"key":"1206_CR27","doi-asserted-by":"publisher","first-page":"3982","DOI":"10.18653\/v1\/D19-1410","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"N Reimers","year":"2019","unstructured":"Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Inui, K., Jiang, J., Ng, V., Wan, X. (eds.) Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982\u20133992. Association for Computational Linguistics, Hong Kong, China (2019). https:\/\/doi.org\/10.18653\/v1\/D19-1410"},{"key":"1206_CR28","volume-title":"Foundations of Statistical Natural Language Processing","author":"CD Manning","year":"1999","unstructured":"Manning, C.D., Sch\u00fctze, H.: Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA (1999)"},{"key":"1206_CR29","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1057\/s41599-024-03363-0","volume":"11","author":"R Gao","year":"2024","unstructured":"Gao, R., Lin, Y., Zhao, N., et al.: Machine translation of chinese classical poetry: a comparison among chatgpt, google translate, and deepl translator. Humanities and Social Sciences Communications 11, 835 (2024)","journal-title":"Humanities and Social Sciences Communications"},{"key":"1206_CR30","first-page":"80","volume-title":"Versos, Sons, Ritmos","author":"N Goldstein","year":"2005","unstructured":"Goldstein, N.: Versos, Sons, Ritmos, 13th edn., p. 80. S\u00e3o Paulo, Editora \u00c1tica (2005)","edition":"13"},{"key":"1206_CR31","doi-asserted-by":"publisher","first-page":"135","DOI":"10.18653\/v1\/2025.eval4nlp-1.12","volume-title":"Proceedings of the 5th Workshop on Evaluation and Comparison of NLP Systems","author":"B At\u0131l","year":"2025","unstructured":"At\u0131l, B., Aykent, S., Chittams, A., Fu, L., Passonneau, R.J., Radcliffe, E., Rajagopal, G.R., Sloan, A., Tudrej, T., Ture, F., Wu, Z., Xu, L., Baldwin, B.: Non-determinism of \u201cdeterministic\u2019\u2019 LLM system settings in hosted environments. In: Proceedings of the 5th Workshop on Evaluation and Comparison of NLP Systems, pp. 135\u2013148. Association for Computational Linguistics, Mumbai, India (2025)"},{"key":"1206_CR32","doi-asserted-by":"publisher","DOI":"10.3390\/app13158566","author":"M Khan","year":"2023","unstructured":"Khan, M., Ullah, K., Alharbi, Y., Alferaidi, A., Alharbi, T.S., Yadav, K., Alsharabi, N., Ahmad, A.: Understanding the research challenges in low-resource language and linking bilingual news articles in multilingual news archive. Appl. Sci. (2023). https:\/\/doi.org\/10.3390\/app13158566","journal-title":"Appl. Sci."},{"key":"1206_CR33","doi-asserted-by":"publisher","unstructured":"Koo, T.K., Li, M.Y.: A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J. Chiropr. Med. 15(2), 155\u2013163 (2016). https:\/\/doi.org\/10.1016\/j.jcm.2016.02.012","DOI":"10.1016\/j.jcm.2016.02.012"},{"key":"1206_CR34","doi-asserted-by":"publisher","unstructured":"Walsh, M., Preus, A., Antoniak, M.: Sonnet or not, bot? poetry evaluation for large models and datasets. In: Al-Onaizan, Y., Bansal, M., Chen, Y.-N. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 15568\u201315603. Association for Computational Linguistics, Miami, Florida, USA (2024). https:\/\/doi.org\/10.18653\/v1\/2024.findings-emnlp.914","DOI":"10.18653\/v1\/2024.findings-emnlp.914"}],"container-title":["International Journal of Data Science and Analytics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-026-01206-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41060-026-01206-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41060-026-01206-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T15:02:35Z","timestamp":1784559755000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41060-026-01206-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,20]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,12]]}},"alternative-id":["1206"],"URL":"https:\/\/doi.org\/10.1007\/s41060-026-01206-6","relation":{},"ISSN":["2364-415X","2364-4168"],"issn-type":[{"value":"2364-415X","type":"print"},{"value":"2364-4168","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,20]]},"assertion":[{"value":"24 February 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 June 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 July 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no conflict of interest.","order":1,"name":"Ethics","label":"Conflict of interest","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"237"}}