{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T19:30:49Z","timestamp":1772652649680,"version":"3.50.1"},"reference-count":36,"publisher":"Elsevier BV","issue":"4","license":[{"start":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T00:00:00Z","timestamp":1741564800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T00:00:00Z","timestamp":1741564800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001230","name":"Macquarie University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001230","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Artif Intell Educ"],"published-print":{"date-parts":[[2025,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>As Machine Translation (MT) technologies become more advanced, the translation errors they generate are often increasingly subtle. When MT is integrated in \u2018Human-in-the-Loop\u2019 (HITL) translation workflows for specialized domains, successful Post-Editing (PE) hinges on the humans involved having in-depth subject competence, as knowledge of the specific terminology and conventions are essential to produce accurate translations. One way of assessing an individual\u2019s expertise is through manual translation tests, a method traditionally used by Language Service Providers (LSPs) and translator educators alike. While manual evaluation can provide the most comprehensive overview of a translator\u2019s abilities, they have the disadvantage of being time-consuming and costly, especially when large numbers of subjects and language pairs are involved. In this work, we report on the experience of creating automated tests with GPT-4 for assessing the ability to recognize domain-specific specialized terminology correspondence in the translation of English-to-Turkish engineering texts in HITL translation workflows. While there may be a level of usefulness in the resulting tests, they are not fit for direct implementation without further refinement.<\/jats:p>","DOI":"10.1007\/s40593-025-00465-x","type":"journal-article","created":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T13:29:35Z","timestamp":1741613375000},"page":"2185-2201","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Creating Terminological Correspondence Recognition Tests with GPT-4: A Case Study in English-to-Turkish Translations in the Engineering Domain"],"prefix":"10.1016","volume":"35","author":[{"given":"Marina","family":"S\u00e1nchez-Torr\u00f3n","sequence":"first","affiliation":[]},{"given":"Egemen","family":"Ipek","sequence":"additional","affiliation":[]},{"given":"Vanessa Enr\u00edquez","family":"Ra\u00eddo","sequence":"additional","affiliation":[]}],"member":"78","published-online":{"date-parts":[[2025,3,10]]},"reference":[{"issue":"3","key":"465_CR1","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1080\/15434303.2019.1635134","volume":"16","author":"D Allen","year":"2019","unstructured":"Allen, D. (2019). Cognate frequency predicts accuracy in tests of lexical knowledge. Language Assessment Quarterly, 16(3), 312\u2013327. https:\/\/doi.org\/10.1080\/15434303.2019.1635134","journal-title":"Language Assessment Quarterly"},{"issue":"2","key":"465_CR2","doi-asserted-by":"publisher","first-page":"211","DOI":"10.3138\/cmlr.2820","volume":"72","author":"R Batista","year":"2016","unstructured":"Batista, R., & Horst, M. (2016). A new receptive vocabulary size test for French. Canadian Modern Language Review, 72(2), 211\u2013233. https:\/\/doi.org\/10.3138\/cmlr.2820","journal-title":"Canadian Modern Language Review"},{"key":"465_CR3","doi-asserted-by":"publisher","unstructured":"Beerepoot, M. T. P. (2023). Formative and summative automated assessment with multiple-choice question Banks. Journal of Chemical Education 100 (8)10. https:\/\/doi.org\/10.1021\/acs.jchemed.3c00120","DOI":"10.1021\/acs.jchemed.3c00120"},{"key":"465_CR4","doi-asserted-by":"publisher","first-page":"75","DOI":"10.6035\/MonTI.2024.16.02","volume":"16","author":"V Briva-Iglesias","year":"2024","unstructured":"Briva-Iglesias, V., Dogru, G., & Cavalheiro Camargo, J. L. (2024). Large language models \u201cad referendum\u201d: How good are they at machine translation in the legal domain? MonTI. Monographs in Translation and Interpreting, 16, 75\u2013107. https:\/\/doi.org\/10.6035\/MonTI.2024.16.02","journal-title":"MonTI. Monographs in Translation and Interpreting"},{"key":"465_CR5","doi-asserted-by":"crossref","unstructured":"Cabr\u00e9, T. (1999). Terminolog\u00eda: Representaci\u00f3n y Comunicaci\u00f3n. Elementos para una teor\u00eda de base comunicativa y otros art\u00edculos. Barcelona: IULA. Universidad Pompeu Fabra.","DOI":"10.1075\/tlrp.1"},{"key":"465_CR6","unstructured":"Cabr\u00e9, T. (2004). La terminolog\u00eda en la traducci\u00f3n especializada. In Manual de documentaci\u00f3n y terminolog\u00eda para la traducci\u00f3n especializada, Gonzalo Garc\u00eda and Garc\u00eda Yebra (eds.), Madrid: Arco Libros. Instrumenta bibliologica, 2004, pp. 89\u2013126."},{"key":"465_CR7","unstructured":"Castilho, S., Quinn Mallon, C., Meister, R., & Yue, S. (2023). Do online machine translation systems care for context? What about a GPT model? In Proceedings of the 24th Annual Conference of the European Association for Machine Translation (pp. 393\u2013417). European Association for Machine Translation. https:\/\/aclanthology.org\/2023.eamt-1.39. Retrieved February 17, 2024."},{"key":"465_CR8","unstructured":"Dijkstra, R., Genc, Z., Kayal, S., & Kamps, J. (2022). Reading comprehension quiz generation using generative pre-trained transformers. In S. Sosnovsky, P. Brusilovsky, & A. Lan (Eds.), Proceedings of the Fourth International Workshop on Intelligent Textbooks 2022: Co-located with 23rd International Conference on Artificial Intelligence in Education (AIED 2022) (pp. 4\u201317). http:\/\/ceur-ws.org\/Vol-3192\/itb22_p1_full5439.pdf"},{"issue":"2","key":"465_CR9","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1177\/0265532212459028","volume":"30","author":"I Elgort","year":"2013","unstructured":"Elgort, I. (2013). Effects of L1 definitions and cognate status of test items on the vocabulary size test. Language Testing, 30(2), 253\u2013272. https:\/\/doi.org\/10.1177\/0265532212459028","journal-title":"Language Testing"},{"key":"465_CR10","first-page":"95","volume":"1","author":"P Faber","year":"2003","unstructured":"Faber, P. (2003). Terminological competence and enhanced knowledge acquisition. Research in Language, 1, 95\u2013116.","journal-title":"Research in Language"},{"key":"465_CR11","doi-asserted-by":"publisher","unstructured":"Fleming, S. L., Morse, K., Kumar, A., Chiang, C.-C., Patel, B., Brunskill, E., & Shah, N. (2023). Assessing the potential of USMLE-like exam questions generated by GPT-4. medRxiv. https:\/\/doi.org\/10.1101\/2023.04.25.23288588","DOI":"10.1101\/2023.04.25.23288588"},{"key":"465_CR12","doi-asserted-by":"publisher","unstructured":"Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education. https:\/\/doi.org\/10.2196\/45312","DOI":"10.2196\/45312"},{"key":"465_CR13","doi-asserted-by":"publisher","unstructured":"Gonsalves, C. (2023). On ChatGPT: what promise remains for multiple choice assessment?. Journal of Learning Development in Higher Education, (27). https:\/\/doi.org\/10.47408\/jldhe.vi27.1009","DOI":"10.47408\/jldhe.vi27.1009"},{"issue":"1","key":"465_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/1750399X.2016.1154339","volume":"10","author":"M Gonz\u00e1lez-Davies","year":"2016","unstructured":"Gonz\u00e1lez-Davies, M., & Enr\u00edquez-Ra\u00eddo, V. (2016). Situated learning in translator and interpreter training: Bridging research and good practice. The Interpreter and Translator Trainer, 10(1), 1\u201311. https:\/\/doi.org\/10.1080\/1750399X.2016.1154339","journal-title":"The Interpreter and Translator Trainer"},{"key":"465_CR15","doi-asserted-by":"publisher","unstructured":"Guallar, J., & Lopezosa, C. (2024). Inteligencia artificial, desinformaci\u00f3n y aspectos \u00e9ticos. In M. Ribera & O. D\u00edaz Montesdeoca (Eds.), ChatGPT y educaci\u00f3n universitaria. Posibilidades y l\u00edmites de ChatGPT como herramienta docente (pp. 87\u201396). A - Llibres Universitat (IDP-ICE, Octaedro). https:\/\/doi.org\/10.36006\/15224-1","DOI":"10.36006\/15224-1"},{"issue":"3","key":"465_CR16","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1207\/S15324818AME1503_5","volume":"15","author":"TM Haladyna","year":"2002","unstructured":"Haladyna, T. M., Downing, S. M., & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309\u2013334. https:\/\/doi.org\/10.1207\/S15324818AME1503_5","journal-title":"Applied Measurement in Education"},{"key":"465_CR17","unstructured":"Hickey, S (2024). The 2024 Nimdzi 100: The ranking of the top 100 largest Language Service Providers. https:\/\/www.nimdzi.com\/nimdzi-100-top-lsp\/. Retrieved June 22, 2024."},{"issue":"9","key":"465_CR18","doi-asserted-by":"publisher","first-page":"4271","DOI":"10.1007\/s00405-023-08051-4","volume":"280","author":"CC Hoch","year":"2023","unstructured":"Hoch, C. C., Wollenberg, B., L\u00fcers, J. C., Knoedler, S., Knoedler, L., Frank, K., Cotofana, S., & Alfertshofer, M. (2023b). ChatGPT\u2019s quiz skills in different otolaryngology subspecialties: An analysis of 2576 single-choice and multiple-choice board certification preparation questions. European Archives of Oto-Rhino-Laryngology: Official Journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS), 280(9), 4271\u20134278. https:\/\/doi.org\/10.1007\/s00405-023-08051-4","journal-title":"European Archives of Oto-Rhino-Laryngology: Official Journal of the European Federation of Oto-Rhino-Laryngological Societies (EUFOS)"},{"key":"465_CR19","doi-asserted-by":"publisher","unstructured":"Hoch, C. C., Wollenberg, B., L\u00fcers, J. C., Knoedler, S., Knoedler, L., Frank, K., Cotofana, S., & Alfertshofer, M. (2023). ChatGPT\u2019s quiz skills in different otolaryngology subspecialties: an analysis of 2576 single-choice and multiple-choice board certification preparation questions.\u00a0European Archives of Oto-Rhino-Laryngology 280(9) https:\/\/doi.org\/10.1007\/s00405-023-08051-4","DOI":"10.1007\/s00405-023-08051-4"},{"key":"465_CR20","doi-asserted-by":"publisher","unstructured":"Ionescu, V. M., & Enescu, M. C. (2023). Using ChatGPT for generating and evaluating online tests. 15th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Bucharest, Romania, 2023, pp. 1\u20136. https:\/\/doi.org\/10.1109\/ECAI58194.2023.10193995","DOI":"10.1109\/ECAI58194.2023.10193995"},{"key":"465_CR21","unstructured":"Kocmi, T., Federmann, C., Grundkiewicz, R., Junczys-Dowmunt, M., Matsushita, H., & Menezes, A. (2021). To ship or not to ship: An extensive evaluation of automatic metrics for machine translation. In Proceedings of the Sixth Conference on Machine Translation (pp. 478\u2013494). Association for Computational Linguistics. https:\/\/aclanthology.org\/2021.wmt-1.57. Retrieved April 17, 2024."},{"key":"465_CR22","unstructured":"L\u00f3pez, E., & Mart\u00edn Guti\u00e9rrez, S. (2023). Gu\u00eda para integrar las tecnolog\u00edas basadas en inteligencia artificial generativa en los procesos de ense\u00f1anza y aprendizaje. Vicerrectorado de Innovaci\u00f3n Educativa, UNED. http:\/\/fediap.com.ar\/wp-content\/uploads\/2023\/12\/Gu_a_para_integrar_las_tecnolog_as_basadas_en_IAG_1702048753-1.pdf. Retrieved February 17, 2024."},{"key":"465_CR23","doi-asserted-by":"publisher","unstructured":"Montero Mart\u00ednez, S. & Faber, P. (2009). Terminological competence in translation. Terminology. Special Issue on Teaching and Learning Terminology: New Strategies and Methods, 15(1): 88\u2013104. https:\/\/doi.org\/10.1075\/term.15.1.05mon","DOI":"10.1075\/term.15.1.05mon"},{"key":"465_CR24","doi-asserted-by":"publisher","unstructured":"Newton, P. M. (2023a). ChatGPT performance on MCQ-based exams. A pragmatic scoping review, Assessment & Evaluation in Higher Education 0(0), 1\u201318. Routledge.https:\/\/doi.org\/10.1080\/02602938.2023.2299059","DOI":"10.1080\/02602938.2023.2299059"},{"key":"465_CR25","unstructured":"Newton, P. M. (2023b). Online exams in the age of ChatGPT; now what? https:\/\/www.youtube.com\/watch?v=YloLWCO3qWY. Retrieved February 22, 2024."},{"key":"465_CR26","unstructured":"OpenAI. (2023). GPT-4 technical report. https:\/\/arxiv.org\/abs\/2303.08774v3. Retrieved February 17, 2024."},{"key":"465_CR27","doi-asserted-by":"publisher","unstructured":"Raftery, D. (2023). Will ChatGPT pass the online quizzes? Adapting an assessment strategy in the age of generative AI. Irish Journal of Technology Enhanced Learning, 7(1).https:\/\/doi.org\/10.22554\/ijtel.v7i1.114","DOI":"10.22554\/ijtel.v7i1.114"},{"key":"465_CR28","doi-asserted-by":"publisher","unstructured":"Robinson, N., Ogayo, P., Mortensen, D. R., & Neubig, G. (2023). ChatGPT MT: Competitive for high- (but not low-) resource languages. Proceedings of the Eighth Conference on Machine Translation (pp. 392\u2013418). Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2023.wmt-1.40","DOI":"10.18653\/v1\/2023.wmt-1.40"},{"key":"465_CR29","unstructured":"Shah, P. (2023). AI and the future of education: Teaching in the age of artificial intelligence (1st Ed.). Indianapolis, IN: Jossey-Bass"},{"key":"465_CR30","doi-asserted-by":"publisher","unstructured":"Siu, S. C. (2023). ChatGPT and GPT-4 for professional translators: Exploring the potential of large language models in translation.\u00a0SSRN Electronic Journalhttps:\/\/doi.org\/10.2139\/ssrn.4448091","DOI":"10.2139\/ssrn.4448091"},{"key":"465_CR31","unstructured":"Slator. (2023). Language industry market report. https:\/\/slator.com\/2023-language-industry-market-report\/. Retrieved February 13, 2024."},{"issue":"1","key":"465_CR32","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s40561-023-00237-x","volume":"10","author":"A Tlili","year":"2023","unstructured":"Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https:\/\/doi.org\/10.1186\/s40561-023-00237-x","journal-title":"Smart Learning Environments"},{"key":"465_CR33","doi-asserted-by":"publisher","unstructured":"Tu, X., Zou, J., Su, W., & Zhang, L. (2024). What should data science education do with large language models?. Harvard Data Science Review, 6(1). https:\/\/doi.org\/10.1162\/99608f92.bff007ab","DOI":"10.1162\/99608f92.bff007ab"},{"key":"465_CR34","doi-asserted-by":"publisher","unstructured":"Wang, L., Lyu, C., Ji, T., Zhang, Z., Yu, D., Shi, S., & Tu, Z. (2023). Document-Level machine translation with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 16646\u201316661). Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2023.emnlp-main.1036","DOI":"10.18653\/v1\/2023.emnlp-main.1036"},{"issue":"2","key":"465_CR35","doi-asserted-by":"publisher","first-page":"5","DOI":"10.14744\/felt.2021.3.2.2","volume":"3","author":"X Yu","year":"2021","unstructured":"Yu, X. (2021). Creating a frequency-based Turkish-English loanword cognates word list (TELCWL). Focus on ELT Journal, 3(2), 5\u201335. https:\/\/doi.org\/10.14744\/felt.2021.3.2.2","journal-title":"Focus on ELT Journal"},{"key":"465_CR36","unstructured":"Zhang, B., Haddow, B., & Birch, A. (2023). Prompting large language model for machine translation: A case study. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato & J. Scarlett (Eds.), Proceedings of the 40th International Conference on Machine Learning. vol. 202, Proceedings of Machine Learning Research, PMLR, (pp. 41092-41110), The Fortieth International Conference on Machine Learning."}],"container-title":["International Journal of Artificial Intelligence in Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-025-00465-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40593-025-00465-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40593-025-00465-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T18:12:39Z","timestamp":1772647959000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40593-025-00465-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,10]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12]]}},"alternative-id":["465"],"URL":"https:\/\/doi.org\/10.1007\/s40593-025-00465-x","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-4187415\/v1","asserted-by":"object"}]},"ISSN":["1560-4292","1560-4306"],"issn-type":[{"value":"1560-4292","type":"print"},{"value":"1560-4306","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,10]]},"assertion":[{"value":"22 February 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 March 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Competing Interests","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}}]}}