{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T05:37:07Z","timestamp":1792906627335,"version":"4.5.0"},"reference-count":56,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T00:00:00Z","timestamp":1791936000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T00:00:00Z","timestamp":1791936000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Background\/Objectives: Machine translation increasingly prepares multilingual product information for EU marketing authorisation, yet no established process exists to adapt or validate a system for the Summary of Product Characteristics (SmPC), whose sections differ in regulatory function, terminology, and clinical risk. This work develops such a process and a qualification procedure against the approved human translation. Methods: An EMA-derived ontology divides the SmPC into seven regulatory domains, each with a provisional ICH Q9 risk profile, following Quality-by-Design principles. A No Language Left Behind model was fine-tuned with Low-Rank Adaptation on an eight-language-pair corpus and compared with the pre-trained model using BLEU, chrF, TER and COMET. Quality was validated by a five-layer, human-sign-off pipeline. Results: Fine-tuning improved every metric for every language pair. Quality varied more by domain than by language, hidden by corpus-level scores, and the domains rated highest-risk, under provisional ratings awaiting external calibration, were not the best translated. Checks on language-independent quantities were reliable everywhere; within adverse-reaction sections, the terminology check was the only partial-anchor instrument, and it showed meaningful gains only after a second language-model stage. Conclusions: The fine-tuned model and its validation pipeline, targeting automated checks where reliable and human review elsewhere, form a promising assistive step under qualified sign-off whose suitability for production-scale deployment requires further validation; assessment must be by regulatory domain, not the document as a whole.<\/jats:p>","DOI":"10.3390\/info17101022","type":"journal-article","created":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T14:14:14Z","timestamp":1791987254000},"page":"1022","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Quality-by-Design Framework for Evaluating AI-Driven Machine Translation in Regulatory Affairs"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9434-7993","authenticated-orcid":false,"given":"Darya","family":"Filatova","sequence":"first","affiliation":[{"name":"Digital & Innovation, Product Life Group, Immeuble Colisee Gardens, 10 Av. de l\u2019Arche Batiment C, 7eme Etage, 92400 Courbevoie, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2995-2810","authenticated-orcid":false,"given":"Hocine","family":"Kadi","sequence":"additional","affiliation":[{"name":"Digital & Innovation, Product Life Group, Immeuble Colisee Gardens, 10 Av. de l\u2019Arche Batiment C, 7eme Etage, 92400 Courbevoie, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4361-3571","authenticated-orcid":false,"given":"Florian","family":"Pereme","sequence":"additional","affiliation":[{"name":"Digital & Innovation, Product Life Group, Immeuble Colisee Gardens, 10 Av. de l\u2019Arche Batiment C, 7eme Etage, 92400 Courbevoie, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1016\/j.jtha.2025.09.041","article-title":"A comprehensive care pathway of gene therapy for hemophilia based on current guideline documents and summary of product characteristics: Communication from the ISTH SSC working group on gene therapy","volume":"24","author":"Mussert","year":"2026","journal-title":"J. Thromb. Haemost."},{"key":"ref_2","first-page":"100810","article-title":"Predictive modeling of error categories in English-Slovak machine translation using automatic evaluation metrics","volume":"23","author":"Munkova","year":"2026","journal-title":"Mach. Learn. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"131062","DOI":"10.1016\/j.eswa.2025.131062","article-title":"Research challenges and future directions in transformer-based neural machine translation","volume":"309","author":"Banerjee","year":"2026","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bojar, O., Chatterjee, R., Federmann, C., Fishel, M., Graham, Y., Haddow, B., Huck, M., Yepes, A.J., Koehn, P., and Monz, C. (2018). Findings of the WMT 2018 Biomedical Translation Shared Task: Evaluation on Medline test sets. Proceedings of the Third Conference on Machine Translation: Shared Task Papers, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-6401"},{"key":"ref_5","unstructured":"Barrault, L., Bojar, O., Bougares, F., Chatterjee, R., Costa-juss\u00e0, M.R., Federmann, C., Fishel, M., Fraser, A., Graham, Y., and Guzman, P. (2020). Findings of the WMT 2020 Biomedical Translation Shared Task: Basque, Italian and Russian as New Additional Languages. Proceedings of the Fifth Conference on Machine Translation, Association for Computational Linguistics."},{"key":"ref_6","unstructured":"Barrault, L., Bojar, O., Bougares, F., Chatterjee, R., Costa-jussa, M.R., Federmann, C., Fishel, M., Fraser, A., Freitag, M., and Graham, Y. (2021). Findings of the WMT 2021 Biomedical Translation Shared Task: Summaries of Animal Experiments as New Test Set. Proceedings of the Sixth Conference on Machine Translation, Association for Computational Linguistics."},{"key":"ref_7","unstructured":"Koehn, P., Barrault, L., Bojar, O., Bougares, F., Chatterjee, R., Costa-juss\u00e0, M.R., Federmann, C., Fishel, M., Fraser, A., and Freitag, M. (2022). Findings of the WMT 2022 Biomedical Translation Shared Task: Monolingual Clinical Case Reports. Proceedings of the Seventh Conference on Machine Translation (WMT), Association for Computational Linguistics."},{"key":"ref_8","unstructured":"Haddow, B., Kocmi, T., Koehn, P., and Monz, C. (2024). Findings of the WMT 2024 Biomedical Translation Shared Task: Test Sets on Abstract Level. Proceedings of the Ninth Conference on Machine Translation, Association for Computational Linguistics."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bojar, O., Buck, C., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Yepes, A.J., Koehn, P., and Kreutzer, J. (2017). Findings of the WMT 2017 Biomedical Translation Shared Task. Proceedings of the Second Conference on Machine Translation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W17-4717"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"37992","DOI":"10.1038\/s41598-025-21867-w","article-title":"A lightweight large language model for regulatory affairs translation in pharmaceutical industry","volume":"15","author":"Chen","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kadi, H., Abdellatif, A., Kemajou Njamen, D., and Pereme, F. (2025). Automatic extraction of SmPC document for IDMP data model construction using foundation LLM and RAG: A preliminary experiment for pharmaceutical regulatory affairs. Front. Med., 1.","DOI":"10.3389\/fmed.2025.1598979"},{"key":"ref_12","first-page":"6579","article-title":"MedCOD: Enhancing English-to-Spanish Medical Translation of Large Language Models Using Enriched Chain-of-Dictionary Framework","volume":"2025","author":"Salim","year":"2025","journal-title":"Front. Med."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Mehandru, N., Robertson, S., and Salehi, N. (2022, January 21\u201324). Reliable and Safe Use of Machine Translation in Medical Settings. Proceedings of the FAccT \u201922: 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea.","DOI":"10.1145\/3531146.3533244"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"101582","DOI":"10.1016\/j.csl.2023.101582","article-title":"Adopting machine translation in the healthcare sector: A methodological multi-criteria review","volume":"84","author":"Zappatore","year":"2024","journal-title":"Comput. Speech Lang."},{"key":"ref_15","unstructured":"Haddow, B., Kocmi, T., Koehn, P., and Monz, C. (2025). Specification-Aware Machine Translation and Evaluation for Purpose Alignment. Proceedings of the Tenth Conference on Machine Translation, Association for Computational Linguistics."},{"key":"ref_16","unstructured":"Conia, S., Li, M., Navigli, R., and Potdar, S. (August, January 31). SemEval-2025 Task 2: Entity-Aware Machine Translation. Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), Vienna, Austria."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"101623","DOI":"10.1016\/j.csl.2024.101623","article-title":"Translating scientific abstracts in the bio-medical domain with structure-aware models","volume":"87","author":"Yvon","year":"2024","journal-title":"Comput. Speech Lang."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1613\/jair.1.13566","article-title":"Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey","volume":"75","author":"Saunders","year":"2022","journal-title":"JAIR"},{"key":"ref_19","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022). LoRA: Low-Rank Adaptation of Large Language Models. arXiv."},{"key":"ref_20","unstructured":"Zheng, J., Hong, H., Liu, F., Wang, X., Su, J., Liang, Y., and Wu, S. (2024). Fine-tuning Large Language Models for Domain-Specific Machine Translation. arXiv."},{"key":"ref_21","unstructured":"McGiff, J., and Nikolov, N.S. (2025). SemiAdapt and SemiLoRA: Efficient Domain Adaptation for Transformer-Based Low-Resource Language Translation with a Case Study on Irish. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Acharya, P., Mondal, H., Saha, D., Das, D., and Bandyopadhyay, S. (2025). JU-NLP: Improving Low-Resource Indic Translation System with Efficient LoRA-Based Adaptation. Proceedings of the Tenth Conference on Machine Translation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.wmt-1.95"},{"key":"ref_23","unstructured":"Garcia-Estrada, E., Escolano, C., and Fonallosa, J.A.R. (2026). Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective. arXiv."},{"key":"ref_24","unstructured":"NLLB Team, Costa-Juss\u00e0, M.R., Cross, J., \u00c7elebi, O., Elbayad, M., Heafield, K., Heffernan, K., Kalbassi, E., Lam, J., and Licht, D. (2022). No Language Left Behind: Scaling Human-Centered Machine Translation. arXiv."},{"key":"ref_25","unstructured":"Wassie, A.K., Molaei, M., and Moslem, Y. (2024). Domain-Specific Translation with Open-Source Large Language Models: Resource-Oriented Analysis. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"455","DOI":"10.5565\/rev\/tradumatica.77","article-title":"Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics","volume":"1","author":"Lommel","year":"2014","journal-title":"Tradum\u00e0tica Tecnol. Traducci\u00f3"},{"key":"ref_27","unstructured":"Freitag, M., Rei, R., Mathur, N., Lo, C.k., Stewart, C., Foster, G., Lavie, A., and Bojar, O. (2021, January 10\u201311). Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-Based Human Evaluations on TED and News Domain. Proceedings of the Sixth Conference on Machine Translation, Online."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1460","DOI":"10.1162\/tacl_a_00437","article-title":"Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation","volume":"9","author":"Freitag","year":"2021","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kocmi, T., and Federmann, C. (2023). GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4. Proceedings of the Eighth Conference on Machine Translation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2023.wmt-1.64"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Fernandes, P., Deutsch, D., Finkelstein, M., Riley, P., Martins, A., Neubig, G., Garg, A., Clark, J., Freitag, M., and Firat, O. (2023). The Devil Is in the Errors: Leveraging Large Language Models for Fine-Grained Machine Translation Evaluation. Proceedings of the Eighth Conference on Machine Translation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2023.wmt-1.100"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lu, Q., Qiu, B., Ding, L., Zhang, K., Kocmi, T., and Tao, D. (2024). Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.findings-acl.520"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Liu, Y., Yang, T., Huang, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., and Zhang, Q. (2024). Calibrating LLM-Based Evaluator. arXiv.","DOI":"10.63317\/3posqwchnb6g"},{"key":"ref_33","unstructured":"Haddow, B., Kocmi, T., Koehn, P., and Monz, C. (2025). GEMBA V2: Ten Judgments Are Better Than One. Proceedings of the Tenth Conference on Machine Translation, Association for Computational Linguistics."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kim, A. (2025). RUBRIC-MQM: Span-Level LLM-as-Judge in Machine Translation for High-End Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.acl-industry.12"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Larionov, D., and Eger, S. (2025). BatchGEMBA: Token-Efficient Machine Translation Evaluation with Batched Prompting and Prompt Compression. arXiv.","DOI":"10.1109\/GACLM67198.2025.11232289"},{"key":"ref_36","unstructured":"ICH Expert Working Group (2009). ICH Harmonised Tripartite Guideline: Pharmaceutical Development Q8(R2), International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. Available online: https:\/\/database.ich.org\/sites\/default\/files\/Q8%28R2%29%20Guideline.pdf."},{"key":"ref_37","unstructured":"ICH Expert Working Group (2026, January 12). ICH Guideline Q9 on Quality Risk Management, Available online: https:\/\/www.ema.europa.eu\/en\/documents\/scientific-guideline\/international-conference-harmonisation-technical-requirements-registration-pharmaceuticals-human-use_en-3.pdf."},{"key":"ref_38","unstructured":"ICH Expert Working Group (2026, January 12). ICH Guideline Q9(R1) on Quality Risk Management; EMA\/CHMP\/ICH\/24235\/2006 Rev. 2, Available online: https:\/\/www.ema.europa.eu\/en\/ich-q9-quality-risk-management."},{"key":"ref_39","first-page":"20","article-title":"QRM, Knowledge Management, and the Importance of ICH Q9(R1)","volume":"48","author":"Kartoglu","year":"2024","journal-title":"Pharm. Technol."},{"key":"#cr-split#-ref_40.1","unstructured":"European Parliament and Council of the European Union (2024). Regulation"},{"key":"#cr-split#-ref_40.2","unstructured":"(EU) 2024\/1689 Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act), European Parliament and Council of the European Union. Technical Report."},{"key":"ref_41","unstructured":"(2018). Failure Modes and Effects Analysis (FMEA and FMECA) (Standard No. IEC 60812:2018)."},{"key":"ref_42","unstructured":"ICH Quality Implementation Working Group (2026, January 12). ICH Q8, Q9 and Q10 Questions and Answers (R5), Available online: https:\/\/www.ema.europa.eu\/en\/documents\/regulatory-procedural-guideline\/ich-guideline-q8-q9-q10-questions-answers-r5_en.pdf."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"44201","DOI":"10.1038\/s41598-025-27879-w","article-title":"Intelligent information management enables quality-by-design in pharmaceutical production","volume":"15","author":"Zhu","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_44","unstructured":"European Medicines Agency (2026, October 02). Linguistic Review\u2014Human. Available online: https:\/\/www.ema.europa.eu\/en\/human-regulatory-overview\/marketing-authorisation\/product-information-requirements\/linguistic-review-human."},{"key":"ref_45","unstructured":"European Commission (2026, October 02). A Guideline on Summary of Product Characteristics (SmPC), Available online: https:\/\/health.ec.europa.eu\/system\/files\/2016-11\/smpc_guideline_rev2_en_0.pdf."},{"key":"ref_46","unstructured":"Baratgin, J., Jacquet, B., Brochier, E., and Yama, H. (2027). Developing Effective Machine Translation System for Regulatory Documents: A Case Study on the Summary of Product Characteristics across European Languages. Human and Artificial Rationalities (HAR 2025), Springer Nature. Lecture Notes in Computer Science."},{"key":"ref_47","unstructured":"Loshchilov, I., and Hutter, F. (2019, January 6\u20139). Decoupled Weight Decay Regularization. Proceedings of the 7th International Conference on Learning Representations (ICLR), New Orleans, LA, USA."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7\u201312). Bleu: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Popovi\u0107, M. (2015). chrF: Character n-gram F-score for automatic MT evaluation. Proceedings of the Tenth Workshop on Statistical Machine Translation, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W15-3049"},{"key":"ref_50","unstructured":"Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006, January 8\u201312). A Study of Translation Edit Rate with Targeted Human Annotation. Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, Cambridge, MA, USA."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Rei, R., Stewart, C., Farinha, A.C., and Lavie, A. (2020). COMET: A Neural Framework for MT Evaluation. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2020.emnlp-main.213"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Post, M. (2018). A Call for Clarity in Reporting BLEU Scores. Proceedings of the Third Conference on Machine Translation: Research Papers, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-6319"},{"key":"ref_53","unstructured":"Yang, A., Yu, B., Li, C., Liu, D., Huang, F., Huang, H., Jiang, J., Tu, J., Zhang, J., and Zhou, J. (2025). Qwen2.5-1M Technical Report. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1080\/01621459.1952.10483441","article-title":"Use of Ranks in One-Criterion Variance Analysis","volume":"47","author":"Kruskal","year":"1952","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1080\/00401706.1964.10490181","article-title":"Multiple Comparisons Using Rank Sums","volume":"6","author":"Dunn","year":"1964","journal-title":"Technometrics"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/10\/1022\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T05:13:55Z","timestamp":1792905235000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/10\/1022"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,14]]},"references-count":56,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["info17101022"],"URL":"https:\/\/doi.org\/10.3390\/info17101022","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,14]]}}}