{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T22:16:27Z","timestamp":1789164987100,"version":"build-2803163510"},"reference-count":48,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Federated learning (FL) enables multi-institutional model training on clinical text without sharing raw data; however, gradient inversion methods can reconstruct sensitive information from shared model updates. The extent of such privacy leakage in FL applied to radiology reports, and the role of tokenizer design, remains unclear.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to quantify gradient-based reconstruction of radiology report text in an FL setting and to compare privacy risk across 3 transformer tokenization strategies in a controlled, tokenizer-aware evaluation.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Six FL clients trained a GPT-2\u2013style transformer (sequence length 32) on 2 public clinical-text corpora comprising 368,751 diagnostic reports, 98,206 discharge summaries, and 1500 MIMIC-CXR (Medical Information Mart for Intensive Care Chest X-Ray) radiology reports. Models were trained using 3 tokenizers (GPT-2, RadBERT, and LLaMA-2) with batch sizes of 64, 128, and 256. An active malicious-server threat model was assumed, and analytic gradient inversion was applied to recover text. Reconstruction fidelity was measured over 5 runs using exact sentence accuracy, sentence-level bilingual evaluation understudy (S-BLEU), and recall-oriented understudy for gisting evaluation (ROUGE-L).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Exact sentence reconstruction ranged from 27% to 75% across tokenizers, datasets, and batch sizes. At batch size 64 on the discharge dataset, accuracy was 64.7% (GPT-2), 70% (RadBERT), and 67.5% (LLaMA-2), decreasing to 27.3%, 28.5%, and 27.5% at batch size 256. S-BLEU declined with increasing batch size (eg, discharge reports from 0.69 to 0.31). Reconstruction fidelity did not differ significantly across tokenizers (all but one of 27 comparisons nonsignificant; none significant after Holm correction), and approximately 75% of clinical concepts were represented in the reconstructed text (a corpus-level upper bound), regardless of tokenizer. Batch size was the dominant factor governing leakage.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Under a worst-case malicious server that tampers with the shared model and observes unprotected per-client gradients (no secure aggregation or differential privacy), substantial portions of radiology-report text can be reconstructed, with up to approximately 75% of reconstructed 32-token sequences and 75% of clinical concepts (not direct patient identifiers) recovered from FL gradients. In a controlled ablation holding model architecture fixed, tokenizer choice, including domain-specific tokenizers, did not significantly affect leakage under the evaluated conditions, whereas batch size was the primary determinant, and no tokenizer significantly reduced the risk. Tokenizer selection should therefore not be treated as a privacy safeguard in this setting. Safeguards such as secure aggregation and differential privacy should therefore be evaluated as candidate protections for FL deployments that must satisfy Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR) requirements in radiology natural language processing (NLP); legal compliance additionally depends on organizational safeguards, risk assessment, and governance beyond the scope of this study.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/88390","type":"journal-article","created":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T14:40:05Z","timestamp":1784817605000},"page":"e88390-e88390","source":"Crossref","is-referenced-by-count":0,"title":["Privacy Leakage in Federated Learning in Radiology Reports: Comparative Evaluation of Tokenizer and Batch-Size Privacy Risks"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-9401-887X","authenticated-orcid":false,"given":"Santhosh","family":"Parampottupadam","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7293-8220","authenticated-orcid":false,"given":"Andr\u00e9s","family":"Mart\u00ednez Mora","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3361-1698","authenticated-orcid":false,"given":"Dimitrios","family":"Bounias","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9096-8768","authenticated-orcid":false,"given":"Sinem","family":"Sav","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6626-2463","authenticated-orcid":false,"given":"Klaus","family":"Maier-Hein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3218-3377","authenticated-orcid":false,"given":"Ralf","family":"Floca","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,11]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1002\/jmrs.424","article-title":"The effect of clinical information on radiology reporting: a systematic review","volume":"68","author":"Castillo","journal-title":"J Med Radiat Sci"},{"issue":"8","key":"R2","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"key":"R3","unstructured":"Radford A Narasimhan K . Improving language understanding by generative pre-training. Semantic Scholar. 2018. URL: https:\/\/api.semanticscholar.org\/CorpusID:49313245 [Accessed 24-08-2026]"},{"key":"R4","doi-asserted-by":"crossref","unstructured":"Veen DV Uden CV Blankemeier L et al. Clinical text summarization: adapting large language models can outperform human experts. Res Sq. Preprint posted online on Oct 30, 2023. [doi: 10.21203\/rs.3.rs-3483777\/v1]","DOI":"10.21203\/rs.3.rs-3483777\/v1"},{"issue":"6","key":"R5","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1016\/j.diii.2023.02.003","article-title":"Revolutionizing radiology with GPT-based models: current applications, future possibilities and limitations of ChatGPT","volume":"104","author":"Lecler","journal-title":"Diagn Interv Imaging"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.2196\/65547","article-title":"Large language models in summarizing radiology report impressions for lung cancer in Chinese: evaluation study","volume":"27","author":"Hu","journal-title":"J Med Internet Res"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.2196\/55799","article-title":"Evaluating large language models for automated reporting and data systems categorization: cross-sectional study","volume":"12","author":"Wu","journal-title":"JMIR Med Inform"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-025-02954-4","article-title":"A systematic review of large language model (LLM) evaluations in clinical medicine","volume":"25","author":"Shool","journal-title":"BMC Med Inform Decis Mak"},{"key":"R9","unstructured":"The HIPAA privacy rule. US Department of Health and Human Services. URL: https:\/\/www.hhs.gov\/hipaa\/for-professionals\/privacy\/index.html [Accessed 24-08-2027]"},{"key":"R10","unstructured":"General Data Protection Regulation (GDPR). GDPR.au. URL: https:\/\/gdpr.eu\/tag\/gdpr\/ [Accessed 24-08-2026]"},{"key":"R11","unstructured":"McMahan B Moore E Ramage D Hampson S Arcas BA . Communication-efficient learning of deep networks from decentralized data. Presented at: 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017; Apr 20-22, 2017:1273-1282; Fort Lauderdale, FL, USA. [doi: 10.48550\/arXiv.1602.05629]"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-72086-4_17","article-title":"Continually tuning a large language model for multi-domain radiology report generation","author":"Sun","journal-title":"Int Conf Med Image Comput Comput-Assist Interv"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232756","article-title":"Chatbots and large language models in radiology: a practical primer for clinical and research applications","volume":"310","author":"Bhayana","journal-title":"Radiology"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","first-page":"80","DOI":"10.4274\/dir.2023.232417","article-title":"Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions","volume":"30","author":"Akinci D\u2019Antonoli","journal-title":"Diagn Interv Radiol"},{"issue":"21","key":"R15","doi-asserted-by":"publisher","first-page":"2529","DOI":"10.1097\/CM9.0000000000003302","article-title":"Leveraging foundation and large language models in medical artificial intelligence","volume":"137","author":"Wong","journal-title":"Chin Med J (Engl)"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Parampottupadam S Floca R Bounias D et al. Client security alone fails in federated learning: 2D and 3D attack insights. In: Medical Information Computing MImA EMERGE 2024 Communications in Computer and Information Science. Springer; 2024:235-244. URL: https:\/\/doi.org\/10.1007\/978-3-031-79103-1_24 [Accessed 12-01-2026]","DOI":"10.1007\/978-3-031-79103-1_24"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.2196\/41588","article-title":"Federated machine learning, privacy-enhancing technologies, and data protection laws in medical research: scoping review","volume":"25","author":"Brauneck","journal-title":"J Med Internet Res"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107861","article-title":"Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI","volume":"169","author":"Zhou","journal-title":"Comput Biol Med"},{"issue":"6","key":"R19","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1038\/s42256-021-00337-8","article-title":"End-to-end privacy preserving deep learning on multi-institutional medical imaging","volume":"3","author":"Kaissis","journal-title":"Nat Mach Intell"},{"issue":"7","key":"R20","doi-asserted-by":"publisher","first-page":"2044","DOI":"10.1109\/TMI.2023.3239391","article-title":"Do gradient inversion attacks make federated learning unsafe?","volume":"42","author":"Hatamizadeh","journal-title":"IEEE Trans Med Imaging"},{"issue":"4","key":"R21","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.240261","article-title":"A new era of text mining in radiology with privacy-preserving LLMs","volume":"6","author":"Akinci D\u2019Antonoli","journal-title":"Radiol Artif Intell"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3458754","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","journal-title":"ACM Trans Comput Healthcare"},{"key":"R23","unstructured":"Zhang X Tian C Yang X Chen L Li Z Petzold LR . AlpaCare: instruction-tuned large language models for medical application. arXiv. Preprint posted online on Oct 23, 2023. URL: https:\/\/arxiv.org\/abs\/2310.14558 [Accessed 24-08-2026]"},{"issue":"4","key":"R24","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.210258","article-title":"RadBERT: adapting transformer-based language models to radiology","volume":"4","author":"Yan","journal-title":"Radiol Artif Intell"},{"key":"R25","unstructured":"Solaiman I Brundage M Clark J et al. Release strategies and the social impacts of language models. arXiv. Preprint posted online on Aug 24, 2019. URL: https:\/\/arxiv.org\/abs\/1908.09203 [Accessed 24-08-2026]"},{"key":"R26","unstructured":"Touvron H Martin L Stone K et al. Llama 2: open foundation and fine-tuned chat models. arXiv. Preprint posted online on Jul 18, 2023. URL: https:\/\/arxiv.org\/abs\/2307.09288 [Accessed 24-08-2026]"},{"key":"R27","unstructured":"Dischargesum dataset. Hugging Face. 2024. URL: https:\/\/huggingface.co\/dischargesum [Accessed 24-08-2026]"},{"key":"R28","doi-asserted-by":"crossref","unstructured":"Johnson AEW Pollard TJ Greenbaum NR et al. MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs. Preprint posted online on Nov 14, 2019. URL: https:\/\/arxiv.org\/abs\/1901.07042 [Accessed 24-08-2026]","DOI":"10.1038\/s41597-019-0322-0"},{"issue":"1-2","key":"R29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000083","article-title":"Advances and open problems in federated learning","volume":"14","author":"Kairouz","journal-title":"Found Trends Mach Learn"},{"key":"R30","unstructured":"Fowl L Geiping J Czaja W Goldblum M Goldstein T . Robbing the fed: directly obtaining private data in federated learning with modified models. arXiv. Preprint posted online on Oct 25, 2022. URL: https:\/\/arxiv.org\/abs\/2110.13057 [Accessed 30-08-2026]"},{"key":"R31","unstructured":"Geiping J Bauermeister H Dr\u00f6ge H Moeller M . Inverting gradients \u2014 how easy is it to break privacy in federated learning? Presented at: 34th International Conference on Neural Information Processing Systems (NeurIPS 2020); Dec 6-12, 2020; Vancouver, Canada. URL: https:\/\/arxiv.org\/abs\/2003.14053 [Accessed 30-08-2026]"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Deng J Wang Y Li J et al. TAG: gradient attack on transformer-based language models. arXiv. Preprint posted online on Mar 11, 2021. [doi: 10.48550\/arXiv.2103.06819]","DOI":"10.18653\/v1\/2021.findings-emnlp.305"},{"key":"R33","unstructured":"Sellergren A Kazemzadeh S Jaroensri T et al. MedGemma technical report. arXiv. Preprint posted online on Apr 6, 2026. [doi: 10.48550\/arXiv.2507.05201]"},{"key":"R34","doi-asserted-by":"crossref","unstructured":"Papineni K Roukos S Ward T Zhu WJ . BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2002:311-318. [doi: 10.3115\/1073083.1073135]","DOI":"10.3115\/1073083.1073135"},{"key":"R35","doi-asserted-by":"crossref","unstructured":"Delbrouck JB Varma M Chambon P Langlotz C . Overview of the RadsSum23 shared task on multi-modal and multi-anatomical radiology report summarization 22nd workshop biomed nat lang process bionlp shar tasks toronto. In: Proceedings of the 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks. Association for Computational Linguistics; 2023:478-482. [doi: 10.18653\/v1\/2023.bionlp-1.45]","DOI":"10.18653\/v1\/2023.bionlp-1.45"},{"key":"R36","unstructured":"Lin CY . ROUGE: a package for automatic evaluation of summaries. Presented at: Text Summarization Branches Out; 74-81; Barcelona, Spain. URL: https:\/\/aclanthology.org\/W04-1013\/ [Accessed 20-02-2026]"},{"key":"R37","doi-asserted-by":"crossref","unstructured":"Boenisch F Dziedzic A Schuster R Shamsabadi AS Shumailov I Papernot N . When the curious abandon honesty: federated learning is not private. Presented at: 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P); Jul 3-7, 2023; Delft, Netherlands. [doi: 10.1109\/EuroSP57164.2023.00020]","DOI":"10.1109\/EuroSP57164.2023.00020"},{"key":"R38","unstructured":"Zhu L Liu Z Han S . Deep leakage from gradients. arXiv. Preprint posted online on Jun 21, 2019. [doi: 10.48550\/arXiv.1906.08935]"},{"key":"R39","unstructured":"Zhao B Mopuri KR Bilen H . IDLG: improved deep leakage from gradients. arXiv. Preprint posted online on Jan 8, 2020. [doi: 10.48550\/arXiv.2001.02610]"},{"key":"R40","unstructured":"Dwork C . Differential privacy. In: Bugliesi M Preneel B Sassone V Wegener I , editors. Autom Lang Program. Springer; 2006:1-12. URL: https:\/\/link.springer.com\/chapter\/10.1007\/11787006_1 [Accessed 12-11-2025]"},{"key":"R41","doi-asserted-by":"crossref","unstructured":"Bonawitz K Ivanov V Kreuter B et al. Practical secure aggregation for privacy-preserving machine learning. In: CCS \u201917: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. Association for Computing Machinery; 2017:1175-1191. [doi: 10.1145\/3133956.3133982]","DOI":"10.1145\/3133956.3133982"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01359-3","article-title":"A scoping review of privacy and utility metrics in medical synthetic data","volume":"8","author":"Kaabachi","journal-title":"NPJ Digit Med"},{"issue":"6","key":"R43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3595185","article-title":"Trading off privacy, utility, and efficiency in federated learning","volume":"14","author":"Zhang","journal-title":"ACM Trans Intell Syst Technol"},{"key":"R44","unstructured":"Akkus A Aghdam MP Li M et al. Generated data with fake privacy: hidden dangers of fine-tuning large language models on generated data. arXiv. Preprint posted online on Jan 29, 2025. [doi: 10.48550\/arXiv.2409.11423]"},{"key":"R45","unstructured":"Stadler T Oprisanu B Troncoso C . Synthetic data \u2014 anonymisation groundhog day. arXiv. Preprint posted online on Jan 24, 2022. [doi: 10.48550\/arXiv.2011.07018]"},{"key":"R46","unstructured":"Jayaraman B Evans D . Evaluating differentially private machine learning in practice. arXiv. Preprint posted online on Aug 12, 2019. [doi: 10.48550\/arXiv.1902.08874]"},{"key":"R47","doi-asserted-by":"crossref","unstructured":"Hosseini SM Sikaroudi M Babaei M Tizhoosh HR . Cluster based secure multi-party computation in federated learning for histopathology images. arXiv. Preprint posted online on Aug 21, 2022. [doi: 10.1007\/978-3-031-18523-6_11]","DOI":"10.1007\/978-3-031-18523-6_11"},{"key":"R48","unstructured":"Privacy leakage in federated learning in radiology reports. GitHub. URL: https:\/\/github.com\/santhoshcameo\/Privacy-Leakage-in-Federated-Learning-in-Radiology-Reports [Accessed 24-08-2026]"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T21:43:11Z","timestamp":1789162991000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e88390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,11]]},"references-count":48,"URL":"https:\/\/doi.org\/10.2196\/88390","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,11]]},"article-number":"v14i1e88390"}}