{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T20:51:56Z","timestamp":1792615916685,"version":"4.5.0"},"reference-count":32,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Cross-domain chest radiograph report generation may produce formally valid reports that are nevertheless repetitive, weakly case-specific, or missing clinically relevant findings. These failures can be missed when evaluation relies on a single language-overlap or clinical-label score.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate cross-domain report generation with a bidirectional layered audit, characterize direction-specific and end point\u2013specific failure profiles under a composite training-side control and 3 decoding policies, and explore whether selected audit dimensions remained informative for a contemporary medical vision language model.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n A fixed R2Gen-style system was evaluated bidirectionally on 575 IU X-ray and 401 MIMIC-CXR (Medical Information Mart for Intensive Care Chest X-Ray) target records. The audit included surface validity, complete-report recurrence and concentration, a predefined prefix end point, lexical diversity, official Stanford CheXpert\/NegBio labels, paired cluster-bootstrap inference, and blinded review by 2 medical trainees. The primary human audit included 210 reports from 70 randomly sampled cases, with blinded adjudication of unresolved primary end point disagreements. A 60-report targeted panel was used for failure-mode analysis. Separately, a fixed MedGemma 1.5 4B configuration generated reports for all 976 records.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n In MIMIC-CXR\u2013to\u2013IU X-ray transfer, out of 575 reports, ON (protocol-defined training-side control enabled)+B4 (beam search with a beam size of 4) reduced the predefined recurrent prefix from 575 (100%) to 8 (1.4%; Holm-adjusted\n P<\/jats:italic>\n =.002). However, 569 (99.0%) ON+B4 outputs still belonged to exact duplicate groups. In IU X-ray\u2013to\u2013MIMIC-CXR transfer, the 14-category CheXpert\/NegBio micro-\n F<\/jats:italic>\n 1<\/jats:sub>\n was higher for ON+B4 than for OFF (protocol-defined training-side control disabled)+B4 (0.1687 vs 0.0744; Holm-adjusted\n P<\/jats:italic>\n =.01). This apparent advantage was not broad-based: after cardiomegaly was excluded in a post hoc descriptive sensitivity analysis, the difference changed from +0.0944 to \u22120.0610. Text-level tracing showed that the original contrast was mainly driven by a labeler response to recurrent, semantically normal heart-size wording. Across the primary human-audit cells, major omissions occurred in 12 to 15 out of 35 reports, and major unsupported findings occurred in 10 to 16 of 35 reports, with no consistent configuration ordering observed. The exploratory MedGemma micro-\n F<\/jats:italic>\n 1<\/jats:sub>\n was 0.4722 on IU X-ray and 0.4340 on MIMIC-CXR, but 381 out of 575 (66.3%) IU outputs belonged to exact duplicate groups.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n No configuration performed consistently well across end points, and no single metric captured all forms of cross-domain degradation. The MedGemma stress test suggested that recurrence and label-traceability checks remain useful across architectures, although it was not a matched performance comparison. Cross-domain report generators should undergo bidirectional evaluation, final-text verification, and image-grounded human review before external validation or clinical deployment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/95888","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T19:25:06Z","timestamp":1789500306000},"page":"e95888-e95888","source":"Crossref","is-referenced-by-count":0,"title":["Cross-Domain Chest Radiograph Report Generation Under Dataset Shift: Bidirectional Audit Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-7771-0592","authenticated-orcid":false,"given":"Yishan","family":"Sun","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5176-7577","authenticated-orcid":false,"given":"Jun","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1540-1208","authenticated-orcid":false,"given":"Yu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,21]]},"reference":[{"key":"R1","doi-asserted-by":"crossref","unstructured":"Jing B Xie P Xing E . On the automatic generation of medical imaging reports. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics; 2018:2577-2586. [doi: 10.18653\/v1\/P18-1240]","DOI":"10.18653\/v1\/P18-1240"},{"key":"R2","unstructured":"Liu G Hsu TMH McDermott M et al. Clinically accurate chest X-ray report generation. Presented at: 4th Machine Learning for Healthcare Conference; Aug 8-10, 2019; Ann Arbor, Michigan, United States. URL: https:\/\/proceedings.mlr.press\/v106\/liu19a.html [Accessed 05-07-2026]"},{"key":"R3","unstructured":"Boag W Hsu TMH McDermott M Berner G Alesentzer E Szolovits P . Baselines for chest X-ray report generation. Presented at: Machine Learning for Health (ML4H) NeurIPS Workshop 2019; Dec 13, 2019; British Columbia, Canada. URL: https:\/\/proceedings.mlr.press\/v116\/boag20a.html [Accessed 05-07-2026]"},{"key":"R4","doi-asserted-by":"crossref","unstructured":"Chen Z Song Y Chang TH Wan X . Generating radiology reports via memory-driven transformer. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics; 2020:1439-1449. [doi: 10.18653\/v1\/2020.emnlp-main.112]","DOI":"10.18653\/v1\/2020.emnlp-main.112"},{"issue":"7","key":"R5","doi-asserted-by":"publisher","first-page":"12910","DOI":"10.1609\/aaai.v34i07.6989","article-title":"When radiology report generation meets knowledge graph","volume":"34","author":"Zhang","journal-title":"Proc AAAI Conf Artif Intell"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"590","DOI":"10.1609\/aaai.v33i01.3301590","article-title":"CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison","volume":"33","author":"Irvin","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"R7","volume":"2017","author":"Peng","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"R8","unstructured":"Endo M Krishnan R Krishna V Ng AY Rajpurkar P . Retrieval-based chest X-ray report generation using a pre-trained contrastive language-image model. Presented at: Proceedings of Machine Learning for Health; Dec 4, 2021; Online. URL: https:\/\/proceedings.mlr.press\/v158\/endo21a.html [Accessed 05-07-2026]"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Nooralahzadeh F Perez Gonzalez N Frauenfelder T Fujimoto K Krauthammer M . Progressive transformer-based generation of radiology reports. In: Findings of the Association for Computational Linguistics. Association for Computational Linguistics; 2021:2824-2832. [doi: 10.18653\/v1\/2021.findings-emnlp.241]","DOI":"10.18653\/v1\/2021.findings-emnlp.241"},{"key":"R10","unstructured":"Jain S Agrawal A Saporta A et al. RadGraph: extracting clinical entities and relations from radiology reports. Presented at: Thirty-Fifth Conference on Neural Information Processing Systems (NeurIPS 2021), Datasets and Benchmarks Track; Dec 6-14, 2021; Online. URL: https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/hash\/c8ffe9a587b126f152ed3d89a146b445-Abstract-round1.html [Accessed 05-07-2026]"},{"key":"R11","unstructured":"Ramesh V Chi NA Rajpurkar P . Improving radiology report generation systems by removing hallucinated references to non-existent priors. Presented at: 2nd Machine Learning for Health (ML4H) Symposium 2022; Nov 28, 2022; New Orleans, Louisiana, USA. URL: https:\/\/proceedings.mlr.press\/v193\/ramesh22a.html [Accessed 05-07-2026]"},{"key":"R12","unstructured":"Jeong J Tian K Li A et al. Multimodal image-text matching improves retrieval-based chest X-ray report generation. Presented at: 6th Conference on Medical Imaging with Deep Learning (MIDL 2023); Jul 10-12, 2023; Nashville, Tennessee, USA. URL: https:\/\/proceedings.mlr.press\/v227\/jeong24a.html [Accessed 29-09-2026]"},{"issue":"5","key":"R13","doi-asserted-by":"publisher","first-page":"888","DOI":"10.1007\/s10278-019-00180-9","article-title":"Generalizable inter-institutional classification of abnormal chest radiographs using efficient convolutional neural networks","volume":"32","author":"Pan","journal-title":"J Digit Imaging"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-49956-8","article-title":"Enhancing domain generalization in the AI-based analysis of chest radiographs with federated learning","volume":"13","author":"Tayebi Arasteh","journal-title":"Sci Rep"},{"issue":"9","key":"R15","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2023.100802","article-title":"Evaluating progress in automatic chest X-ray radiology report generation","volume":"4","author":"Yu","journal-title":"Patterns (N Y)"},{"key":"R16","unstructured":"Nguyen D Chen C He H Tan C . Pragmatic radiology report generation. Presented at: 3rd Machine Learning for Health (ML4H) Symposium; Dec 10, 2023; New Orleans, Louisiana, USA. URL: https:\/\/proceedings.mlr.press\/v225\/nguyen23a.html [Accessed 05-07-2026]"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1109\/ichi61247.2024.00058","article-title":"Leveraging professional radiologists\u2019 expertise to enhance LLMs\u2019 evaluation for AI-generated radiology reports","volume":"2024","author":"Zhu","journal-title":"Proc (IEEE Int Conf Healthc Inform)"},{"key":"R18","doi-asserted-by":"crossref","unstructured":"Delbrouck JB Xu J Moll J et al. Automated structured radiology report generation. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1). Association for Computational Linguistics; 2025:26813-26829. [doi: 10.18653\/v1\/2025.acl-long.1301]","DOI":"10.18653\/v1\/2025.acl-long.1301"},{"key":"R19","unstructured":"Moon JH Choi G Rabaey P et al. Lunguage: a benchmark for structured and sequential chest X-ray interpretation. PhysioNet. 2026. URL: https:\/\/physionet.org\/content\/lunguage\/1.0.0\/ [Accessed 29-09-2026]"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Xu J Zhang X Abderezaei J et al. RadEval: a framework for radiology text evaluation. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2025:546-557. [doi: 10.18653\/v1\/2025.emnlp-demos.40]","DOI":"10.18653\/v1\/2025.emnlp-demos.40"},{"issue":"2","key":"R21","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.242575","article-title":"Generative large language models trained for detecting errors in radiology reports","volume":"315","author":"Sun","journal-title":"Radiology"},{"issue":"40","key":"R22","doi-asserted-by":"publisher","first-page":"33953","DOI":"10.1609\/aaai.v40i40.40688","article-title":"A disease-aware dual-stage framework for chest X-ray report generation","volume":"40","author":"Wu","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Xiao J Liu H Dong R Liu J Yu H . Automated radiology report generation based on topic-keyword semantic guidance. Presented at: 2025 IEEE International Conference on Multimedia and Expo (ICME); Jun 30 to Jul 4, 2025; Nantes, France. [doi: 10.1109\/ICME59968.2025.11209777]","DOI":"10.1109\/ICME59968.2025.11209777"},{"key":"R24","doi-asserted-by":"crossref","unstructured":"Nimalsiri W Hennayake M Rathnayake K Ambegoda TD Meedeniya D . Automated radiology report generation using transformers. Presented at: 2023 3rd International Conference on Advanced Research in Computing (ICARC); Feb 23-24, 2023; Belihuloya, Sri Lanka. [doi: 10.1109\/ICARC57651.2023.10145699]","DOI":"10.1109\/ICARC57651.2023.10145699"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2025.3606961","article-title":"Chest X-ray report generation using abnormality guided vision language model","volume":"13","author":"Edirisinghe","journal-title":"IEEE Access"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1007\/s00779-022-01676-y","article-title":"MRI image analysis for cerebrum tumor detection and feature extraction using 2D U-ConvNet and SVM classification","volume":"27","author":"Pitchai","journal-title":"Pers Ubiquit Comput"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","volume":"6","author":"Johnson","journal-title":"Sci Data"},{"issue":"2","key":"R28","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1093\/jamia\/ocv080","article-title":"Preparing a collection of radiology examinations for distribution and retrieval","volume":"23","author":"Demner-Fushman","journal-title":"J Am Med Inform Assoc"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Li J Galley M Brockett C Gao J Dolan B . A diversity-promoting objective function for neural conversation models. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics; 2026:110-119. [doi: 10.18653\/v1\/N16-1014]","DOI":"10.18653\/v1\/N16-1014"},{"key":"R30","unstructured":"MedGemma 1.5 model card. Google for Developers. Apr 21, 2026. URL: https:\/\/developers.google.com\/health-ai-developer-foundations\/medgemma\/model-card [Accessed 05-07-2026]"},{"key":"R31","unstructured":"Johnson A Pollard T Mark R Berkowitz S Horng S . MIMIC-CXR database. PhysioNet. 2024. URL: https:\/\/physionet.org\/content\/mimic-cxr\/2.1.0\/ [Accessed 29-09-2026]"},{"key":"R32","unstructured":"National Health Commission of the People\u2019s Republic of ChinaMinistry of Education of the People\u2019s Republic of ChinaMinistry of Science and Technology of the People\u2019s Republic of ChinaNational Administration of Traditional Chinese Medicine. Measures for the ethical review of life sciences and medical research involving humans. Data Compliance China. Feb 18, 2023. URL: https:\/\/datacompliancechina.com\/laws\/life-sciences-medical-research-ethics-review-measures\/ [Accessed 05-07-2026]"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T20:14:13Z","timestamp":1792613653000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e95888"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,21]]},"references-count":32,"URL":"https:\/\/doi.org\/10.2196\/95888","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,21]]},"article-number":"v14i7e95888"}}