{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T17:03:56Z","timestamp":1791911036013,"version":"4.3.4"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2022R1F1A1068529"],"award-info":[{"award-number":["NRF-2022R1F1A1068529"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Institute of Information and Communications Technology Planning and Evaluation (IITP), Korean Government","award":["No.2019-0-01842"],"award-info":[{"award-number":["No.2019-0-01842"]}]},{"name":"NAVER Digital Bio Innovation Research Fund","award":["3720230020"],"award-info":[{"award-number":["3720230020"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Objective<\/jats:title>\n            <jats:p>This study aimed to develop an open-source multimodal large language model (CXR-LLaVA) for interpreting chest X-ray images (CXRs), leveraging recent advances in large language models (LLMs) to potentially replicate the image interpretation skills of human radiologists.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Materials and methods<\/jats:title>\n            <jats:p>For training, we collected 592,580 publicly available CXRs, of which 374,881 had labels for certain radiographic abnormalities (Dataset 1) and 217,699 provided free-text radiology reports (Dataset 2). After pre-training a vision transformer with Dataset 1, we integrated it with an LLM influenced by the LLaVA network. Then, the model was fine-tuned, primarily using Dataset 2. The model\u2019s diagnostic performance for major pathological findings was evaluated, along with the acceptability of radiologic reports by human radiologists, to gauge its potential for autonomous reporting.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>The model demonstrated impressive performance in test sets, achieving an average F1 score of 0.81 for six major pathological findings in the MIMIC internal test set and 0.56 for six major pathological findings in the external test set. The model\u2019s F1 scores surpassed those of GPT-4-vision and Gemini-Pro-Vision in both test sets. In human radiologist evaluations of the external test set, the model achieved a 72.7% success rate in autonomous reporting, slightly below the 84.0% rate of ground truth reports.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>This study highlights the significant potential of multimodal LLMs for CXR interpretation, while also acknowledging the performance limitations. Despite these challenges, we believe that making our model open-source will catalyze further research, expanding its effectiveness and applicability in various clinical contexts.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Key Points<\/jats:title>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Question<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>How can a multimodal large language model be adapted to interpret chest X-rays and generate radiologic reports?<\/jats:italic>\n            <\/jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Findings<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>The developed CXR-LLaVA model effectively detects major pathological findings in chest X-rays and generates radiologic reports with a higher accuracy compared to general-purpose models.<\/jats:italic>\n            <\/jats:p>\n            <jats:p>\n              <jats:bold>\n                <jats:italic>Clinical relevance<\/jats:italic>\n              <\/jats:bold>\n              <jats:italic>This study demonstrates the potential of multimodal large language models to support radiologists by autonomously generating chest X-ray reports, potentially reducing diagnostic workloads and improving radiologist efficiency.<\/jats:italic>\n            <\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s00330-024-11339-6","type":"journal-article","created":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T15:39:42Z","timestamp":1736955582000},"page":"4374-4386","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":75,"title":["CXR-LLaVA: a multimodal large language model for interpreting chest X-ray images"],"prefix":"10.1007","volume":"35","author":[{"given":"Seowoo","family":"Lee","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiwon","family":"Youn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyungjin","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mansu","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3700-0165","authenticated-orcid":false,"given":"Soon Ho","family":"Yoon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,15]]},"reference":[{"key":"11339_CR1","doi-asserted-by":"crossref","unstructured":"Shamshad F, Khan S, Zamir SW et al (2023) Transformers in medical imaging: a survey. Med Image Anal 88:102802","DOI":"10.1016\/j.media.2023.102802"},{"key":"11339_CR2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00811-0","volume":"6","author":"S-C Huang","year":"2023","unstructured":"Huang S-C, Pareek A, Jensen M, Lungren MP, Yeung S, Chaudhari AS (2023) Self-supervised learning for medical image classification: a systematic review and implementation guidelines. NPJ Digit Med 6:74","journal-title":"NPJ Digit Med"},{"key":"11339_CR3","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","volume":"19","author":"D Shen","year":"2017","unstructured":"Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221\u2013248","journal-title":"Annu Rev Biomed Eng"},{"key":"11339_CR4","unstructured":"Gozalo-Brizuela R, Garrido-Merchan EC (2023) ChatGPT is not all you need. A state of the art review of large generative AI models. Preprint at https:\/\/arxiv.org\/abs\/2301.04655"},{"key":"11339_CR5","unstructured":"Radford A, Kim JW, Hallacy C et al (2021) Learning transferable visual models from natural language supervision. PMLR 139:8748\u20138763"},{"key":"11339_CR6","unstructured":"Li J, Li D, Savarese S, Hoi S (2023) Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. International conference on machine learning. PMLR, pp 19730\u201319742"},{"key":"11339_CR7","unstructured":"Liu H, Li C, Wu Q, Lee YJ (2024). Visual instruction tuning. Advances in Neural Information Processing Systems, 36."},{"key":"11339_CR8","unstructured":"Xu S, Yang L, Kelly C et al (2023) ELIXR: towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders. Preprint at https:\/\/arxiv.org\/abs\/2308.01317"},{"key":"11339_CR9","doi-asserted-by":"crossref","unstructured":"Li C, Wong C, Zhang S et al (2024). Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36.","DOI":"10.32388\/VLXB6M"},{"key":"11339_CR10","first-page":"768","volume":"33","author":"V Markoti\u0107","year":"2021","unstructured":"Markoti\u0107 V, Poju\u017eina T, Radan\u010devi\u0107 D, Miljko M, Pokraj\u010di\u0107 V (2021) The radiologist workload increase; where is the limit?: Mini review and case study. Psychiatr Danub 33:768\u2013770","journal-title":"Psychiatr Danub"},{"key":"11339_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2023.110817","volume":"147","author":"S Sajed","year":"2023","unstructured":"Sajed S, Sanati A, Garcia JE, Rostami H, Keshavarz A, Teixeira A (2023) The effectiveness of deep learning vs. traditional methods for lung disease diagnosis using chest X-ray images: a systematic review. Appl Soft Comput 147:110817","journal-title":"Appl Soft Comput"},{"key":"11339_CR12","unstructured":"Lee S, Youn J, Kim M, Yoon SH (2023) CXR-LLAVA: multimodal large language model for interpreting chest x-ray images. Preprint at https:\/\/arxiv.org\/abs\/2310.18341"},{"key":"11339_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2021.102046","volume":"71","author":"A Signoroni","year":"2021","unstructured":"Signoroni A, Savardi M, Benini S et al (2021) BS-Net: learning COVID-19 pneumonia severity on a large chest X-ray dataset. Med Image Anal 71:102046","journal-title":"Med Image Anal"},{"key":"11339_CR14","doi-asserted-by":"crossref","unstructured":"Irvin J, Rajpurkar P, Ko M et al (2019) CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. Preprint at https:\/\/arxiv.org\/abs\/1901.07031","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"11339_CR15","doi-asserted-by":"crossref","unstructured":"Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2097\u20132106","DOI":"10.1109\/CVPR.2017.369"},{"key":"11339_CR16","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2020.101797","volume":"66","author":"A Bustos","year":"2020","unstructured":"Bustos A, Pertusa A, Salinas J-M, De La Iglesia-Vaya M (2020) Padchest: a large chest x-ray image dataset with multi-label annotated reports. Med Image Anal 66:101797","journal-title":"Med Image Anal"},{"key":"11339_CR17","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1007\/s10278-022-00706-8","volume":"36","author":"P Lakhani","year":"2023","unstructured":"Lakhani P, Mongan J, Singhal C et al (2023) The 2021 SIIM-FISABIO-RSNA machine learning COVID-19 challenge: annotation and standard exam classification of COVID-19 chest radiographs. J Digit Imaging 36:365\u2013372","journal-title":"J Digit Imaging"},{"key":"11339_CR18","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-022-01498-w","volume":"9","author":"HQ Nguyen","year":"2022","unstructured":"Nguyen HQ, Lam K, Le LT et al (2022) VinDr-CXR: an open dataset of chest X-rays with radiologist\u2019s annotations. Sci Data 9:429","journal-title":"Sci Data"},{"key":"11339_CR19","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0322-0","volume":"6","author":"AE Johnson","year":"2019","unstructured":"Johnson AE, Pollard TJ, Berkowitz SJ et al (2019) MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Sci Data 6:317","journal-title":"Sci Data"},{"key":"11339_CR20","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1093\/jamia\/ocv080","volume":"23","author":"D Demner-Fushman","year":"2016","unstructured":"Demner-Fushman D, Kohli MD, Rosenman MB et al (2016) Preparing a collection of radiology examinations for distribution and retrieval. J Am Med Inform Assoc 23:304\u2013310","journal-title":"J Am Med Inform Assoc"},{"key":"11339_CR21","unstructured":"Touvron H, Martin L, Stone K et al (2023) Llama 2: open foundation and fine-tuned chat models. Preprint at https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"11339_CR22","doi-asserted-by":"publisher","unstructured":"Yang Z, Yao Z, Tasmin M et al (2023) Performance of multimodal GPT-4V on USMLE with image: potential for imaging diagnostic support with explanations. Preprint at https:\/\/doi.org\/10.1101\/2023.10.26.23297629:2023.2010.2026.23297629","DOI":"10.1101\/2023.10.26.23297629:2023.2010.2026.23297629"},{"key":"11339_CR23","doi-asserted-by":"crossref","unstructured":"Brin D, Sorin V, Barash Y et al (2024) Assessing GPT-4 multimodal performance in radiological image analysis. European Radiology 1\u20137","DOI":"10.1101\/2023.11.15.23298583"},{"key":"11339_CR24","doi-asserted-by":"crossref","unstructured":"Tu T, Azizi S, Driess D et al (2024) Towards generalist biomedical AI. NEJM AI 1:AIoa2300138","DOI":"10.1056\/AIoa2300138"},{"key":"11339_CR25","doi-asserted-by":"publisher","first-page":"1399","DOI":"10.1038\/s41551-022-00936-9","volume":"6","author":"E Tiu","year":"2022","unstructured":"Tiu E, Talius E, Patel P, Langlotz CP, Ng AY, Rajpurkar P (2022) Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning. Nat Biomed Eng 6:1399\u20131406","journal-title":"Nat Biomed Eng"},{"key":"11339_CR26","unstructured":"Thawkar O, Shaker A, Mullappilly SS et al (2023) XrayGPT: chest radiographs summarization using medical vision-language models. Preprint at https:\/\/arxiv.org\/abs\/2306.07971"},{"key":"11339_CR27","unstructured":"Lee H, Kim W, Kim J-H et al (2024) Vision-Language Generative Model for View-Specific Chest X-ray Generation. Conference on Health, Inference, and Learning. PMLR, pp 280\u2013296"},{"key":"11339_CR28","unstructured":"Lee S, Kim WJ, Chang J, Ye JC (2023) LLM-CXR: instruction-finetuned LLM for CXR image understanding and generation. Preprint at https:\/\/arxiv.org\/abs\/2305.11490"},{"key":"11339_CR29","unstructured":"Chen Z, Varma M, Delbrouck J-B et al (2024) A Vision-Language Foundation Model to Enhance Efficiency of Chest X-ray Interpretation. Preprint at https:\/\/arxiv.org\/abs\/2401.12208"}],"container-title":["European Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-024-11339-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00330-024-11339-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00330-024-11339-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,13]],"date-time":"2025-06-13T15:42:19Z","timestamp":1749829339000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00330-024-11339-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,15]]},"references-count":29,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["11339"],"URL":"https:\/\/doi.org\/10.1007\/s00330-024-11339-6","relation":{},"ISSN":["1432-1084"],"issn-type":[{"value":"1432-1084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,15]]},"assertion":[{"value":"19 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 October 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 December 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The scientific guarantor of this publication is Seowoo Lee.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Guarantor"}},{"value":"The authors of this manuscript declare relationships with the following companies: Medical IP and RadiSen. Seowoo Lee, Hyungjin Kim, and Soon Ho Yoon hold stock options in Medical IP. Hyungjin Kim receives consulting fees from RadiSen and is a member of the Scientific Editorial Board for European Radiology (section: chest) and, as such, did not participate in the selection nor review processes for this article. The author\u2019s roles in these organizations are unrelated to the content of the submitted work. Other authors do not have conflicts of interest.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"No complex statistical methods were necessary for this paper.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Statistics and biometry"}},{"value":"Written informed consent was not required for this study due to the exclusive use of publicly available datasets.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed consent"}},{"value":"Institutional Review Board (IRB) approval was not required for this study because it utilized publicly available datasets. This research adheres to the terms of use associated with the datasets, ensuring compliance with ethical standards.","order":6,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"A preliminary version of this work has been made publicly available as a preprint. The preprint of this study, titled \u201cCXR-LLAVA: a multimodal large language model for interpreting chest X-ray images,\u201d has been uploaded to the arXiv repository. It can be accessed through the following .","order":7,"name":"Ethics","group":{"name":"EthicsHeading","label":"Study subjects or cohorts overlap"}},{"order":8,"name":"Ethics","group":{"name":"EthicsHeading","label":"Methodology"}}]}}