{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T18:06:26Z","timestamp":1790186786405,"version":"4.1.0"},"reference-count":37,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004000","name":"Guangzhou Municipal Science and Technology Program key projects","doi-asserted-by":"publisher","award":["Nos. 2023A03J0592 and 2025A03J4246"],"award-info":[{"award-number":["Nos. 2023A03J0592 and 2025A03J4246"]}],"id":[{"id":"10.13039\/501100004000","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["82572508"],"award-info":[{"award-number":["82572508"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>Pneumonia is a common infection in critically ill patients with poor outcomes, and its identification relies heavily on radiological evidence. Most MIMIC-based pneumonia studies rely on structured codes rather than radiological text, which may limit the precision of pneumonia-related case identification. This study evaluated large language models (LLMs) and retrieval-augmented generation (RAG) frameworks for automated detection of radiographic evidence of pneumonia from radiology reports. Radiological reports from the MIMIC-IV database were analyzed using pre-trained models in an inference-only setting without additional training, fine-tuning, or dataset splitting. Model performance was evaluated using accuracy, precision, sensitivity, specificity, and F1-score, with point estimates calculated from the full dataset and corresponding 95% confidence intervals estimated using 1,000 patient-level clustered bootstrap replicates. Independent evaluation was subsequently performed using a GZ dataset from a tertiary hospital in Guangzhou. RAG generally improved model performance, with gemma3-27b achieving the highest accuracy of 96.4% on the MIMIC-IV dataset and maintaining an accuracy of 87.6% on the GZ dataset. Model performance did not increase linearly with parameter size, suggesting that task-specific model suitability may be more important than parameter scale for this application. Although larger models required longer inference times, their increased computational costs did not consistently translate into proportional performance gains. Overall, LLMs demonstrated high accuracy in interpreting radiology reports for detecting radiographic evidence of pneumonia. The RAG architecture further enhanced robustness and generalizability through knowledge augmentation, providing a methodological basis for automated radiology report analysis and large-scale database curation.<\/jats:p>","DOI":"10.1371\/journal.pdig.0001104","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:28:38Z","timestamp":1790184518000},"page":"e0001104","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Large language models for pneumonia detection in radiology reports via text analysis"],"prefix":"10.1371","volume":"5","author":[{"given":"Weisheng","family":"Chen","sequence":"first","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Intensive Care Unit"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinya","family":"Li","sequence":"additional","affiliation":[{"name":"School of Nursing, Jinan University","place":["Guangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhongkai","family":"Qu","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Intensive Care Unit"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting","family":"Liu","sequence":"additional","affiliation":[{"name":"Shenzhen People\u2019s Hospital, The Second Clinical Medical College, Jinan University","place":["Shenzhen, China"],"department":["Department of Critical Care Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bo","family":"Chen","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Intensive Care Unit"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liying","family":"Huang","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Pharmacy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ningxia","family":"He","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Pharmacy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhigang","family":"Wang","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Intensive Care Unit"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2237-8771","authenticated-orcid":true,"given":"Jun","family":"Lyu","sequence":"additional","affiliation":[{"name":"The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"],"department":["Department of Pharmacy"]},{"name":"Clinical Research Center, The First Affiliated Hospital of Jinan University","place":["Guangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"issue":"1","key":"pdig.0001104.ref001","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1186\/s13054-024-05180-y","article-title":"D-PRISM: a global survey-based study to assess diagnostic and treatment approaches in pneumonia managed in intensive care","volume":"28","author":"LF Reyes","year":"2024","journal-title":"Crit Care"},{"key":"pdig.0001104.ref002","article-title":"Pneumonia","volume":"25","author":"A Torres","year":"2021","journal-title":"Nat Rev Dis Primers 7"},{"issue":"3","key":"pdig.0001104.ref003","first-page":"317","article-title":"Treatment strategies with combined agency against severe viral pneumonia in patients with advanced cancer","volume":"12","author":"F Zhang","year":"2024","journal-title":"J Transl Int Med"},{"issue":"8","key":"pdig.0001104.ref004","doi-asserted-by":"crossref","first-page":"710","DOI":"10.1016\/S2213-2600(19)30140-7","article-title":"Pathophysiological role of respiratory dysbiosis in hospital-acquired pneumonia","volume":"7","author":"A Roquilly","year":"2019","journal-title":"Lancet Respir Med"},{"issue":"4","key":"pdig.0001104.ref005","doi-asserted-by":"crossref","first-page":"825","DOI":"10.1016\/j.idc.2022.07.003","article-title":"Management of Unique Pneumonias Seen in the Intensive Care Unit","volume":"36","author":"BK Decker","year":"2022","journal-title":"Infect Dis Clin North Am"},{"issue":"5","key":"pdig.0001104.ref006","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1007\/s00134-025-07903-3","article-title":"How to approach a patient hospitalized for pneumonia who is not responding to treatment?","volume":"51","author":"P Povoa","year":"2025","journal-title":"Intensive Care Med"},{"issue":"3","key":"pdig.0001104.ref007","doi-asserted-by":"crossref","first-page":"1700582","DOI":"10.1183\/13993003.00582-2017","article-title":"International ERS\/ESICM\/ESCMID\/ALAT guidelines for the management of hospital-acquired pneumonia and ventilator-associated pneumonia: Guidelines for the management of hospital-acquired pneumonia (HAP)\/ventilator-associated pneumonia (VAP) of the European Respiratory Society (ERS), European Society of Intensive Care Medicine (ESICM), European Society of Clinical Microbiology and Infectious Diseases (ESCMID) and Asociaci\u00f3n Latinoamericana del T\u00f3rax (ALAT)","volume":"50","author":"A Torres","year":"2017","journal-title":"Eur Respir J"},{"issue":"1","key":"pdig.0001104.ref008","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1186\/s13054-020-03091-2","article-title":"Update of the treatment of nosocomial pneumonia in the ICU","volume":"24","author":"R Zaragoza","year":"2020","journal-title":"Crit Care"},{"issue":"1","key":"pdig.0001104.ref009","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1186\/s12890-024-03441-8","article-title":"Antibiotics versus Non-Antibiotic in the treatment of Aspiration Pneumonia: analysis of the MIMIC-IV database","volume":"24","author":"D Zhang","year":"2024","journal-title":"BMC Pulm Med"},{"issue":"1","key":"pdig.0001104.ref010","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1007\/s11739-024-03631-6","article-title":"Bidirectional association between pneumonia and intestinal infection: an analysis of the MIMIC-IV database","volume":"20","author":"W Hou","year":"2025","journal-title":"Intern Emerg Med"},{"key":"pdig.0001104.ref011","doi-asserted-by":"crossref","first-page":"1605032","DOI":"10.3389\/fnut.2025.1605032","article-title":"Association between prognostic nutritional index and all-cause mortality in critically ill patients with ventilator-associated pneumonia: a retrospective study based on MIMIC-IV database","volume":"12","author":"W Ji","year":"2025","journal-title":"Front Nutr"},{"issue":"1","key":"pdig.0001104.ref012","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1183\/09031936.01.00213501","article-title":"Imaging of pneumonia: trends and algorithms","volume":"18","author":"T Franquet","year":"2001","journal-title":"Eur Respir J"},{"issue":"1","key":"pdig.0001104.ref013","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.jaci.2025.02.004","article-title":"Evaluating large language model performance to support the diagnosis and management of patients with primary immune disorders","volume":"156","author":"NL Rider","year":"2025","journal-title":"J Allergy Clin Immunol"},{"issue":"4","key":"pdig.0001104.ref014","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review","volume":"333","author":"S Bedi","year":"2025","journal-title":"JAMA"},{"issue":"1","key":"pdig.0001104.ref015","first-page":"44","article-title":"Data mining in clinical big data: the frequently used databases, steps, and methodological models","volume":"8","author":"W-T Wu","year":"2021","journal-title":"Mil Med Res"},{"key":"pdig.0001104.ref016","author":"A Johnson","journal-title":"Mimic-iv. PhysioNet"},{"issue":"1","key":"pdig.0001104.ref017","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"AEW Johnson","year":"2023","journal-title":"Sci Data"},{"issue":"1","key":"pdig.0001104.ref018","first-page":"50","article-title":"How to use the Surveillance, Epidemiology, and End Results (SEER) data: research design and methodology","volume":"10","author":"W-Q Che","year":"2023","journal-title":"Mil Med Res"},{"issue":"3","key":"pdig.0001104.ref019","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1111\/jebm.12548","article-title":"Exploring patient medication adherence and data mining methods in clinical big data: A contemporary review","volume":"16","author":"Y Xu","year":"2023","journal-title":"J Evid Based Med"},{"key":"pdig.0001104.ref020","doi-asserted-by":"crossref","first-page":"101970","DOI":"10.1016\/j.eclinm.2023.101970","article-title":"Subphenotyping heterogeneous patients with chronic critical illness to guide individualised fluid balance treatment using machine learning: a retrospective cohort study","volume":"59","author":"P Liu","year":"2023","journal-title":"EClinicalMedicine"},{"key":"pdig.0001104.ref021","author":"SiliconFlow"},{"key":"pdig.0001104.ref022","author":"DeepSeek"},{"issue":"1","key":"pdig.0001104.ref023","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1186\/s13054-023-04466-x","article-title":"Impact of macrolide treatment on long-term mortality in patients admitted to the ICU due to CAP: a targeted maximum likelihood estimation and survival analysis","volume":"27","author":"LF Reyes","year":"2023","journal-title":"Crit Care"},{"key":"pdig.0001104.ref024","doi-asserted-by":"crossref","first-page":"110028","DOI":"10.1016\/j.compbiomed.2025.110028","article-title":"Machine learning prediction models for multidrug-resistant organism infections in ICU ventilator-associated pneumonia patients: Analysis using the MIMIC-IV database","volume":"190","author":"Z Cui","year":"2025","journal-title":"Comput Biol Med"},{"key":"pdig.0001104.ref025","doi-asserted-by":"crossref","first-page":"1402386","DOI":"10.3389\/fphar.2024.1402386","article-title":"Aspirin reduces the mortality risk of patients with community-acquired pneumonia: a retrospective propensity-matched analysis of the MIMIC-IV database","volume":"15","author":"G Wang","year":"2024","journal-title":"Front Pharmacol"},{"issue":"2","key":"pdig.0001104.ref026","doi-asserted-by":"crossref","DOI":"10.1148\/radiol.243774","article-title":"Comparison of a Specialized Large Language Model with GPT-4o for CT and MRI Radiology Report Summarization","volume":"316","author":"S Zheng","year":"2025","journal-title":"Radiology"},{"key":"pdig.0001104.ref027","article-title":"Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer","volume":"308","author":"MA Fink","year":"2023","journal-title":"Radiology"},{"key":"pdig.0001104.ref028","article-title":"Large Language Models and Artificial Neural Networks for Assessing 1-Year Mortality in Patients With Myocardial Infarction: Analysis From the Medical Information Mart for Intensive Care IV (MIMIC-IV) Database","volume":"27","author":"B Shi","year":"2025","journal-title":"J Med Internet Res"},{"issue":"1","key":"pdig.0001104.ref029","article-title":"A large language model framework for literature-based disease-gene association prediction","volume":"26","author":"P-H Li","year":"2024","journal-title":"Brief Bioinform"},{"issue":"1","key":"pdig.0001104.ref030","doi-asserted-by":"crossref","first-page":"7526","DOI":"10.1038\/s41467-025-62781-z","article-title":"Large language model powered knowledge graph construction for mental health exploration","volume":"16","author":"S Gao","year":"2025","journal-title":"Nat Commun"},{"key":"pdig.0001104.ref031","article-title":"Applications of Federated Large Language Model for Adverse Drug Reactions Prediction: Scoping Review","volume":"27","author":"D Guo","year":"2025","journal-title":"J Med Internet Res"},{"issue":"6","key":"pdig.0001104.ref032","doi-asserted-by":"crossref","first-page":"3539","DOI":"10.1007\/s00464-025-11723-3","article-title":"Large language model-generated clinical practice guideline for appendicitis","volume":"39","author":"A Boyle","year":"2025","journal-title":"Surg Endosc"},{"issue":"4","key":"pdig.0001104.ref033","doi-asserted-by":"crossref","first-page":"102056","DOI":"10.1016\/j.xcrm.2025.102056","article-title":"MetaGP: A generative foundation model integrating electronic health records and multimodal imaging for addressing unmet clinical needs","volume":"6","author":"F Liu","year":"2025","journal-title":"Cell Rep Med"},{"issue":"6","key":"pdig.0001104.ref034","doi-asserted-by":"crossref","first-page":"756","DOI":"10.3390\/diagnostics15060756","article-title":"An Ensemble Patient Graph Framework for Predictive Modelling from Electronic Health Records and Medical Notes","volume":"15","author":"S Daphne","year":"2025","journal-title":"Diagnostics (Basel)"},{"issue":"4","key":"pdig.0001104.ref035","doi-asserted-by":"crossref","first-page":"102896","DOI":"10.1016\/j.rpth.2025.102896","article-title":"Using a transformer language model to curate a pulmonary embolism dataset from the Medical Information Mart for Intensive Care IV: MIMIC-IV-Ext-PE","volume":"9","author":"BD Lam","year":"2025","journal-title":"Res Pract Thromb Haemost"},{"issue":"3","key":"pdig.0001104.ref036","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1093\/ehjdh\/ztae008","article-title":"Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease","volume":"5","author":"M Falter","year":"2024","journal-title":"Eur Heart J Digit Health"},{"issue":"4","key":"pdig.0001104.ref037","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1093\/jamia\/ocaf008","article-title":"Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines","volume":"32","author":"S Liu","year":"2025","journal-title":"J Am Med Inform Assoc"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001104","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:28:44Z","timestamp":1790184524000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001104"}},"subtitle":[],"editor":[{"given":"Goksu","family":"Bozdereli Berikol","sequence":"first","affiliation":[{"name":"Atilim University: Atilim Universitesi","place":["T\u00dcRKIYE"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":37,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001104","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]}}}