{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T16:25:01Z","timestamp":1787761501274,"version":"build-2784847793"},"reference-count":41,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Although large language models (LLMs) have demonstrated the ability to generate the impression section from radiology findings automatically, the incremental diagnostic value of clinical information for these models remains unclear.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the incremental diagnostic value of clinical information for LLMs and compare their performance with that of radiologists.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n This retrospective study included radiology reports from patients with histopathologically confirmed liver, lung, and breast diseases from 2 institutions between October 2021 and February 2025. We defined three progressive information input scenarios: (1) basic patient information and imaging findings, (2) scenario A plus chief complaint or clinical history, and (3) scenario B plus key laboratory results. Scenario-based data were input into 3 general-purpose LLMs (DeepSeek-R1, Gemini 2.5 Pro, and GPT-4o), generating 2709 entries. Diagnostic accuracy was assessed for both benign-malignant differentiation and disease diagnosis, with histopathology serving as the reference standard. Accuracy was compared among scenarios and against radiologist performance using the McNemar test, and\n P<\/jats:italic>\n values were adjusted using the Holm-Bonferroni correction for multiple comparisons.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n A total of 301 patients with pathologically confirmed diseases were included (mean age 53.5, SD 12.0 years; women: n=208, 69.1%). In the liver cohort, a numerical trend toward higher accuracy was observed in scenario C compared with scenario A across all 3 models (scenario C range: 72.3%\u201076.2% vs scenario A range: 64.4%\u201068.3%); these differences did not reach statistical significance after Holm-Bonferroni correction (all adjusted\n P<\/jats:italic>\n >.99). Notably, the DeepSeek-R1 model in scenario C achieved the highest diagnostic accuracy (77\/101, 76.2%), with no evidence of a difference compared with radiologists (82\/101, 81.2%; adjusted\n P<\/jats:italic>\n >.99). In contrast, results in the lung and breast cohorts were more heterogeneous. In the lung cohort, GPT-4o achieved its highest accuracy in scenario A for disease diagnosis (68\/92, 73.9%), which exceeded its performance in scenario B (64\/92, 69.6%) and scenario C (66\/92, 71.7%), suggesting that additional clinical information did not confer a consistent benefit. Gemini 2.5 Pro in scenario B achieved the highest accuracy in this cohort (72\/92, 78.3%); however, no statistically significant difference was found compared with radiologists (80\/92, 87.0%; adjusted\n P<\/jats:italic>\n =.25). In the breast cohort, DeepSeek-R1 achieved the numerically highest diagnostic accuracy in scenario A, and it decreased numerically with the addition of laboratory tests, although no significant difference was found between scenarios A and C (73\/108, 67.6% vs 71\/108, 65.7%; adjusted\n P<\/jats:italic>\n >.99).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n While the addition of clinical information was associated with a numeric trend toward higher diagnostic accuracy overall, this trend was heterogeneous across models and disease types, and no statistically significant improvement was demonstrated after adjustment for multiple comparisons.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/94904","type":"journal-article","created":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T15:59:05Z","timestamp":1787759945000},"page":"e94904-e94904","source":"Crossref","is-referenced-by-count":0,"title":["Incremental Diagnostic Value of Clinical Information for Large Language Models Across Multiple Organs: Retrospective Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5329-4299","authenticated-orcid":false,"given":"Jinqi","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7917-9028","authenticated-orcid":false,"given":"Xiaoyi","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6606-9535","authenticated-orcid":false,"given":"Yanfeng","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0533-4650","authenticated-orcid":false,"given":"Xiaolin","family":"Cui","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9012-3644","authenticated-orcid":false,"given":"Liyun","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4511-4736","authenticated-orcid":false,"given":"Xinming","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6421-4126","authenticated-orcid":false,"given":"Hongmei","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4605-9783","authenticated-orcid":false,"given":"Zheng","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,26]]},"reference":[{"issue":"5","key":"R1","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230582","article-title":"Performance of ChatGPT on a radiology board-style examination: insights into current strengths and limitations","volume":"307","author":"Bhayana","journal-title":"Radiology"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1097\/JS9.0000000000003406","article-title":"From algorithms to operating room: can large language models master China\u2019s attending anesthesiology exam? A cross-sectional evaluation","volume":"112","author":"He","journal-title":"Int J Surg"},{"issue":"6","key":"R3","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.17641","article-title":"Performance of large language models on medical oncology examination questions","volume":"7","author":"Longwell","journal-title":"JAMA Netw Open"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232561","article-title":"Accuracy of ChatGPT, Google Bard, and Microsoft Bing for simplifying radiology reports","volume":"309","author":"Amin","journal-title":"Radiology"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"101035","DOI":"10.1016\/j.jocmr.2024.101035","article-title":"Generative Pre-trained Transformer 4 makes cardiovascular magnetic resonance reports easy to understand","volume":"26","author":"Salam","journal-title":"J Cardiovasc Magn Reson"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1186\/s42492-023-00136-5","article-title":"Translating radiology reports into plain language using ChatGPT and GPT-4 with prompt learning: results, limitations, and potential","volume":"6","author":"Lyu","journal-title":"Vis Comput Ind Biomed Art"},{"issue":"5","key":"R7","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230922","article-title":"How AI responds to common lung cancer questions: ChatGPT vs Google Bard","volume":"307","author":"Rahsepar","journal-title":"Radiology"},{"issue":"4","key":"R8","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230424","article-title":"Appropriateness of breast cancer prevention and screening recommendations provided by ChatGPT","volume":"307","author":"Haver","journal-title":"Radiology"},{"issue":"5","key":"R9","doi-asserted-by":"publisher","first-page":"701","DOI":"10.2214\/AJR.23.29622","article-title":"Use of ChatGPT, GPT-4, and Bard to improve readability of ChatGPT\u2019s answers to common questions about lung cancer and lung cancer screening","volume":"221","author":"Haver","journal-title":"AJR Am J Roentgenol"},{"issue":"4","key":"R10","doi-asserted-by":"publisher","DOI":"10.1148\/ryct.230115","article-title":"Evaluating ChatGPT\u2019s accuracy in lung cancer prevention and screening recommendations","volume":"5","author":"Haver","journal-title":"Radiol Cardiothorac Imaging"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.230364","article-title":"Performance of an open-source large language model in extracting information from free-text radiology reports","volume":"6","author":"Le Guellec","journal-title":"Radiol Artif Intell"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232741","article-title":"Data extraction from free-text reports on mechanical thrombectomy in acute ischemic stroke using ChatGPT: a retrospective analysis","volume":"311","author":"Lehnen","journal-title":"Radiology"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231362","article-title":"Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer","volume":"308","author":"Fink","journal-title":"Radiology"},{"issue":"3","key":"R14","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.233117","article-title":"Large language models for automated synoptic reports and resectability categorization in pancreatic cancer","volume":"311","author":"Bhayana","journal-title":"Radiology"},{"issue":"4","key":"R15","doi-asserted-by":"publisher","DOI":"10.2214\/AJR.24.32341","article-title":"Use of ChatGPT large language models to extract details of recommendations for additional imaging from free-text impressions of radiology reports","volume":"224","author":"Li","journal-title":"AJR Am J Roentgenol"},{"issue":"2","key":"R16","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.233270","article-title":"Evaluating GPT-V4 (GPT-4 with vision) on detection of radiologic findings on chest radiographs","volume":"311","author":"Zhou","journal-title":"Radiology"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232714","article-title":"Potential of GPT-4 for detecting errors in radiology reports: implications for reporting accuracy","volume":"311","author":"Gertz","journal-title":"Radiology"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240701","article-title":"Large-scale validation of the feasibility of GPT-4 as a proofreading tool for head CT reports","volume":"314","author":"Kim","journal-title":"Radiology"},{"issue":"5","key":"R19","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231259","article-title":"Evaluating GPT4 on impressions generation in radiology reports","volume":"307","author":"Sun","journal-title":"Radiology"},{"issue":"3","key":"R20","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240885","article-title":"Constructing a large language model to generate impressions from findings in radiology reports","volume":"312","author":"Zhang","journal-title":"Radiology"},{"issue":"2","key":"R21","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.243774","article-title":"Comparison of a specialized large language model with GPT-4o for CT and MRI radiology report summarization","volume":"316","author":"Zheng","journal-title":"Radiology"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231167","article-title":"Feasibility of differential diagnosis based on imaging patterns using a large language model","volume":"308","author":"Kottlors","journal-title":"Radiology"},{"issue":"1","key":"R23","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240609","article-title":"Evaluation of GPT large language model performance on RSNA 2023 case of the day questions","volume":"313","author":"Mukherjee","journal-title":"Radiology"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231040","article-title":"ChatGPT\u2019s diagnostic performance from patient history and imaging findings on the diagnosis please quizzes","volume":"308","author":"Ueda","journal-title":"Radiology"},{"issue":"4","key":"R25","doi-asserted-by":"publisher","first-page":"586","DOI":"10.1007\/s11604-024-01712-2","article-title":"Structured clinical reasoning prompt enhances LLM\u2019s diagnostic capabilities in diagnosis please quiz cases","volume":"43","author":"Sonoda","journal-title":"Jpn J Radiol"},{"issue":"9","key":"R26","doi-asserted-by":"publisher","first-page":"1927","DOI":"10.1007\/s00247-025-06317-z","article-title":"Accuracy of large language models in generating differential diagnosis from clinical presentation and imaging findings in pediatric cases","volume":"55","author":"Jung","journal-title":"Pediatr Radiol"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","first-page":"41571872","DOI":"10.1038\/s41746-026-02380-4","article-title":"Evaluation of large language models for diagnostic impression generation from brain MRI report findings: a multicenter benchmark and reader study","volume":"9","author":"Wang","journal-title":"NPJ Digit Med"},{"issue":"8081","key":"R28","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1038\/s41586-025-09422-z","article-title":"DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning","volume":"645","author":"Guo","journal-title":"Nature"},{"issue":"8","key":"R29","doi-asserted-by":"publisher","first-page":"2496","DOI":"10.1038\/s41591-025-03850-0","article-title":"Open-source LLM DeepSeek on a par with proprietary models in clinical decision making","volume":"31","journal-title":"Nat Med"},{"issue":"10","key":"R30","doi-asserted-by":"publisher","first-page":"7401","DOI":"10.1097\/JS9.0000000000002784","article-title":"The advantage of DeepSeek in local deployment technology bridges the gap between research and practice","volume":"111","author":"JianBo","journal-title":"Int J Surg"},{"issue":"10","key":"R31","doi-asserted-by":"publisher","first-page":"7156","DOI":"10.21037\/qims-23-396","article-title":"Differentiating mass-forming intrahepatic cholangiocarcinoma from atypical hepatocellular carcinoma using Gd-EOB-DTPA-enhanced magnetic resonance imaging combined with serum markers in at-risk patients with hepatitis B virus","volume":"13","author":"Han","journal-title":"Quant Imaging Med Surg"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01488-3","article-title":"Benchmarking the diagnostic performance of open source LLMs in 1933 Eurorad case reports","volume":"8","author":"Kim","journal-title":"NPJ Digit Med"},{"issue":"9","key":"R33","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1016\/j.jacr.2023.06.025","article-title":"\u201cShortcuts\u201d causing bias in radiology artificial intelligence: causes, evaluation, and mitigation","volume":"20","author":"Banerjee","journal-title":"J Am Coll Radiol"},{"issue":"10","key":"R34","doi-asserted-by":"publisher","first-page":"7888","DOI":"10.1007\/s00330-021-07833-w","article-title":"Pneumothorax detection in chest radiographs: optimizing artificial intelligence system for accuracy and confounding bias reduction using in-image annotations in algorithm training","volume":"31","author":"Rueckel","journal-title":"Eur Radiol"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1038\/s41571-023-00844-0","article-title":"Lung cancer in patients who have never smoked - an emerging disease","volume":"21","author":"LoPiccolo","journal-title":"Nat Rev Clin Oncol"},{"issue":"3","key":"R36","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.241476","article-title":"Diagnostic accuracy and clinical value of a domain-specific multimodal generative AI model for chest radiograph report generation","volume":"314","author":"Hong","journal-title":"Radiology"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","first-page":"e12","DOI":"10.1016\/S2589-7500(23)00225-X","article-title":"Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study","volume":"6","author":"Zack","journal-title":"Lancet Digit Health"},{"issue":"1","key":"R38","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232133","article-title":"BI-RADS category assignments by GPT-3.5, GPT-4, and Google Bard: a multilanguage study","volume":"311","author":"Cozzi","journal-title":"Radiology"},{"issue":"3","key":"R39","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.241736","article-title":"Large language model ability to translate CT and MRI free-text radiology reports into multiple languages","volume":"313","author":"Meddeb","journal-title":"Radiology"},{"issue":"2","key":"R40","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.241051","article-title":"Assessing completeness of clinical histories accompanying imaging orders using adapted open-source and closed-source large language models","volume":"314","author":"Larson","journal-title":"Radiology"},{"issue":"2","key":"R41","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.242134","article-title":"Leveraging large language models to generate clinical histories for oncologic imaging requisitions","volume":"314","author":"Bhayana","journal-title":"Radiology"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T15:59:06Z","timestamp":1787759946000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e94904"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,26]]},"references-count":41,"URL":"https:\/\/doi.org\/10.2196\/94904","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,26]]},"article-number":"v28i5e94904"}}