{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T06:12:40Z","timestamp":1788502360993,"version":"build-2803163510"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T00:00:00Z","timestamp":1788480000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Radiology reports contain critical information for monitoring patients' clinical progress and evaluating treatment outcomes. Sequential radiological images are particularly valuable, providing not only current findings but also comparative assessments against previous reports. Automatically detecting temporal changes between free-text reports remains a significant challenge in Natural Language Processing.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This study evaluated the performance of Large Language Models in detecting temporal novelty using the LUNGUAGE dataset, which contains sequential chest radiography reports. The comparative multi-model and multi-prompt evaluation was carried out on a fixed, class-stratified subset of 1,500 of these examples. Seven language models from OpenAI, Anthropic, and Google Gemini were tested across five prompt strategies: zero-shot, few-shot, structured reasoning, memory-augmented, and temporal graph prompting. Each model classified target findings as new, improved, worsened, stable, or resolved by evaluating current and previous reports together. Performance was measured using accuracy, macro-F1, weighted-F1, and class-based metrics. The \u201cnew\u201d class was analyzed separately due to its clinical importance. Model errors were categorized into clinically interpretable types, including missed new findings, missed resolved findings, temporal errors, and change direction errors. Bootstrap confidence intervals and paired McNemar significance tests assessed performance uncertainty.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>GPT-4.1 emerged as the top-performing model. Its few-shot prompt strategy achieved the best results: 0.8369 accuracy, 0.7888 macro-F1, 0.8322 weighted-F1, and 0.5385 recall for the \u201cnew\u201d class.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>These findings demonstrate that Large Language Models hold substantial promise for temporal clinical reasoning in sequential radiology reports, and that performance depends not only on model architecture but also on prompt strategy and the class being evaluated.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdgth.2026.1919939","type":"journal-article","created":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T05:29:55Z","timestamp":1788499795000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Detecting temporal clinical change in sequential chest radiography reports: a multi-model and multi-prompt evaluation of large language models"],"prefix":"10.3389","volume":"8","author":[{"given":"Ert\u00fcrk","family":"Erda\u011f\u0131","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Istanbul Medeniyet University","place":["\u0130stanbul, T\u00fcrkiye"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,4]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","volume":"6","author":"Johnson","year":"2019","journal-title":"Sci Data"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e79202","DOI":"10.2196\/79202","article-title":"Large language models for health care text classification: systematic review","volume":"5","author":"Sakai","year":"2026","journal-title":"JMIR AI"},{"key":"B3","doi-asserted-by":"publisher","first-page":"631","DOI":"10.3390\/bioengineering12060631","article-title":"Large language models in healthcare and medical applications: a review","volume":"12","author":"Maity","year":"2025","journal-title":"Bioengineering"},{"key":"B4","doi-asserted-by":"publisher","first-page":"7657","DOI":"10.1109\/JBHI.2025.3574254","article-title":"Comparing text-based clinical risk prediction in critical care: a note-specific hierarchical network and large language models","volume":"29","author":"Liu","year":"2025","journal-title":"IEEE J Biomed Health Inform"},{"key":"B5","doi-asserted-by":"publisher","first-page":"48987","DOI":"10.1109\/ACCESS.2024.3384496","article-title":"Evaluation and analysis of large language models for clinical text augmentation and generation","volume":"12","author":"Latif","year":"2024","journal-title":"IEEE Access"},{"key":"B6","doi-asserted-by":"publisher","first-page":"102900","DOI":"10.1016\/j.artmed.2024.102900","article-title":"Transformers and large language models in healthcare: a review","volume":"154","author":"Nerella","year":"2024","journal-title":"Artif Intell Med"},{"key":"B7","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1038\/s41746-024-01219-0","article-title":"A scoping review of large language model based approaches for information extraction from radiology reports","volume":"7","author":"Reichenpfader","year":"2024","journal-title":"npj Digit Med"},{"key":"B8","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1007\/s11604-024-01552-0","article-title":"The impact of large language models on radiology: a guide for radiologists on the latest innovations in AI","volume":"42","author":"Nakaura","year":"2024","journal-title":"Jpn J Radiol"},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-6543-1","volume-title":"Temporal Information Systems in Medicine","author":"Combi","year":"2010"},{"key":"B10","doi-asserted-by":"publisher","first-page":"2353630","DOI":"10.1080\/09540091.2024.2353630","article-title":"Global insights and the impact of generative AI-ChatGPT on multidisciplinary: a systematic review and bibliometric analysis","volume":"36","author":"Khan","year":"2024","journal-title":"Conn Sci"},{"key":"B11","doi-asserted-by":"publisher","first-page":"102988","DOI":"10.1016\/j.artmed.2024.102988","article-title":"Model development for bespoke large language models for digital triage assistance in mental health care","volume":"157","author":"Taylor","year":"2024","journal-title":"Artif Intell Med"},{"key":"B12","doi-asserted-by":"publisher","first-page":"103001","DOI":"10.1016\/j.artmed.2024.103001","article-title":"OphGLM: an ophthalmology large language-and-vision assistant","volume":"157","author":"Deng","year":"2024","journal-title":"Artif Intell Med"},{"key":"B13","doi-asserted-by":"publisher","first-page":"3639","DOI":"10.1007\/s10278-025-01728-8","article-title":"Performance of natural language processing model in extracting information from free-text radiology reports: a systematic review and meta-analysis","volume":"39","author":"Yang","year":"2026","journal-title":"J Imaging Inform Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e230364","DOI":"10.1148\/ryai.230364","article-title":"Performance of an open-source large language model in extracting information from free-text radiology reports","volume":"6","author":"Le Guellec","year":"2024","journal-title":"Radiol Artif Intell"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e65547","DOI":"10.2196\/65547","article-title":"Large language models in summarizing radiology report impressions for lung cancer in Chinese: evaluation study","volume":"27","author":"Hu","year":"2025","journal-title":"J Med Internet Res"},{"key":"B16","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.imed.2025.11.003","article-title":"From radiology findings to artificial intelligence-powered impressions: a retrospective study on the comparative performance of recent large language models","volume":"6","author":"Tasneem","year":"2026","journal-title":"Intell Med"},{"key":"B17","doi-asserted-by":"publisher","first-page":"5529107","DOI":"10.1049\/sfw2\/5529107","article-title":"Radiology report generation via self-critical multisource knowledge reasoning with large language models","volume":"2026","author":"Allaberdiev","year":"2026","journal-title":"IET Software"},{"key":"B18","doi-asserted-by":"publisher","first-page":"55","DOI":"10.2478\/jaiscr-2026-0003","article-title":"Chestxgen: dynamic memory-augmented vision-language transformer with context-aware gating for radiology report generation","volume":"16","author":"Allaberdiev","year":"2026","journal-title":"JAISCR"},{"key":"B19","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.jacr.2025.09.004","article-title":"Leveraging large language models to enhance radiology report readability: a systematic review","volume":"23","author":"Patwardhan","year":"2026","journal-title":"J Am Coll Radiol"},{"key":"B20","doi-asserted-by":"publisher","first-page":"ooaf121","DOI":"10.1093\/jamiaopen\/ooaf121","article-title":"Using large language models for temporal relation extraction from pediatric clinical reports","volume":"8","author":"Andrew","year":"2025","journal-title":"Jamia Open"},{"key":"B21","volume-title":"CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest x-rays}}","author":"Lee","year":"2025"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.13026\/pk42-4v91","article-title":"\u2018Lunguage: a benchmark for structured and sequential chest x-ray Interpretation","author":"Moon","year":"2026","journal-title":"PhysioNet"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2505.21190","article-title":"\u2018Lunguage: a benchmark for structured and sequential chest x-ray Interpretation","author":"Moon","year":"2025","journal-title":"arXiv"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.13026\/jsn5-t979","article-title":"MIMIC-CXR-JPG\u2014chest radiographs with structured labels","author":"Johnson","year":"2024","journal-title":"PhysioNet"},{"key":"B25","year":"2025"},{"key":"B26","doi-asserted-by":"publisher","first-page":"412","DOI":"10.3390\/fi17090412","article-title":"GPT-4.1 sets the standard in automated experiment design using novel python libraries","volume":"17","author":"Fachada","year":"2025","journal-title":"Future Internet"},{"key":"B27","doi-asserted-by":"publisher","first-page":"3694","DOI":"10.1038\/s41433-023-02564-2","article-title":"Comparison of GPT-3.5, GPT-4, and human user performance on a practice ophthalmology written examination","volume":"37","author":"Lin","year":"2023","journal-title":"Eye"},{"key":"B28","article-title":"\u201cGPT-4.1 mini Model\u201d, OpenAI","year":"2025","journal-title":"OpenAI"},{"key":"B29","year":"2024"},{"key":"B30","doi-asserted-by":"publisher","first-page":"443","DOI":"10.32708\/uutfd.1543370","article-title":"\u2018A comparative analysis of GPT-3.5, GPT-4 and GPT-4.o in heart Failure","volume":"50","author":"G\u00fcnay Polatkan","year":"2025","journal-title":"J Uludag Univ Med Fac"},{"key":"B31","doi-asserted-by":"publisher","first-page":"543","DOI":"10.3390\/info15090543","article-title":"From GPT-3.5 to GPT-4. o: a leap in AI's Medical exam performance","volume":"15","author":"Kipp","year":"2024","journal-title":"Information"},{"key":"B32","year":"2026"},{"key":"B33","year":"2025"},{"key":"B34","year":"2025"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1919939\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,4]],"date-time":"2026-09-04T05:29:57Z","timestamp":1788499797000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1919939\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,4]]},"references-count":34,"alternative-id":["10.3389\/fdgth.2026.1919939"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1919939","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,4]]},"article-number":"1919939"}}