{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,29]],"date-time":"2026-10-29T00:13:46Z","timestamp":1793232826310,"version":"4.5.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T00:00:00Z","timestamp":1776297600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012681","name":"Universit\u00e4tsklinikum K\u00f6ln","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100012681","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Follow-up imaging recommendations vary across radiologists despite established guidelines. We evaluated whether a large language model (GPT-4o) can standardize follow-up timing and modality selection from routine radiology reports. In this retrospective, two-center study, 100 CT\/MRI cases (25 each: head\/neck, liver, lung, pancreas) were randomly sampled. GPT-4o and two human readers (R1, resident; R2, board-certified) generated follow-up recommendations from report text. Expert consensus, blinded to rater, assessed completeness, appropriateness of imaging modality, timing accuracy, and global follow up quality on a 5-point scale, with 5 being the highest score. Median global quality was 4 (2\u20135) for GPT-4o, 4 (1\u20135) for R1, and 4 (1\u20135) for R2; GPT-4o exceeded R1 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.01) and did not differ from 2 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.06). Relative treatment effect showed a reader effect (GPT4o 0.56 vs. R1 0.43 vs. R2 0.51;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.01) and no center effect (0.50 vs. 0.50;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.91). Correctness of follow-up timing was 96% (96\/100) for GPT-4o, 75% (75\/100) for R1, and 90% (90\/100) for R2 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001 for GPT-4o vs. R1). Completeness of follow-up was 92% (92\/100) for GPT-4o, 91% (91\/100) for R1, and 80% (80\/100) for R2 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.014 for GPT-4o vs. Reader 2). No significant differences were observed regarding appropriateness of imaging modality. GPT-4o generated follow-up recommendations with overall quality comparable to an experienced radiologist and superior to a trainee, with high completeness and generally appropriate follow up timing and modality, supporting its role as decision support for standardized, guideline-aligned follow-up.\n                  <\/jats:p>","DOI":"10.1038\/s41598-026-40317-9","type":"journal-article","created":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T16:07:37Z","timestamp":1776355657000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["GPT-4o for Automated Determination of Follow-up Examinations Based on Radiology Reports from Clinical Routine"],"prefix":"10.1038","volume":"16","author":[{"given":"Kenan","family":"Kaya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"M\u00fcller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thorsten","family":"Persigehl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Zopfs","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Nelles","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Dratsch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andra-Iza","family":"Iuga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan Paul","family":"Janssen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Sch\u00f6mig","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan","family":"Kottlors","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carsten","family":"Gietzen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Maintz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lenhard","family":"Pennig","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Jorg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nedim","family":"Beste","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Terzis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,16]]},"reference":[{"issue":"11","key":"40317_CR1","doi-asserted-by":"publisher","first-page":"1429","DOI":"10.1016\/j.jacr.2017.07.018","volume":"14","author":"RM Gore","year":"2017","unstructured":"Gore, R. M. et al. Management of incidental liver lesions on CT: A white paper of the ACR incidental findings committee. J. Am. Coll. Radiol. 14 (11), 1429\u20131437 (2017).","journal-title":"J. Am. Coll. Radiol."},{"issue":"1","key":"40317_CR2","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1148\/radiol.2017161659","volume":"284","author":"H MacMahon","year":"2017","unstructured":"MacMahon, H. et al. Guidelines for management of incidental pulmonary nodules detected on CT images: from the Fleischner society 2017. Radiology 284 (1), 228\u2013243 (2017).","journal-title":"Radiology"},{"issue":"3","key":"40317_CR3","doi-asserted-by":"publisher","first-page":"700","DOI":"10.1148\/radiol.2019182826","volume":"291","author":"LR Cochon","year":"2019","unstructured":"Cochon, L. R. et al. Variation in Follow-up imaging recommendations in radiology reports: Patient, Modality, and radiologist predictors. Radiology 291 (3), 700\u2013707 (2019).","journal-title":"Radiology"},{"issue":"1","key":"40317_CR4","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1148\/radiol.09091556","volume":"255","author":"RL Eisenberg","year":"2010","unstructured":"Eisenberg, R. L., Bankier, A. A. & Boiselle, P. M. Compliance with Fleischner society guidelines for management of small lung nodules: a survey of 834 radiologists. Radiology 255 (1), 218\u2013224 (2010).","journal-title":"Radiology"},{"issue":"1","key":"40317_CR5","doi-asserted-by":"publisher","first-page":"e232714","DOI":"10.1148\/radiol.232714","volume":"311","author":"RJ Gertz","year":"2024","unstructured":"Gertz, R. J. et al. Potential of GPT-4 for detecting errors in radiology reports: implications for reporting accuracy. Radiology 311 (1), e232714 (2024).","journal-title":"Radiology"},{"issue":"1","key":"40317_CR6","doi-asserted-by":"publisher","first-page":"e232640","DOI":"10.1148\/radiol.232640","volume":"312","author":"KR Laukamp","year":"2024","unstructured":"Laukamp, K. R. et al. Monitoring patients with glioblastoma by using a large Language model: accurate summarization of radiology reports with GPT-4. Radiology 312 (1), e232640 (2024).","journal-title":"Radiology"},{"key":"40317_CR7","doi-asserted-by":"crossref","unstructured":"Gertz, R. J. et al. GPT-4 for automated determination of radiological study and protocol based on radiology request forms: A feasibility study. Radiology, 307(5). (2023).","DOI":"10.1148\/radiol.230877"},{"issue":"5","key":"40317_CR8","doi-asserted-by":"publisher","first-page":"e231259","DOI":"10.1148\/radiol.231259","volume":"307","author":"Z Sun","year":"2023","unstructured":"Sun, Z. et al. Evaluating GPT4 on impressions generation in radiology reports. Radiology 307 (5), e231259 (2023).","journal-title":"Radiology"},{"issue":"7","key":"40317_CR9","doi-asserted-by":"publisher","first-page":"808","DOI":"10.1007\/s11547-023-01651-4","volume":"128","author":"CA Mallio","year":"2023","unstructured":"Mallio, C. A. et al. Large Language models for structured reporting in radiology: performance of GPT-4, ChatGPT-3.5, perplexity and Bing. Radiol. Med. 128 (7), 808\u2013812 (2023).","journal-title":"Radiol. Med."},{"issue":"10","key":"40317_CR10","doi-asserted-by":"publisher","first-page":"990","DOI":"10.1016\/j.jacr.2023.05.003","volume":"20","author":"A Rao","year":"2023","unstructured":"Rao, A. et al. Evaluating GPT as an adjunct for radiologic decision making: GPT-4 versus GPT-3.5 in a breast imaging pilot. J. Am. Coll. Radiol. 20 (10), 990\u2013997 (2023).","journal-title":"J. Am. Coll. Radiol."},{"key":"40317_CR11","doi-asserted-by":"crossref","unstructured":"Terzis, R. et al. GPT-4 for automated sequence-level determination of MRI protocols based on radiology request forms from clinical routine. Eur. Radiol., (2025).","DOI":"10.1007\/s00330-025-11888-4"},{"issue":"2","key":"40317_CR12","doi-asserted-by":"publisher","first-page":"101068","DOI":"10.1016\/j.jocmr.2024.101068","volume":"26","author":"K Kaya","year":"2024","unstructured":"Kaya, K. et al. Generative Pre-trained transformer 4 analysis of cardiovascular magnetic resonance reports in suspected myocarditis: A multicenter study. J. Cardiovasc. Magn. Reson. 26 (2), 101068 (2024).","journal-title":"J. Cardiovasc. Magn. Reson."},{"key":"40317_CR13","doi-asserted-by":"publisher","first-page":"112341","DOI":"10.1016\/j.ejrad.2025.112341","volume":"191","author":"R Terzis","year":"2025","unstructured":"Terzis, R. et al. Evaluation of GPT-4o for multilingual translation of radiology reports across imaging modalities. Eur. J. Radiol. 191, 112341 (2025).","journal-title":"Eur. J. Radiol."},{"issue":"5","key":"40317_CR14","doi-asserted-by":"publisher","first-page":"3173","DOI":"10.1007\/s00330-021-08431-6","volume":"32","author":"M Gabelloni","year":"2022","unstructured":"Gabelloni, M. et al. Bridging gaps between images and data: a systematic update on imaging biobanks. Eur. Radiol. 32 (5), 3173\u20133186 (2022).","journal-title":"Eur. Radiol."},{"key":"40317_CR15","doi-asserted-by":"crossref","unstructured":"Zhou, J. et al. Large Language models in biomedicine and healthcare. Npj Artif. Intell., 1(1). (2025).","DOI":"10.1038\/s44387-025-00047-1"},{"issue":"5","key":"40317_CR16","doi-asserted-by":"publisher","first-page":"331","DOI":"10.6004\/jnccn.2024.0035","volume":"22","author":"WJ Gradishar","year":"2024","unstructured":"Gradishar, W. J. et al. Version 3.2024, NCCN clinical practice guidelines in oncology. J. Natl. Compr. Canc Netw. 22 (5), 331\u2013357 (2024).","journal-title":"J. Natl. Compr. Canc Netw."},{"issue":"7","key":"40317_CR17","doi-asserted-by":"publisher","first-page":"1313","DOI":"10.3174\/ajnr.A7116","volume":"42","author":"AL Fourez","year":"2021","unstructured":"Fourez, A. L. et al. Optimal duration of MRI Follow-up to safely identify middle ear residual cholesteatoma. AJNR Am. J. Neuroradiol. 42 (7), 1313\u20131319 (2021).","journal-title":"AJNR Am. J. Neuroradiol."},{"issue":"3","key":"40317_CR18","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1055\/a-1350-1078","volume":"75","author":"T Kraus","year":"2021","unstructured":"Kraus, T. & Teschler, H. [Update of the AWMF S2k guideline diagnostics and assessment of occupational Asbestos-Related Diseases - What\u2019s New?]. Pneumologie 75 (3), 201\u2013205 (2021).","journal-title":"Pneumologie"},{"issue":"5","key":"40317_CR19","doi-asserted-by":"publisher","first-page":"738","DOI":"10.1016\/j.pan.2017.07.007","volume":"17","author":"M Tanaka","year":"2017","unstructured":"Tanaka, M. et al. Revisions of international consensus Fukuoka guidelines for the management of IPMN of the pancreas. Pancreatology 17 (5), 738\u2013753 (2017).","journal-title":"Pancreatology"},{"issue":"3","key":"40317_CR20","first-page":"203","volume":"66","author":"EE Cohen","year":"2016","unstructured":"Cohen, E. E. et al. American cancer society head and neck cancer survivorship care guideline. CA Cancer J. Clin. 66 (3), 203\u2013239 (2016).","journal-title":"CA Cancer J. Clin."},{"key":"40317_CR21","doi-asserted-by":"publisher","first-page":"100655","DOI":"10.1016\/j.ejro.2025.100655","volume":"14","author":"J Wen","year":"2025","unstructured":"Wen, J. et al. Evaluation of large Language models in generating pulmonary nodule follow-up recommendations. Eur. J. Radiol. Open. 14, 100655 (2025).","journal-title":"Eur. J. Radiol. Open."},{"issue":"1","key":"40317_CR22","doi-asserted-by":"publisher","first-page":"e2532729","DOI":"10.2214\/AJR.25.32729","volume":"225","author":"Z Zhu","year":"2025","unstructured":"Zhu, Z. et al. Multimodal large Language model with knowledge retrieval using flowchart embedding for forming Follow-Up recommendations for pancreatic cystic lesions. AJR Am. J. Roentgenol. 225 (1), e2532729 (2025).","journal-title":"AJR Am. J. Roentgenol."},{"issue":"10","key":"40317_CR23","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1016\/j.ijom.2024.04.002","volume":"53","author":"Y Cai","year":"2024","unstructured":"Cai, Y. et al. Exploring the use of ChatGPT\/GPT-4 for patient follow-up after oral surgeries. Int. J. Oral Maxillofac. Surg. 53 (10), 867\u2013872 (2024).","journal-title":"Int. J. Oral Maxillofac. Surg."},{"issue":"5","key":"40317_CR24","doi-asserted-by":"publisher","first-page":"541","DOI":"10.6004\/jnccn.2021.0022","volume":"19","author":"AB Benson","year":"2021","unstructured":"Benson, A. B. et al. Version 2.2021, NCCN clinical practice guidelines in oncology. J. Natl. Compr. Canc Netw. 19 (5), 541\u2013565 (2021).","journal-title":"J. Natl. Compr. Canc Netw."},{"key":"40317_CR25","doi-asserted-by":"crossref","unstructured":"Busch, F. et al. Large Language models for structured reporting in radiology: past, present, and future. Eur. Radiol., (2024).","DOI":"10.1007\/s00330-024-11107-6"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-40317-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-40317-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-40317-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T22:13:39Z","timestamp":1781648019000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-40317-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,16]]},"references-count":25,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["40317"],"URL":"https:\/\/doi.org\/10.1038\/s41598-026-40317-9","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,16]]},"assertion":[{"value":"13 December 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"12587"}}