{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T06:38:13Z","timestamp":1781073493833,"version":"3.54.1"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:00:00Z","timestamp":1753401600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:00:00Z","timestamp":1753401600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005713","name":"Technische Universit\u00e4t M\u00fcnchen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005713","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Efficient processing of radiology reports for monitoring disease progression is crucial in oncology. Although large language models (LLMs) show promise in extracting structured information from medical reports, privacy concerns limit their clinical implementation. This study evaluates the feasibility and accuracy of two of the most recent Llama models for generating structured lymphoma progression reports from cross-sectional imaging data in a privacy-preserving, real-world clinical setting.\u00a0This single-center, retrospective study included adult lymphoma patients who underwent cross-sectional imaging and treatment between July 2023 and July 2024. We established a chain-of-thought prompting strategy to leverage the locally deployed Llama-3.3-70B-Instruct and Llama-4-Scout-17B-16E-Instruct models to generate lymphoma disease progression reports across three iterations. Two radiologists independently scored nodal and extranodal involvement, as well as Lugano staging and treatment response classifications. For each LLM and task, we calculated the F1 score, accuracy, recall, precision, and specificity per label, as well as the case-weighted average with 95% confidence intervals (CIs).\u00a0Both LLMs correctly implemented the template structure for all 65 patients included in this study. Llama-4-Scout-17B-16E-Instruct demonstrated significantly greater accuracy in extracting nodal and extranodal involvement information (nodal: 0.99 [95% CI\u2009=\u20090.98\u20130.99] vs. 0.97 [95% CI\u2009=\u20090.95\u20130.96],\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001; extranodal: 0.99 [95% CI\u2009=\u20090.99\u20131.00] vs. 0.99 [95% CI\u2009=\u20090.98\u20130.99],\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.013). This difference was more pronounced when predicting Lugano stage and treatment response (stage: 0.85 [95% CI\u2009=\u20090.79\u20130.89] vs. 0.60 [95% CI\u2009=\u20090.53\u20130.67],\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001; treatment response: 0.88 [95% CI\u2009=\u20090.83\u20130.92] vs. 0.65 [95% CI\u2009=\u20090.58\u20130.71],\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001). Neither model produced hallucinations of newly involved nodal or extranodal sites. The highest relative error rates were found when interpreting the level of disease after treatment.\u00a0In conclusion,\u00a0privacy-preserving LLMs can effectively extract clinical information from lymphoma imaging reports. While they excel at data extraction, they are limited in their ability to generate new clinical inferences from the extracted information. Our findings suggest their potential utility in streamlining documentation and highlight areas requiring optimization before clinical implementation.\n                  <\/jats:p>","DOI":"10.1007\/s10278-025-01618-z","type":"journal-article","created":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T17:01:45Z","timestamp":1753462905000},"page":"1868-1878","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Privacy-Preserving Generation of Structured Lymphoma Progression Reports from Cross-sectional Imaging: A Comparative Analysis of Llama 3.3 and Llama 4"],"prefix":"10.1007","volume":"39","author":[{"given":"Philipp","family":"Prucker","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keno K.","family":"Bressem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Su Hwan","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dominik","family":"Weller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Avan","family":"Kader","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felix J.","family":"Dorfner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sebastian","family":"Ziegelmayer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus M.","family":"Graf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tristan","family":"Lemke","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florian","family":"Gassert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elif","family":"Can","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aymen","family":"Meddeb","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Truhn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin","family":"Hadamitzky","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus R.","family":"Makowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa C.","family":"Adams","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9770-8555","authenticated-orcid":false,"given":"Felix","family":"Busch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,25]]},"reference":[{"key":"1618_CR1","doi-asserted-by":"crossref","unstructured":"Langlotz CP. The Radiology Report: A Guide to Thoughtful Communication for Radiologists and  Other Medical Professionals. Radiology. 2016; 278(3):677-8.\u00a0","DOI":"10.1148\/radiol.2016164001"},{"key":"1618_CR2","doi-asserted-by":"crossref","unstructured":"Atutornu J, Hayre CMa. Personalised Medicine and Medical Imaging: Opportunities and Challenges for Contemporary Health Care. J Med Imaging Radiat Sci. 2018; 49(4):352\u20139.","DOI":"10.1016\/j.jmir.2018.07.002"},{"key":"1618_CR3","doi-asserted-by":"crossref","unstructured":"Goldberg-Stein S, Chernyak V. Adding Value in Radiology Reporting. Journal of the American College of Radiology. 2019; 16(9, Part B):1292\u20138.","DOI":"10.1016\/j.jacr.2019.05.042"},{"key":"1618_CR4","doi-asserted-by":"crossref","unstructured":"McDonald RJ, Schwartz KM, Eckel LJ, et al. The effects of changes in utilization and technological  advancements of cross-sectional imaging on radiologist workload. Acad Radiol. 2015; 22(9):1191-8.","DOI":"10.1016\/j.acra.2015.05.007"},{"key":"1618_CR5","doi-asserted-by":"crossref","unstructured":"Lesslie MD, Parikh JR. Multidisciplinary Tumor Boards: An Opportunity for Radiologists to  Demonstrate Value. Acad Radiol. 2017; 24(1):107-10.","DOI":"10.1016\/j.acra.2016.09.006"},{"key":"1618_CR6","doi-asserted-by":"crossref","unstructured":"Busch F, Hoffmann L, dos Santos DP, et al. Large language models for structured reporting in radiology: past, present, and future. European Radiology. 2024.","DOI":"10.1007\/s00330-024-11107-6"},{"key":"1618_CR7","doi-asserted-by":"crossref","unstructured":"Clusmann J, Kolbinger FR, Muti HS, et al. The future landscape of large language models in medicine.  Communications Medicine. 2023; 3(1):141.\u00a0","DOI":"10.1038\/s43856-023-00370-1"},{"key":"1618_CR8","doi-asserted-by":"crossref","unstructured":"Change C-H, Lucas MM, Lu-Yao G, Yang CC. Classifying cancer stage with open-source clinical large language models. 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI): IEEE; 2024; 76\u201382.","DOI":"10.1109\/ICHI61247.2024.00018"},{"key":"1618_CR9","doi-asserted-by":"crossref","unstructured":"Busch F, Hoffmann L, Rueger C, et al. Current applications and challenges in large language models  for patient care: a systematic review. Communications Medicine. 2025; 5(1):26.","DOI":"10.1038\/s43856-024-00717-2"},{"key":"1618_CR10","doi-asserted-by":"crossref","unstructured":"Annas GJ. HIPAA regulations - a new era of medical-record privacy? N Engl J Med. 2003;  348(15):1486-9","DOI":"10.1056\/NEJMlim035027"},{"key":"1618_CR11","unstructured":"Regulation GDP. Regulation of the European Parliament and of the Council. European Commission [online] http:\/\/ec europa eu\/justice\/data-protection\/document\/review2012\/com_2012_11_en pdf (accessed 13 September 2013). 2012."},{"key":"1618_CR12","doi-asserted-by":"crossref","unstructured":"Recht MP, Dewey M, Dreyer K, et al. Integrating artificial intelligence into the clinical practice of  radiology: challenges and recommendations. Eur Radiol. 2020; 30(6):3576-84.","DOI":"10.1007\/s00330-020-06672-5"},{"key":"1618_CR13","doi-asserted-by":"crossref","unstructured":"Sasaki F, Tatekawa H, Mitsuyama Y, et al. Bridging Language and Stylistic Barriers in IR Standardized Reporting: Enhancing Translation and Structure Using ChatGPT-4. Journal of Vascular and Interventional Radiology. 2024; 35(3):472-5.e1.","DOI":"10.1016\/j.jvir.2023.11.014"},{"key":"1618_CR14","doi-asserted-by":"crossref","unstructured":"Mallio CA, Bernetti C, Sertorio AC, Zobel BB. ChatGPT in radiology structured reporting: analysis of  ChatGPT-3.5 Turbo and GPT-4 in reducing word count and recalling findings. Quantitative Imaging in Medicine  and Surgery. 2024; 14(2):2096-102.\u00a0","DOI":"10.21037\/qims-23-1300"},{"key":"1618_CR15","doi-asserted-by":"crossref","unstructured":"Adams LC, Truhn D, Busch F, et al. Leveraging GPT-4 for Post Hoc Transformation of Free-text  Radiology Reports into Structured Reporting: A Multilingual Feasibility Study. Radiology. 2023;  307(4):e230725.","DOI":"10.1148\/radiol.230725"},{"key":"1618_CR16","doi-asserted-by":"crossref","unstructured":"Bosbach WA, Senge JF, Nemeth B, et al. Ability of ChatGPT to generate competent radiology reports  for distal radius fracture by use of RSNA template items and integrated AO classifier. Current Problems in  Diagnostic Radiology. 2024; 53(1):102-10.","DOI":"10.1067\/j.cpradiol.2023.04.001"},{"key":"1618_CR17","doi-asserted-by":"crossref","unstructured":"Wang Z, Guo R, Sun P, Qian L, Hu X. Enhancing Diagnostic Accuracy and Efficiency with GPT-4- Generated Structured Reports: A Comprehensive Study. Journal of Medical and Biological Engineering. 2024;  44(1):144-53.\u00a0","DOI":"10.1007\/s40846-024-00849-9"},{"key":"1618_CR18","doi-asserted-by":"crossref","unstructured":"Jiang H, Xia S, Yang Y, et al. Transforming free-text radiology reports into structured reports using  ChatGPT: A study on thyroid ultrasonography. European Journal of Radiology. 2024; 175:111458.","DOI":"10.1016\/j.ejrad.2024.111458"},{"key":"1618_CR19","unstructured":"Pan Y, Fang J, Zhu C, Li M, Wu H. Towards an Automatic Transformer to Fhir Structured Radiology Report Via Gpt-4. Available at SSRN 4717860."},{"key":"1618_CR20","doi-asserted-by":"crossref","unstructured":"Bergomi L, Buonocore TM, Antonazzo P, et al. Reshaping Free-Text Radiology Notes Into Structured Reports With Generative Transformers. arXiv preprint arXiv:240318938. 2024.","DOI":"10.1016\/j.artmed.2024.102924"},{"key":"1618_CR21","doi-asserted-by":"crossref","unstructured":"Li H, Wang H, Sun X, He H, Feng J. Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM. arXiv preprint arXiv:240411209. 2024.","DOI":"10.1109\/ICME57554.2024.10687707"},{"key":"1618_CR22","unstructured":"Grattafiori A, Dubey A, Jauhri A, et al. The llama 3 herd of models. arXiv preprint arXiv:240721783. 2024."},{"key":"1618_CR23","unstructured":"Meta. The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation. Available at: https:\/\/ai.meta.com\/blog\/llama-4-multimodal-intelligence\/. Accessed 09\/07\/2025."},{"key":"1618_CR24","unstructured":"Rosenberg SA, Boiron M, DeVita VT, Jr., et al. Report of the Committee on Hodgkin\u2019s Disease Staging Procedures. Cancer Res. 1971; 31(11):1862-3."},{"key":"1618_CR25","unstructured":"Cheson BD. Staging and response assessment in lymphomas: the new Lugano classification. Chin Clin  Oncol. 2015; 4(1):5.\u00a0"},{"key":"1618_CR26","doi-asserted-by":"crossref","unstructured":"Cheson BD, Fisher RI, Barrington SF, et al. Recommendations for initial evaluation, staging, and  response assessment of Hodgkin and non-Hodgkin lymphoma: the Lugano classification. J Clin Oncol. 2014;  32(27):3059-68.","DOI":"10.1200\/JCO.2013.54.8800"},{"key":"1618_CR27","doi-asserted-by":"crossref","unstructured":"Le Guellec B, Lef\u00e8vre A, Geay C, et al. Performance of an Open-Source Large Language Model in Extracting Information from Free-Text Radiology Reports. Radiology: Artificial Intelligence. 2024;6(4):e230364.","DOI":"10.1148\/ryai.230364"},{"key":"1618_CR28","doi-asserted-by":"crossref","unstructured":"Reichenpfader D, M\u00fcller H, Denecke K. A scoping review of large language model based approaches  for information extraction from radiology reports. npj Digital Medicine. 2024; 7(1):222.","DOI":"10.1038\/s41746-024-01219-0"},{"key":"1618_CR29","doi-asserted-by":"crossref","unstructured":"Fast D, Adams LC, Busch F, et al. Autonomous medical evaluation for guideline adherence of large  language models. NPJ Digital Medicine. 2024; 7(1):1-14.","DOI":"10.1038\/s41746-024-01356-6"},{"key":"1618_CR30","unstructured":"Hamer DMd, Schoor P, Polak TB, Kapitan D. Improving patient pre-screening for clinical trials: assisting physicians with large language models. arXiv preprint arXiv:230407396. 2023."},{"key":"1618_CR31","unstructured":"Li Y, Xu J, Liang T, et al. Dancing with critiques: Enhancing llm reasoning with stepwise natural language self-critique. arXiv preprint arXiv:250317363. 2025."},{"key":"1618_CR32","unstructured":"Gao Y, Xiong Y, Gao X, et al. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:231210997. 2023."},{"key":"1618_CR33","doi-asserted-by":"crossref","unstructured":"Wu J, Zhu J, Qi Y, et al. Medical graph rag: Towards safe medical large language model via graph retrieval-augmented generation. arXiv preprint arXiv:240804187. 2024.","DOI":"10.18653\/v1\/2025.acl-long.1381"},{"key":"1618_CR34","doi-asserted-by":"crossref","unstructured":"Eichenauer DA, Aleman BM, Andr\u00e9 M, et al. Hodgkin lymphoma: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Annals of Oncology. 2018; 29:iv19-iv29.","DOI":"10.1093\/annonc\/mdy080"},{"key":"1618_CR35","doi-asserted-by":"crossref","unstructured":"Hoppe RT, Advani RH, Ai WZ, et al. NCCN Guidelines\u00ae insights: Hodgkin lymphoma, version  2.2022: Featured updates to the NCCN Guidelines. Journal of the National Comprehensive Cancer Network.  2022; 20(4):322-34.","DOI":"10.6004\/jnccn.2022.0021"},{"key":"1618_CR36","doi-asserted-by":"crossref","unstructured":"Li Y, Wehbe RM, Ahmad FS, Wang H, Luo Y. A comparative study of pretrained language models for  long clinical text. J Am Med Inform Assoc. 2023; 30(2):340-7.","DOI":"10.1093\/jamia\/ocac225"},{"key":"1618_CR37","doi-asserted-by":"crossref","unstructured":"Jin C, Zhang M, Ma W, et al. RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining2024; 5218\u201329.","DOI":"10.1145\/3637528.3671644"},{"key":"1618_CR38","unstructured":"Guo D, Yang D, Zhang H, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:250112948. 2025."},{"key":"1618_CR39","doi-asserted-by":"crossref","unstructured":"Nagarajan R, Kondo M, Salas F, et al. Economics and equity of large language models: health care  perspective. Journal of Medical Internet Research. 2024; 26:e64226.","DOI":"10.2196\/64226"},{"key":"1618_CR40","unstructured":"Amazon. Amazon Web Services. Available at: https:\/\/aws.amazon.com\/. Accessed 16\/02\/2025."},{"key":"1618_CR41","doi-asserted-by":"crossref","unstructured":"Seth D, Najana M, Ranjan P. Compliance and regulatory challenges in cloud computing: a sector-wise analysis. International Journal of Global Innovations and Solutions (IJGIS). 2024.","DOI":"10.21428\/e90189c8.68b5dea5"},{"key":"1618_CR42","unstructured":"Li M, Huang J, Yeung J, et al. Cancerllm: A large language model in cancer domain. arXiv preprint arXiv:240610459. 2024."},{"key":"1618_CR43","unstructured":"Dorfner FJ, Dada A, Busch F, et al. Biomedical Large Languages Models Seem not to be Superior to Generalist Models on Unseen Medical Data. arXiv preprint arXiv:240813833. 2024."}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-025-01618-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-025-01618-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-025-01618-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T16:22:06Z","timestamp":1776874926000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-025-01618-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,25]]},"references-count":43,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["1618"],"URL":"https:\/\/doi.org\/10.1007\/s10278-025-01618-z","relation":{},"ISSN":["2948-2933"],"issn-type":[{"value":"2948-2933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,25]]},"assertion":[{"value":"19 April 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 July 2025","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 July 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Keno K Bressem reports grants from the European Union (101079894), the Wilhelm Sander Foundation, the Bundesministerium f\u00fcr Bildung und Forschung (BMBF), the Else Kr\u00f6ner Foundation, Bayern Innovativ, and the Max Kade Foundation; Keno K Bressem reports speaker fees from Canon Medical Systems Corporation and GE Healthcare; Keno K Bressem is a member of the advisory board of the EU Horizon 2020 LifeChamps project (875329) and the EU IHI project IMAGIO (101112053). None of the other authors declares potential conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}