{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T02:50:21Z","timestamp":1792551021002,"version":"4.5.0"},"reference-count":27,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T00:00:00Z","timestamp":1750118400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Tomography"],"abstract":"<jats:p>Objectives: to assess the ability of high-performing open-weight large language models (LLMs) in extracting key radiological features from prostate MRI reports. Methods: Five LLMs (Llama3.3, DeepSeek-R1-Llama3.3, Phi4, Gemma-2, and Qwen2.5-14B) were used to analyze free-text MRI reports retrieved from clinical practice. Each LLM processed reports three times using specialized prompts to extract (1) dimensions, (2) volume and PSA density, and (3) lesion characteristics. An experienced radiologist manually annotated the dataset, defining entities (Exam) and sub-entities (Lesion, Dimension). Feature- and physician-level performance were then assessed. Results: 250 MRI exams reported by 7 radiologists were analyzed by the LLMs. Feature-level performances showed that DeepSeek-R1-Llama3.3 exhibited the highest average score (98.6% \u00b1 2.1%), followed by Phi4 (98.1% \u00b1 2.2%), Llama3.3 (98.0% \u00b1 3.0%), Qwen2.5 (97.5% \u00b1 3.9%), and Gemma2 (96.0% \u00b1 3.4%). All models excelled in extracting PSA density (100%) and volume (\u226598.4%), while lesions\u2019 extraction showed greater variability (88.4\u201394.0%). LLMs\u2019 performance varied among radiologists: Physician B\u2019s reports yielded the highest mean score (99.9% \u00b1 0.2%), while Physician C\u2019s resulted in the lowest (94.4% \u00b1 2.3%). Conclusions: LLMs showed promising results in automated feature-extraction from radiology reports, with DeepSeek-R1-Llama3.3 achieving the highest overall score. These models can improve clinical workflows by structuring unstructured medical text. However, a preliminary analysis of reporting styles is necessary to identify potential challenges and optimize prompt design to better align with individual physician reporting styles. This approach can further enhance the robustness and adaptability of LLM-driven clinical data extraction.<\/jats:p>","DOI":"10.3390\/tomography11060069","type":"journal-article","created":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:08:06Z","timestamp":1750219686000},"page":"69","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Structured Transformation of Unstructured Prostate MRI Reports Using Large Language Models"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-0598-7354","authenticated-orcid":false,"given":"Luca","family":"Di Palma","sequence":"first","affiliation":[{"name":"CDI Centro Diagnostico Italiano, Saint Bon 20, 20147 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fatemeh","family":"Darvizeh","sequence":"additional","affiliation":[{"name":"CDI Centro Diagnostico Italiano, Saint Bon 20, 20147 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Al\u00ec","sequence":"additional","affiliation":[{"name":"CDI Centro Diagnostico Italiano, Saint Bon 20, 20147 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deborah","family":"Fazzini","sequence":"additional","affiliation":[{"name":"CDI Centro Diagnostico Italiano, Saint Bon 20, 20147 Milan, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1148\/radiol.2523081992","article-title":"Toward best practices in radiology reporting","volume":"252","author":"Kahn","year":"2009","journal-title":"Radiology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1007\/s11547-021-01418-9","article-title":"Structured reporting of computed tomography in the staging of colon cancer: A Delphi consensus proposal","volume":"127","author":"Granata","year":"2022","journal-title":"Radiol. Med."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pesapane, F., Tantrige, P., De Marco, P., Carriero, S., Zugni, F., Nicosia, L., Bozzini, A.C., Rotili, A., Latronico, A., and Abbate, F. (2023). Advancements in standardizing radiological reports: A comprehensive review. Med. Kaunas, 59.","DOI":"10.3390\/medicina59091679"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1055\/s-0039-1695791","article-title":"Interactive NLP in clinical care: Identifying incidental findings in radiology reports","volume":"10","author":"Trivedi","year":"2019","journal-title":"Appl. Clin. Inf."},{"key":"ref_5","unstructured":"Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional Transformers for language understanding. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s10278-022-00712-w","article-title":"Natural language processing model for identifying critical findings-A multi-institutional study","volume":"36","author":"Banerjee","year":"2023","journal-title":"J. Digit. Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Doshi, R., Amin, K.S., Khosla, P., Bajaj, S., Chheang, S., and Forman, H.P. (2024). Quantitative evaluation of large language models to streamline radiology report impressions: A multimodal retrospective analysis. Radiology, 310.","DOI":"10.1148\/radiol.231593"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.fas.2024.01.008","article-title":"From jargon to clarity: Improving the readability of foot and ankle radiology reports with an artificial intelligence large language model","volume":"30","author":"Butler","year":"2024","journal-title":"Foot Ankle Surg."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.1002\/ksa.12133","article-title":"From technical to understandable: Artificial Intelligence Large Language Models improve the readability of knee radiology reports","volume":"32","author":"Butler","year":"2024","journal-title":"Knee Surg. Sports Traumatol. Arthrosc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1016\/j.spinee.2024.10.021","article-title":"MRI spine request form enhancement and auto protocoling using a secure institutional large language model","volume":"25","author":"Hallinan","year":"2025","journal-title":"Spine J."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gertz, R.J., Dratsch, T., Bunck, A.C., Lennartz, S., Iuga, A.-I., Hellmich, M.G., Persigehl, T., Pennig, L., Gietzen, C.H., and Fervers, P. (2024). Potential of GPT-4 for detecting errors in radiology reports: Implications for reporting accuracy. Radiology, 311.","DOI":"10.1148\/radiol.232714"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Reichenpfader, D., M\u00fcller, H., and Denecke, K. (2024). A scoping review of large language model based approaches for information extraction from radiology reports. NPJ Digit. Med., 7.","DOI":"10.1038\/s41746-024-01219-0"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Fervers, P., Hahnfeldt, R., Kottlors, J., Wagner, A., Maintz, D., dos Santos, D.P., Lennartz, S., and Persigehl, T. (2024). ChatGPT yields low accuracy in determining LI-RADS scores based on free-text and structured radiology reports in German language. Front. Radiol., 4.","DOI":"10.3389\/fradi.2024.1390774"},{"key":"ref_14","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The Llama 3 Herd of Models. arXiv."},{"key":"ref_15","unstructured":"DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., and Wang, P. (2025). DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv."},{"key":"ref_16","unstructured":"Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R.J., Javaheripi, M., and Kauffmann, P. (2024). Phi-4 Technical Report. arXiv."},{"key":"ref_17","unstructured":"Riviere, M., Pathak, S., Sessa, P.G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ram\u00e9, A., and Ferret, J. (2024). Gemma 2: Improving Open Language Models at a Practical Size. arXiv."},{"key":"ref_18","unstructured":"Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., and Wei, H. (2024). Qwen2.5 Technical Report. arXiv."},{"key":"ref_19","unstructured":"Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., and Huang, F. (2024). Qwen2 Technical Report. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jiang, H., Xia, S., Yang, Y., Xu, J., Hua, Q., Mei, Z., Hou, Y., Wei, M., Lai, L., and Li, N. (2024). Transforming free-text radiology reports into structured reports using ChatGPT: A study on thyroid ultrasonography. Eur. J. Radiol., 175.","DOI":"10.1016\/j.ejrad.2024.111458"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"808","DOI":"10.1007\/s11547-023-01651-4","article-title":"Large language models for structured reporting in radiology: Performance of GPT-4, ChatGPT-3.5, Perplexity and Bing","volume":"128","author":"Mallio","year":"2023","journal-title":"Radiol. Med."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Siepmann, R.M., Baldini, G., Schmidt, C.S., Truhn, D., M\u00fcller-Franzes, G.A., Dada, A., Kleesiek, J., Nensa, F., and Hosch, R. (2025). An automated information extraction model for unstructured discharge letters using large language models and GPT-4. Healthc. Anal., 7.","DOI":"10.1016\/j.health.2024.100378"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Nowak, S., Wulff, B., Layer, Y.C., Theis, M., Isaak, A., Salam, B., Block, W., Kuetting, D., Pieper, C.C., and Luetkens, J.A. (2025). Privacy-ensuring open-weights large language models are competitive with closed-weights GPT-4o in extracting chest radiography findings from free-text reports. Radiology, 314.","DOI":"10.1148\/radiol.240895"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Fink, M.A., Bischoff, A., Fink, C.A., Moll, M., Kroschke, J., Dulz, L., Heu\u00dfel, C.P., Kauczor, H.-U., and Weber, T.F. (2023). Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology, 308.","DOI":"10.1148\/radiol.231362"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Adams, L.C., Truhn, D., Busch, F., Kader, A., Niehues, S.M., Makowski, M.R., and Bressem, K.K. (2023). Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: A multilingual feasibility study. Radiology, 307.","DOI":"10.1148\/radiol.230725"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2018","DOI":"10.1007\/s00330-024-11074-y","article-title":"Automatic structuring of radiology reports with on-premise open-source large language models","volume":"35","author":"Laqua","year":"2024","journal-title":"Eur. Radiol."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mukherjee, P., Hou, B., Lanfredi, R.B., and Summers, R.M. (2023). Feasibility of using the privacy-preserving Large language model Vicuna for labeling radiology reports. Radiology, 309.","DOI":"10.1148\/radiol.231147"}],"container-title":["Tomography"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2379-139X\/11\/6\/69\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:53:56Z","timestamp":1760032436000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2379-139X\/11\/6\/69"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,17]]},"references-count":27,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["tomography11060069"],"URL":"https:\/\/doi.org\/10.3390\/tomography11060069","relation":{},"ISSN":["2379-139X"],"issn-type":[{"value":"2379-139X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,17]]}}}