{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T21:45:26Z","timestamp":1775252726724,"version":"3.50.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T00:00:00Z","timestamp":1724630400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T00:00:00Z","timestamp":1724630400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004721","name":"The University of Tokyo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Early detection of patients with impending bone metastasis is crucial for prognosis improvement. This study aimed to investigate the feasibility of a fine-tuned, locally run large language model (LLM) in extracting patients with bone metastasis in unstructured Japanese radiology report and to compare its performance with manual annotation. This retrospective study included patients with \u201cmetastasis\u201d in radiological reports (April 2018\u2013January 2019, August\u2013May 2022, and April\u2013December 2023 for training, validation, and test datasets of 9559, 1498, and 7399 patients, respectively). Radiologists reviewed the clinical indication and diagnosis sections of the radiological report (used as input data) and classified them into groups 0 (no bone metastasis), 1 (progressive bone metastasis), and 2 (stable or decreased bone metastasis). The data for group 0 was under-sampled in training and test datasets due to group imbalance. The best-performing model from the validation set was subsequently tested using the testing dataset. Two additional radiologists (readers 1 and 2) were involved in classifying radiological reports within the test dataset for testing purposes. The fine-tuned LLM, reader 1, and reader 2 demonstrated an accuracy of 0.979, 0.996, and 0.993, sensitivity for groups 0\/1\/2 of 0.988\/0.947\/0.943, 1.000\/1.000\/0.966, and 1.000\/0.982\/0.954, and time required for classification (s) of 105, 2312, and 3094 in under-sampled test dataset (<jats:italic>n<\/jats:italic>\u2009=\u2009711), respectively. Fine-tuned LLM extracted patients with bone metastasis, demonstrating satisfactory performance that was comparable to or slightly lower than manual annotation by radiologists in a noticeably shorter time.<\/jats:p>","DOI":"10.1007\/s10278-024-01242-3","type":"journal-article","created":{"date-parts":[[2024,8,26]],"date-time":"2024-08-26T17:02:16Z","timestamp":1724691736000},"page":"865-872","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["The Fine-Tuned Large Language Model for Extracting the Progressive Bone Metastasis from Unstructured Radiology Reports"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0864-7154","authenticated-orcid":false,"given":"Noriko","family":"Kanemaru","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0324-6562","authenticated-orcid":false,"given":"Koichiro","family":"Yasaka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4314-5475","authenticated-orcid":false,"given":"Nana","family":"Fujita","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-8203-8665","authenticated-orcid":false,"given":"Jun","family":"Kanzawa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1180-2629","authenticated-orcid":false,"given":"Osamu","family":"Abe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,8,26]]},"reference":[{"key":"1242_CR1","doi-asserted-by":"crossref","unstructured":"Ibrahim T, Flamini E, Fabbri L, Serra P, Mercatali L, Ricci R, Sacanna E, Falasconi M.C, Casadei R, Galassi R, Giannini M, Bazzocchi O, Calzolari F, Nunziatini R, Gaudio M, Maltoni M, Amadori D. Multidisciplinary approach to the treatment of bone metastases: Osteo-Oncology Center, a new organizational model. Tumori. 95:291\u2013297, 2009","DOI":"10.1177\/030089160909500304"},{"key":"1242_CR2","doi-asserted-by":"crossref","unstructured":"Kimura T. Multidisciplinary Approach for Bone Metastasis: A Review. Cancers (Basel). 10:1\u201310, 2018","DOI":"10.3390\/cancers10060156"},{"key":"1242_CR3","doi-asserted-by":"crossref","unstructured":"Sloan CE, Chadalavada SC, Cook TS, Langlotz CP, Schnall MD, Zafar HM. Assessment of follow-up completeness and notification preferences for imaging findings of possible cancer: what happens after radiologists submit their reports? Acad Radiol. 21:1579\u20131586, 2014.","DOI":"10.1016\/j.acra.2014.07.006"},{"key":"1242_CR4","doi-asserted-by":"crossref","unstructured":"Callen JL, Westbrook JI, Georgiou A, Li J. Failure to follow-up test results for ambulatory patients: A systematic review. J Gen Intern Med. 27:1334\u20131348, 2012.","DOI":"10.1007\/s11606-011-1949-5"},{"key":"1242_CR5","unstructured":"Solberg A, Bremnes RM. Metastatic spinal cord compression: diagnostic delay, treatment, and outcome. Anticancer Res. 19:677\u2013684, 1999."},{"key":"1242_CR6","unstructured":"Loven D, G\u00f8rnish M, Fenig GE, Sulkes A, Rappaport Z, Klir I, Rotenberg Z, Gadoth N. [Malignant epidural cord compression]. Harefuah. 131:457\u2013462, 1996."},{"key":"1242_CR7","doi-asserted-by":"crossref","unstructured":"Nakamura Y, Hanaoka S, Nomura Y, Nakao T, Miki S, Watadani T, Yoshikawa T, Hayashi N, Abe O. Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers. BMC Med Inform Decis Mak. 21:262,2021","DOI":"10.1186\/s12911-021-01623-6"},{"key":"1242_CR8","doi-asserted-by":"publisher","unstructured":"Adams L C, Truhn D, Busch F, Kader A, Niehues S M, Makowski M R, Bressem K K. Leveraging GPT-4 for Post Hoc Transformation of Free-text Radiology Reports into Structured Reporting: A Multilingual Feasibility Study. Radiology. https:\/\/doi.org\/10.1148\/radiol.230725, Apr 4, 2023.","DOI":"10.1148\/radiol.230725"},{"key":"1242_CR9","doi-asserted-by":"publisher","unstructured":"Mukherjee P, Hou B, Lanfredi RB, Summers RM. Feasibility of Using the Privacy-preserving Large Language Model Vicuna for Labeling Radiology Reports. Radiology. https:\/\/doi.org\/10.1148\/radiol.231147, October 10,2023.","DOI":"10.1148\/radiol.231147"},{"key":"1242_CR10","doi-asserted-by":"publisher","unstructured":"Yasaka K, Kanzawa J, Kanemaru N, Koshino S, Abe O. Fine Tuned Large Language Model for Extracting Patients on Pretreatment for Lung Cancer from a Picture Archiving and Communication System Based on Radiological Reports. J Imaging Informatics Med. https:\/\/doi.org\/10.1007\/s10278-024-01186-8, July 2, 2024.","DOI":"10.1007\/s10278-024-01186-8"},{"key":"1242_CR11","unstructured":"Eghbali N, Siegal D, Klochko C, Ghassemi MM. Automation of Protocoling Advanced MSK Examinations Using Natural Language Processing Techniques. AMIA Jt Summits Transl Sci. 2023:118\u2013127, 2023."},{"key":"1242_CR12","doi-asserted-by":"crossref","unstructured":"Wong KA, Hatef A, Ryu JL, Nguyen X V., Makary MS, Prevedello LM. An Artificial Intelligence Tool for Clinical Decision Support and Protocol Selection for Brain MRI. Am J Neuroradiol. 44:11\u201316, 2023","DOI":"10.3174\/ajnr.A7736"},{"key":"1242_CR13","doi-asserted-by":"crossref","unstructured":"Talebi S, Tong E, Li A, Yamin G, Zaharchuk G, Mofrad MRK. Exploring the performance and explainability of fine-tuned BERT models for neuroradiology protocol assignment. BMC Med Inform Decis Mak. 24:1\u201312, 2024","DOI":"10.1186\/s12911-024-02444-z"},{"key":"1242_CR14","doi-asserted-by":"publisher","unstructured":"Kanzawa J, Yasaka K, Fujita N, Fujiwara S, Abe O. Automated classification of brain MRI reports using fine-tuned large language models. Neuroradiology. https:\/\/doi.org\/10.1007\/s00234-024-03427-7. July 12, 2024.","DOI":"10.1007\/s00234-024-03427-7"},{"key":"1242_CR15","doi-asserted-by":"crossref","unstructured":"Doi K, Takegawa H, Yui M, Anetai Y, Koike Y, Nakamura S, Tanigawa N, Koziumi M, Nishio T. Deep learning-based detection of patients with bone metastasis from Japanese radiology reports. Jpn J Radiol. 41:900\u2013908, 2023","DOI":"10.1007\/s11604-023-01413-2"},{"key":"1242_CR16","doi-asserted-by":"crossref","unstructured":"Do RKG, Lupton K, Causa Andrieu PI, Luthra A, Taya M, Batch K, Nguyen H, Rahurkar P, Gazit L, Nicholas K, Fong CJ, Gangai N, Schultz N, Zulkernine F, Sevilimedu V, Juluru K, Simpson A, Hricak H. Patterns of Metastatic Disease in Patients with Cancer Derived from Natural Language Processing of Structured CT Radiology Reports over a 10-year Period. Radiology. 301:115\u2013122, 2021.","DOI":"10.1148\/radiol.2021210043"},{"key":"1242_CR17","doi-asserted-by":"crossref","unstructured":"Kehl KL, Elmarakeby H, Nishino M, Van Allen EM, Lepisto EM, Hassett MJ, Johnson BE, Schrag D. Assessment of Deep Natural Language Processing in Ascertaining Oncologic Outcomes From Radiology Reports. JAMA Oncol. 5:1421\u20131429, 2019.","DOI":"10.1001\/jamaoncol.2019.1800"},{"key":"1242_CR18","doi-asserted-by":"crossref","unstructured":"Landis JR, Koch GG. Agreement of categorical data. Biometrics. 33:159\u2013174, 1977","DOI":"10.2307\/2529310"},{"key":"1242_CR19","doi-asserted-by":"publisher","unstructured":"Cai W. Feasibility and Prospect of Privacy-preserving Large Language Models in Radiology. Radiology. https:\/\/doi.org\/10.1148\/radiol.232335, October10, 2023.","DOI":"10.1148\/radiol.232335"},{"key":"1242_CR20","doi-asserted-by":"crossref","unstructured":"Xavier BA, Chen PH. Natural Language Processing for Imaging Protocol Assignment: Machine Learning for Multiclass Classification of Abdominal CT Protocols Using Indication Text Data. J Digit Imaging. 35:1120\u20131130, 2022.","DOI":"10.1007\/s10278-022-00633-8"},{"key":"1242_CR21","doi-asserted-by":"crossref","unstructured":"Yasaka K, Akai H, Abe O, Kiryu S. Deep learning with CNN showed high diagnostic performance in differentiation of liver masses at dynamic CT. Radiology. 286:887-896, 2018.","DOI":"10.1148\/radiol.2017170706"},{"key":"1242_CR22","doi-asserted-by":"publisher","unstructured":"Wei J, Zou K. EDA: Easy data augmentation techniques for boosting performance on text classification tasks. EMNLP-IJCNLP 2019, https:\/\/doi.org\/10.18653\/v1\/d19-1670, 2019.","DOI":"10.18653\/v1\/d19-1670"},{"key":"1242_CR23","doi-asserted-by":"publisher","unstructured":"Sennrich R, Haddow B, Birch A. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, https:\/\/doi.org\/10.18653\/v1\/p16-1009, August 2018.","DOI":"10.18653\/v1\/p16-1009"},{"key":"1242_CR24","doi-asserted-by":"publisher","unstructured":"Fadaee M, Bisazza A, Monz C. Data augmentation for low-Resource neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, https:\/\/doi.org\/10.18653\/v1\/P17-2090, July 2017.","DOI":"10.18653\/v1\/P17-2090"},{"key":"1242_CR25","doi-asserted-by":"crossref","unstructured":"Olthof AW, van Ooijen PMA, Cornelissen LJ. Deep Learning-Based Natural Language Processing in Radiology: The Impact of Report Complexity, Disease Prevalence, Dataset Size, and Algorithm Type on Model Performance. J Med Syst. 45(91), 2021.","DOI":"10.1007\/s10916-021-01761-4"},{"key":"1242_CR26","doi-asserted-by":"crossref","unstructured":"Mirels H. Metastatic disease in long bones. A proposed scoring system for diagnosing impending pathologic fractures. Clin Orthop Relat Res. 249:256\u2013264. 1989.","DOI":"10.1097\/00003086-198912000-00027"},{"key":"1242_CR27","doi-asserted-by":"crossref","unstructured":"Van der Linden YM, Dijkstra PDS, Kroon HM, Lok JJ, Noordijk EM, Leer JW, Marijnen CA. Comparative analysis of risk factors for pathological fracture with femoral metastases. J Bone Joint Surg Br. 86:566\u2013573, 2004.","DOI":"10.1302\/0301-620X.86B4.14703"}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01242-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-024-01242-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01242-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T14:15:58Z","timestamp":1743344158000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-024-01242-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,26]]},"references-count":27,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["1242"],"URL":"https:\/\/doi.org\/10.1007\/s10278-024-01242-3","relation":{},"ISSN":["2948-2933"],"issn-type":[{"value":"2948-2933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,26]]},"assertion":[{"value":"24 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 August 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 August 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 August 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Research Ethics Committee of The University of Tokyo (Ethics approval number: 2561-(22)), and informed consent was waived due to retrospective nature.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}