{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T21:45:20Z","timestamp":1775252720329,"version":"3.50.1"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T00:00:00Z","timestamp":1737504000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T00:00:00Z","timestamp":1737504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["JP23K19977"],"award-info":[{"award-number":["JP23K19977"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Missed critical imaging findings, particularly those indicating cancer, are a common issue that can result in delays in patient follow-up and treatment. To address this, we developed a rule-based natural language processing (NLP) algorithm to detect cancer-suspicious findings from Japanese radiology reports. The dataset used consisted of chest and abdomen CT reports from six institutions. Reports from our institution were used for algorithm development and internal evaluation, while reports from the other five institutions were used for external evaluation. To create the gold standard, reports were annotated by two experienced physicians. Data were statistically analyzed using precision, recall and F1 score with 1000 bootstrap iterations. BERT was used as a baseline deep learning model, and its performance was compared with the proposed rule-based method. At the report level of detection, the overall precision, recall, and F-1 score were 0.886, 0.886, and 0.883, respectively, for the rule-based algorithm, which were higher than those of the deep learning algorithm (0.851, 0.679, and 0.733). The overall results include both internal and external validation data. For the internal validation set, the precision, recall, and F-1 score were 0.929, 0.929, and 0.927, respectively. For the external validation set, the precision, recall, and F-1 score were 0.875, 0.879, and 0.873, demonstrating generalizability. In conclusion, we show the rule-based NLP algorithm exhibited a high performance in detecting cancer-suspicious findings from multi-institutional CT reports.<\/jats:p>","DOI":"10.1007\/s10278-024-01338-w","type":"journal-article","created":{"date-parts":[[2025,1,22]],"date-time":"2025-01-22T12:42:13Z","timestamp":1737549733000},"page":"3375-3385","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Automated Detection of Cancer-Suspicious Findings in Japanese Radiology Reports with Natural Language Processing: A Multicenter Study"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6874-2399","authenticated-orcid":false,"given":"Kento","family":"Sugimoto","sequence":"first","affiliation":[]},{"given":"Shoya","family":"Wada","sequence":"additional","affiliation":[]},{"given":"Shozo","family":"Konishi","sequence":"additional","affiliation":[]},{"given":"Junya","family":"Sato","sequence":"additional","affiliation":[]},{"given":"Katsuki","family":"Okada","sequence":"additional","affiliation":[]},{"given":"Shoji","family":"Kido","sequence":"additional","affiliation":[]},{"given":"Noriyuki","family":"Tomiyama","sequence":"additional","affiliation":[]},{"given":"Yasushi","family":"Matsumura","sequence":"additional","affiliation":[]},{"given":"Toshihiro","family":"Takeda","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,1,22]]},"reference":[{"issue":"4","key":"1338_CR1","doi-asserted-by":"publisher","first-page":"459","DOI":"10.1197\/jamia.M2280","volume":"14","author":"H Singh","year":"2007","unstructured":"Singh H, Arora HS, Vij MS, et al. Communication outcomes of critical imaging results in a computerized notification system. J Am Med Inform Assoc. 2007;14(4):459\u201366.","journal-title":"J Am Med Inform Assoc."},{"issue":"20","key":"1338_CR2","doi-asserted-by":"publisher","first-page":"1619","DOI":"10.1001\/jama.1994.03520200075039","volume":"272","author":"W Levinson","year":"1994","unstructured":"Levinson W. Physician-patient communication: A key to malpractice prevention. JAMA. 1994;272(20):1619.","journal-title":"JAMA."},{"issue":"2","key":"1338_CR3","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.jacr.2009.11.007","volume":"7","author":"LL Lucey","year":"2010","unstructured":"Lucey LL, Kushner DC. The ACR Guideline on Communication: To Be or Not to Be, That Is the Question. J Am Coll Radiol. 2010;7(2):109\u201314.","journal-title":"J Am Coll Radiol."},{"key":"1338_CR4","doi-asserted-by":"crossref","unstructured":"Berlin L. Failure of radiologic communication: An increasing cause of malpractice litigation and harm to patients. Appl Radiol. 2010;17\u201323.","DOI":"10.37549\/AR1734"},{"issue":"6","key":"1338_CR5","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1016\/j.jacr.2013.12.016","volume":"11","author":"PA Larson","year":"2014","unstructured":"Larson PA, Berland LL, Griffith B, Kahn CE Jr, Liebscher LA. Actionable Findings and the Role of IT Support: Report of the ACR Actionable Reporting Work Group. J Am Coll Radiol. 2014;11(6):552\u20138.","journal-title":"J Am Coll Radiol."},{"issue":"3","key":"1338_CR6","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1148\/radiol.2531082217","volume":"252","author":"HH Abujudeh","year":"2009","unstructured":"Abujudeh HH, Kaewlai R, Choy G, Whelton DG, Rosenthal DI. Important imaging finding e-mail alert system: experience after 3 years of implementation. Radiology. 2009;252(3):747\u201353.","journal-title":"Radiology."},{"issue":"9","key":"1338_CR7","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1016\/j.jacr.2015.04.005","volume":"12","author":"HM Zafar","year":"2015","unstructured":"Zafar HM, Chadalavada SC, Kahn CE Jr, et al. Code Abdomen: An Assessment Coding Scheme for Abdominal Imaging Findings Possibly Representing Cancer. J Am Coll Radiol. 2015;12(9):947\u201350.","journal-title":"J Am Coll Radiol."},{"issue":"2","key":"1338_CR8","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/j.annemergmed.2013.02.001","volume":"62","author":"S Dutta","year":"2013","unstructured":"Dutta S, Long WJ, Brown DFM, Reisner AT. Automated detection using natural language processing of radiologists recommendations for additional imaging of incidental findings. Ann Emerg Med. 2013;62(2):162\u20139.","journal-title":"Ann Emerg Med."},{"key":"1338_CR9","first-page":"1090","volume":"245","author":"T Mabotuwana","year":"2017","unstructured":"Mabotuwana T, Hall CS, Dalal S, Tieder J, Gunn ML. Extracting Follow-Up Recommendations and Associated Anatomy from Radiology Reports. Stud Health Technol Inform. 2017;245:1090\u20134.","journal-title":"Stud Health Technol Inform."},{"issue":"11","key":"1338_CR10","doi-asserted-by":"publisher","first-page":"1587","DOI":"10.1016\/j.jacr.2019.04.026","volume":"16","author":"SK Kang","year":"2019","unstructured":"Kang SK, Garry K, Chung R, et al. Natural Language Processing for Identification of Incidental Pulmonary Nodules in Radiology Reports. J Am Coll Radiol. 2019;16(11):1587\u201394.","journal-title":"J Am Coll Radiol."},{"issue":"2","key":"1338_CR11","doi-asserted-by":"publisher","first-page":"313","DOI":"10.2214\/AJR.07.3508","volume":"191","author":"PA Dang","year":"2008","unstructured":"Dang PA, Kalra MK, Blake MA, et al. Extraction of recommendation features in radiology with natural language processing: exploratory study. AJR Am J Roentgenol. 2008;191(2):313\u201320.","journal-title":"AJR Am J Roentgenol."},{"issue":"1","key":"1338_CR12","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1007\/s10278-019-00271-7","volume":"33","author":"R Lou","year":"2020","unstructured":"Lou R, Lalevic D, Chambers C, Zafar HM, Cook TS. Automated Detection of Radiology Reports that Require Follow-up Imaging Using Natural Language Processing Feature Engineering and Machine Learning Classification. J Digit Imaging. 2020;33(1):131\u20136.","journal-title":"J Digit Imaging."},{"issue":"2","key":"1338_CR13","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1016\/j.jbi.2012.12.005","volume":"46","author":"M Yetisgen-Yildiz","year":"2013","unstructured":"Yetisgen-Yildiz M, Gunn ML, Xia F, Payne TH. A text processing pipeline to extract recommendations from radiology reports. J Biomed Inform. 2013;46(2):354\u201362.","journal-title":"J Biomed Inform."},{"issue":"1","key":"1338_CR14","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1007\/s10278-022-00712-w","volume":"36","author":"I Banerjee","year":"2023","unstructured":"Banerjee I, Davis MA, Vey BL, et al. Natural Language Processing Model for Identifying Critical Findings-A Multi-Institutional Study. J Digit Imaging. 2023;36(1):105\u201313.","journal-title":"J Digit Imaging."},{"issue":"1","key":"1338_CR15","doi-asserted-by":"publisher","first-page":"262","DOI":"10.1186\/s12911-021-01623-6","volume":"21","author":"Y Nakamura","year":"2021","unstructured":"Nakamura Y, Hanaoka S, Nomura Y, et al. Automatic detection of actionable radiology reports using bidirectional encoder representations from transformers. BMC Med Inform Decis Mak. 2021;21(1):262.","journal-title":"BMC Med Inform Decis Mak."},{"key":"1338_CR16","doi-asserted-by":"publisher","first-page":"103729","DOI":"10.1016\/j.jbi.2021.103729","volume":"116","author":"K Sugimoto","year":"2021","unstructured":"Sugimoto K, Takeda T, Oh J-H, et al. Extracting clinical terms from radiology reports with deep learning. J Biomed Inform. 2021;116:103729.","journal-title":"J Biomed Inform."},{"key":"1338_CR17","doi-asserted-by":"publisher","first-page":"e49041","DOI":"10.2196\/49041","volume":"11","author":"K Sugimoto","year":"2023","unstructured":"Sugimoto K, Wada S, Konishi S, et al. Extracting Clinical Information From Japanese Radiology Reports Using a 2-Stage Deep Learning Approach: Algorithm Development and Validation. JMIR Med Inform. 2023;11:e49041.","journal-title":"JMIR Med Inform."},{"key":"1338_CR18","first-page":"569","volume":"310","author":"K Sugimoto","year":"2024","unstructured":"Sugimoto K, Wada S, Konishi S, et al. Classification of diagnostic certainty in radiology reports with deep learning. Stud Health Technol Inform. 2024;310:569\u201373.","journal-title":"Stud Health Technol Inform."},{"key":"1338_CR19","doi-asserted-by":"crossref","unstructured":"N\u00e9v\u00e9ol A, Dalianis H, Velupillai S, Savova G, Zweigenbaum P. Clinical Natural Language Processing in languages other than English: opportunities and challenges. J Biomed Semantics. 2018;9(1).","DOI":"10.1186\/s13326-018-0179-8"},{"key":"1338_CR20","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"J Devlin","year":"2019","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Minneapolis, Minnesota: Association for Computational Linguistics; 2019. p. 4171\u201386."},{"issue":"11","key":"1338_CR21","doi-asserted-by":"publisher","first-page":"e0259763","DOI":"10.1371\/journal.pone.0259763","volume":"16","author":"Y Kawazoe","year":"2021","unstructured":"Kawazoe Y, Shibata D, Shinohara E, Aramaki E, Ohe K. A clinical specific BERT developed using a huge Japanese clinical text corpus. PLoS One. 2021;16(11):e0259763.","journal-title":"PLoS One."},{"issue":"1","key":"1338_CR22","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","volume":"20","author":"J Cohen","year":"1960","unstructured":"Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37\u201346.","journal-title":"Educ Psychol Meas."},{"issue":"1","key":"1338_CR23","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","volume":"33","author":"JR Landis","year":"1977","unstructured":"Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159\u201374.","journal-title":"Biometrics."},{"issue":"1149","key":"1338_CR24","doi-asserted-by":"publisher","first-page":"20220769","DOI":"10.1259\/bjr.20220769","volume":"96","author":"E Yang","year":"2023","unstructured":"Yang E, Li MD, Raghavan S, et al. Transformer versus traditional natural language processing: how much data is enough for automated radiology report classification? Br J Radiol. 2023;96(1149):20220769.","journal-title":"Br J Radiol."},{"issue":"2","key":"1338_CR25","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1197\/jamia.M1809","volume":"13","author":"H van der Sijs","year":"2006","unstructured":"van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138\u201347.","journal-title":"J Am Med Inform Assoc."},{"issue":"12","key":"1338_CR26","doi-asserted-by":"publisher","first-page":"2310","DOI":"10.1377\/hlthaff.2010.1111","volume":"30","author":"AS Kesselheim","year":"2011","unstructured":"Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical Decision Support Systems Could Be Modified To Reduce \u2018Alert Fatigue\u2019 While Still Minimizing The Risk Of Litigation. Health Affairs. 2011;30(12):2310\u20137.","journal-title":"Health Affairs."},{"key":"1338_CR27","doi-asserted-by":"crossref","unstructured":"Ancker JS, Edwards A, Nosal S, et al. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017;17(1).","DOI":"10.1186\/s12911-017-0430-8"}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01338-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-024-01338-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-024-01338-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T22:48:17Z","timestamp":1761778097000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-024-01338-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,22]]},"references-count":27,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["1338"],"URL":"https:\/\/doi.org\/10.1007\/s10278-024-01338-w","relation":{},"ISSN":["2948-2933"],"issn-type":[{"value":"2948-2933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,22]]},"assertion":[{"value":"16 September 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 November 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 November 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 January 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was approved by our institutional review board (Permission number: 24017). Informed consent was waived because of the retrospective nature of the study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}