{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T02:57:54Z","timestamp":1792897074366,"version":"4.5.0"},"reference-count":22,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003625","name":"Ministry of Health and Welfare","doi-asserted-by":"publisher","award":["HI21C1161"],"award-info":[{"award-number":["HI21C1161"]}],"id":[{"id":"10.13039\/501100003625","id-type":"DOI","asserted-by":"publisher"}]},{"name":"RINK-CR","award":["RINK-CR"],"award-info":[{"award-number":["RINK-CR"]}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["RS-2023-00208227"],"award-info":[{"award-number":["RS-2023-00208227"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Cancer Imaging"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>To assess whether a deep learning-based system (DLS) with black-blood imaging for brain metastasis (BM) improves the diagnostic workflow in a multi-center setting.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Materials and methods<\/jats:title>\n                <jats:p>In this retrospective study, a DLS was developed in 101 patients and validated on 264 consecutive patients (with lung cancer) having newly developed BM from two tertiary university hospitals, which performed black-blood imaging between January 2020 and April 2021. Four neuroradiologists independently evaluated BM either with segmented masks and BM counts provided (with DLS) or not provided (without DLS) on a clinical trial imaging management system (CTIMS). To assess reading reproducibility, BM count agreement between the readers and the reference standard were calculated using limits of agreement (LoA). Readers\u2019 workload was assessed with reading time, which was automatically measured on CTIMS, and were compared between with and without DLS using linear mixed models considering the imaging center.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>In the validation cohort, the detection sensitivity and positive predictive value of the DLS were 90.2% (95% confidence interval [CI]: 88.1\u201392.2) and 88.2% (95% CI: 85.7\u201390.4), respectively. The difference between the readers and the reference counts was larger without DLS (LoA: \u22120.281, 95% CI: \u22122.888, 2.325) than with DLS (LoA: \u22120.163, 95% CI: \u22122.692, 2.367). The reading time was reduced from mean 66.9\u00a0s (interquartile range: 43.2\u201390.6) to 57.3\u00a0s (interquartile range: 33.6\u201381.0) (<jats:italic>P<\/jats:italic>\u2009&lt;.001) in the with DLS group, regardless of the imaging center.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusion<\/jats:title>\n                <jats:p>Deep learning-based BM detection and counting with black-blood imaging improved reproducibility and reduced reading time, on multi-center validation.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s40644-024-00669-9","type":"journal-article","created":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T09:02:23Z","timestamp":1709283743000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Deep learning-based metastasis detection in patients with lung cancer to enhance reproducibility and reduce workload in brain metastasis screening with MRI: a multi-center study"],"prefix":"10.1186","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8907-5401","authenticated-orcid":false,"given":"Yae Won","family":"Park","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4419-4682","authenticated-orcid":false,"given":"Ji Eun","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0503-5558","authenticated-orcid":false,"given":"Sung Soo","family":"Ahn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5687-7237","authenticated-orcid":false,"given":"Kyunghwa","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8756-8587","authenticated-orcid":false,"given":"NakYoung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joo Young","family":"Oh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7593-0403","authenticated-orcid":false,"given":"Da Hyun","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0570-3365","authenticated-orcid":false,"given":"So Yeon","family":"Won","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2046-9426","authenticated-orcid":false,"given":"Ilah","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9477-7421","authenticated-orcid":false,"given":"Ho Sung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5646-4072","authenticated-orcid":false,"given":"Seung-Koo","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,1]]},"reference":[{"key":"669_CR1","doi-asserted-by":"publisher","first-page":"1332","DOI":"10.1016\/j.annonc.2021.07.016","volume":"32","author":"E Le Rhun","year":"2021","unstructured":"Le Rhun E, et al. EANO-ESMO clinical practice guidelines for diagnosis, treatment and follow-up of patients with brain metastasis from solid tumours. Ann Oncol. 2021;32:1332\u201347. https:\/\/doi.org\/10.1016\/j.annonc.2021.07.016.","journal-title":"Ann Oncol"},{"key":"669_CR2","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1093\/neuonc\/noaa030","volume":"22","author":"TJ Kaufmann","year":"2020","unstructured":"Kaufmann TJ, et al. Consensus recommendations for a standardized brain tumor imaging protocol for clinical trials in brain metastases. Neurooncology. 2020;22:757\u201372. https:\/\/doi.org\/10.1093\/neuonc\/noaa030.","journal-title":"Neurooncology"},{"key":"669_CR3","doi-asserted-by":"publisher","first-page":"1469","DOI":"10.3174\/ajnr.A6158","volume":"40","author":"D Kim","year":"2019","unstructured":"Kim D, et al. Usefulness of the Delay alternating with nutation for tailored excitation pulse with T1-Weighted sampling perfection with application-optimized contrasts using different flip Angle Evolution in the detection of cerebral metastases: comparison with MPRAGE Imaging. AJNR Am J Neuroradiol. 2019;40:1469\u201375. https:\/\/doi.org\/10.3174\/ajnr.A6158.","journal-title":"AJNR Am J Neuroradiol"},{"key":"669_CR4","doi-asserted-by":"publisher","first-page":"e0178265","DOI":"10.1371\/journal.pone.0178265","volume":"12","author":"L Sunwoo","year":"2017","unstructured":"Sunwoo L, et al. Computer-aided detection of brain metastasis on 3D MR imaging: Observer performance study. PLoS ONE. 2017;12:e0178265. https:\/\/doi.org\/10.1371\/journal.pone.0178265.","journal-title":"PLoS ONE"},{"key":"669_CR5","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1093\/neuonc\/noaa232","volume":"23","author":"SJ Cho","year":"2021","unstructured":"Cho SJ, et al. Brain metastasis detection using machine learning: a systematic review and meta-analysis. Neurooncology. 2021;23:214\u201325. https:\/\/doi.org\/10.1093\/neuonc\/noaa232.","journal-title":"Neurooncology"},{"key":"669_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-021-07783-3","author":"YW Park","year":"2021","unstructured":"Park YW, et al. Robust performance of deep learning for automatic detection and segmentation of brain metastases using three-dimensional black-blood and three-dimensional gradient echo imaging. Eur Radiol. 2021. https:\/\/doi.org\/10.1007\/s00330-021-07783-3.","journal-title":"Eur Radiol"},{"key":"669_CR7","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1002\/jmri.26766","volume":"51","author":"E Gr\u00f8vik","year":"2020","unstructured":"Gr\u00f8vik E, et al. Deep learning enables automatic detection and segmentation of brain metastases on multisequence MRI. J Magn Reson Imaging: JMRI. 2020;51:175\u201382. https:\/\/doi.org\/10.1002\/jmri.26766.","journal-title":"J Magn Reson Imaging: JMRI"},{"key":"669_CR8","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1148\/radiol.2020191479","volume":"295","author":"Z Zhou","year":"2020","unstructured":"Zhou Z, et al. Computer-aided detection of Brain metastases in T1-weighted MRI for stereotactic Radiosurgery using deep learning single-shot detectors. Radiology. 2020;295:407\u201315. https:\/\/doi.org\/10.1148\/radiol.2020191479.","journal-title":"Radiology"},{"key":"669_CR9","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1093\/neuonc\/noz234","volume":"22","author":"J Xue","year":"2020","unstructured":"Xue J, et al. Deep learning-based detection and segmentation-assisted management of brain metastases. Neurooncology. 2020;22:505\u201314. https:\/\/doi.org\/10.1093\/neuonc\/noz234.","journal-title":"Neurooncology"},{"key":"669_CR10","doi-asserted-by":"publisher","first-page":"e14310","DOI":"10.2196\/14310","volume":"7","author":"Y Shin","year":"2019","unstructured":"Shin Y, et al. A good practice\u2013compliant clinical trial Imaging Management System for Multicenter clinical trials: Development and Validation Study. JMIR Med Inf. 2019;7:e14310. https:\/\/doi.org\/10.2196\/14310.","journal-title":"JMIR Med Inf"},{"key":"669_CR11","doi-asserted-by":"publisher","first-page":"4952","DOI":"10.1002\/hbm.24750","volume":"40","author":"F Isensee","year":"2019","unstructured":"Isensee F, et al. Automated brain extraction of multisequence MRI using artificial neural networks. J Time Ser Anal. 2019;40:4952\u201364. https:\/\/doi.org\/10.1002\/hbm.24750.","journal-title":"J Time Ser Anal"},{"key":"669_CR12","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1038\/s41592-020-01008-z","volume":"18","author":"F Isensee","year":"2021","unstructured":"Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods. 2021;18:203\u201311. https:\/\/doi.org\/10.1038\/s41592-020-01008-z.","journal-title":"Nat Methods"},{"key":"669_CR13","doi-asserted-by":"publisher","unstructured":"Nolden M, et al. The Medical Imaging Interaction Toolkit: challenges and advances: 10 years of open-source development. Int J Comput Assist Radiol Surg. 2013;8:607\u201320. https:\/\/doi.org\/10.1007\/s11548-013-0840-8.","DOI":"10.1007\/s11548-013-0840-8"},{"key":"669_CR14","doi-asserted-by":"publisher","first-page":"1630","DOI":"10.1136\/bmj.f1630","volume":"346","author":"P Sedgwick","year":"2013","unstructured":"Sedgwick P. Limits of agreement (bland-Altman method). BMJ. 2013;346:1630. https:\/\/doi.org\/10.1136\/bmj.f1630.","journal-title":"BMJ"},{"key":"669_CR15","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1177\/096228029900800204","volume":"8","author":"JM Bland","year":"1999","unstructured":"Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135\u201360. https:\/\/doi.org\/10.1177\/096228029900800204.","journal-title":"Stat Methods Med Res"},{"key":"669_CR16","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/noac189","author":"P Vollmuth","year":"2022","unstructured":"Vollmuth P, et al. AI-based decision support improves reproducibility of tumor response assessment in neuro-oncology: an international multi-reader study. Neurooncology. 2022. https:\/\/doi.org\/10.1093\/neuonc\/noac189.","journal-title":"Neurooncology"},{"key":"669_CR17","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/noac025","author":"S Yin","year":"2022","unstructured":"Yin S, et al. Development and validation of a deep-learning model for detecting brain metastases on 3D post-contrast MRI: a multi-center multi-reader evaluation study. Neurooncology. 2022. https:\/\/doi.org\/10.1093\/neuonc\/noac025.","journal-title":"Neurooncology"},{"key":"669_CR18","doi-asserted-by":"publisher","unstructured":"Charron O, et al. Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network. Comput Biol Med. 2018;95:43\u201354. https:\/\/doi.org\/10.1016\/j.compbiomed.2018.02.004.","DOI":"10.1016\/j.compbiomed.2018.02.004"},{"key":"669_CR19","doi-asserted-by":"publisher","DOI":"10.1002\/jmri.28274","author":"G Chartrand","year":"2022","unstructured":"Chartrand G, et al. Automated detection of Brain metastases on T1-Weighted MRI using a convolutional neural network: impact of volume aware loss and Sampling Strategy. J Magn Reson Imaging: JMRI. 2022. https:\/\/doi.org\/10.1002\/jmri.28274.","journal-title":"J Magn Reson Imaging: JMRI"},{"key":"669_CR20","doi-asserted-by":"publisher","first-page":"1608","DOI":"10.1002\/jmri.27741","volume":"54","author":"ST J\u00fcnger","year":"2021","unstructured":"J\u00fcnger ST, et al. Fully automated MR Detection and Segmentation of Brain metastases in Non-small Cell Lung Cancer using deep learning. J Magn Reson Imaging: JMRI. 2021;54:1608\u201322. https:\/\/doi.org\/10.1002\/jmri.27741.","journal-title":"J Magn Reson Imaging: JMRI"},{"key":"669_CR21","doi-asserted-by":"publisher","unstructured":"Suh CH, Jung SC, Kim KW, Pyo J. The detectability of brain metastases using contrast-enhanced spin-echo or gradient-echo images: a systematic review and meta-analysis. Journal of neuro-oncology 129, 363\u2013371 (2016). https:\/\/doi.org\/10.1007\/s11060-016-2185-y.","DOI":"10.1007\/s11060-016-2185-y"},{"key":"669_CR22","doi-asserted-by":"publisher","unstructured":"Cagney DN et al. Incidence and prognosis of patients with brain metastases at diagnosis of systemic malignancy: a population-based study. Neuro-oncology 19, 1511\u20131521 (2017). https:\/\/doi.org\/10.1093\/neuonc\/nox077.","DOI":"10.1093\/neuonc\/nox077"}],"updated-by":[{"DOI":"10.1186\/s40644-024-00688-6","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T00:00:00Z","timestamp":1710979200000}}],"container-title":["Cancer Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40644-024-00669-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40644-024-00669-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40644-024-00669-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T11:09:38Z","timestamp":1712920178000},"score":1,"resource":{"primary":{"URL":"https:\/\/cancerimagingjournal.biomedcentral.com\/articles\/10.1186\/s40644-024-00669-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,1]]},"references-count":22,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["669"],"URL":"https:\/\/doi.org\/10.1186\/s40644-024-00669-9","relation":{"correction":[{"id-type":"doi","id":"10.1186\/s40644-024-00688-6","asserted-by":"object"}]},"ISSN":["1470-7330"],"issn-type":[{"value":"1470-7330","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,1]]},"assertion":[{"value":"25 October 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 March 2024","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1186\/s40644-024-00688-6","URL":"https:\/\/doi.org\/10.1186\/s40644-024-00688-6","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declaration"}},{"value":"This multi-center retrospective study was approved by the institutional review boards of the participating institutions. Requirement for patient consent was waived owing to the retrospective study design.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The author from the medical industry (N.K., technical director) provided technical support of this work by providing a deep learning-based detection algorithm. The algorithm is not a product or service from the company and there was no conflict of interest.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"32"}}