{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,12]],"date-time":"2026-09-12T17:01:28Z","timestamp":1789232488208,"version":"build-2803163510"},"reference-count":50,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T00:00:00Z","timestamp":1635292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF-2015R1C1A1A02037475, NRF-2018R1C1B6007917"],"award-info":[{"award-number":["NRF-2015R1C1A1A02037475, NRF-2018R1C1B6007917"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["IITP-2021-2017-0-01630"],"award-info":[{"award-number":["IITP-2021-2017-0-01630"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oncol."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Although accurate treatment response assessment for brain metastases (BMs) is crucial, it is highly labor intensive. This retrospective study aimed to develop a computer-aided detection (CAD) system for automated BM detection and treatment response evaluation using deep learning.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We included 214 consecutive MRI examinations of 147 patients with BM obtained between January 2015 and August 2016. These were divided into the training (174 MR images from 127 patients) and test datasets according to temporal separation (temporal test set #1; 40 MR images from 20 patients). For external validation, 24 patients with BM and 11 patients without BM from other institutions were included (geographic test set). In addition, we included 12 MRIs from BM patients obtained between August 2017 and March 2020 (temporal test set #2). Detection sensitivity, dice similarity coefficient (DSC) for segmentation, and agreements in one-dimensional and volumetric Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) criteria between CAD and radiologists were assessed.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In the temporal test set #1, the sensitivity was 75.1% (95% confidence interval [CI]: 69.6%, 79.9%), mean DSC was 0.69 \u00b1 0.22, and false-positive (FP) rate per scan was 0.8 for BM \u2265 5 mm. Agreements in the RANO-BM criteria were moderate (<jats:bold>\u03ba<\/jats:bold>, 0.52) and substantial (<jats:bold>\u03ba<\/jats:bold>, 0.68) for one-dimensional and volumetric, respectively. In the geographic test set, sensitivity was 87.7% (95% CI: 77.2%, 94.5%), mean DSC was 0.68 \u00b1 0.20, and FP rate per scan was 1.9 for BM \u2265 5 mm. In the temporal test set #2, sensitivity was 94.7% (95% CI: 74.0%, 99.9%), mean DSC was 0.82 \u00b1 0.20, and FP per scan was 0.5 (6\/12) for BM \u2265 5 mm.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>Our CAD showed potential for automated treatment response assessment of BM \u2265 5 mm.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fonc.2021.739639","type":"journal-article","created":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T08:15:11Z","timestamp":1635322511000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Deep Learning-Based Computer-Aided Detection System for Automated Treatment Response Assessment of Brain Metastases on 3D MRI"],"prefix":"10.3389","volume":"11","author":[{"given":"Jungheum","family":"Cho","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Young Jae","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonard","family":"Sunwoo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gi Pyo","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Toan Quang","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Se Jin","family":"Cho","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sung Hyun","family":"Baik","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yun Jung","family":"Bae","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Byung Se","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cheolkyu","family":"Jung","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chul-Ho","family":"Sohn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jung-Ho","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chae-Yong","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kwang Gi","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jae Hyoung","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2021,10,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/nos152","article-title":"Toward Determining the Lifetime Occurrence of Metastatic Brain Tumors Estimated From 2007 United States Cancer Incidence Data","volume":"14","author":"Davis","year":"2012","journal-title":"Neuro Oncol"},{"key":"B2","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s11060-004-8093-6","article-title":"Brain Metastases: Epidemiology and Pathophysiology","volume":"75","author":"Gavrilovic","year":"2005","journal-title":"J Neurooncol"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1016\/s1470-2045(14)70061-0","article-title":"Stereotactic Radiosurgery for Patients With Multiple Brain Metastases (JLGK0901): A Multi-Institutional Prospective Observational Study","volume":"15","author":"Yamamoto","year":"2014","journal-title":"Lancet Oncol"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.3171\/2010.8.Gks10994","article-title":"Analysis of Radiosurgical Results in Patients With Brain Metastases According to the Number of Brain Lesions: Is Stereotactic Radiosurgery Effective for Multiple Brain Metastases","author":"Chang","year":"2010","journal-title":"J\u00a0Neurosurg"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/noaa030","article-title":"Consensus Recommendations for a Standardized Brain Tumor Imaging Protocol for Clinical Trials in Brain Metastases","volume":"22","author":"Kaufmann","year":"2020","journal-title":"Neuro Oncol"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1007\/s00330-007-0599-9","article-title":"Detection of Brain Metastasis at 3T: Comparison Among SE, IR-FSE and 3D-GRE Sequences","volume":"17","author":"Kakeda","year":"2007","journal-title":"Eur Radiol"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0178265","article-title":"Computer-Aided Detection of Brain Metastasis on 3D MR Imaging: Observer Performance Study","volume":"12","author":"Sunwoo","year":"2017","journal-title":"PloS One"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1002\/jmri.25207","article-title":"Brain Metastases Detection on MR by Means of Three-Dimensional Tumor-Appearance Template Matching","volume":"44","author":"P\u00e9rez-Ram\u00edrez","year":"2016","journal-title":"J Magn Reson Imaging"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1097\/RLI.0b013e318277f078","article-title":"Computer-Aided Detection of Metastatic Brain Tumors Using Magnetic Resonance Black-Blood Imaging","volume":"48","author":"Yang","year":"2013","journal-title":"Invest Radiol"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1016\/j.mri.2012.02.024","article-title":"An Approach for Computer-Aided Detection of Brain Metastases in Post-Gd T1-W MRI","volume":"30","author":"Farjam","year":"2012","journal-title":"Magn Reson Imaging"},{"key":"B11","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1002\/jmri.22009","article-title":"Computer-Aided Detection of Metastatic Brain Tumors Using Automated Three-Dimensional Template Matching","volume":"31","author":"Ambrosini","year":"2010","journal-title":"J Magn Reson Imaging"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2362040864","article-title":"Computer-Aided Detection With Screening Mammography in a University Hospital Setting","volume":"236","author":"Birdwell","year":"2005","journal-title":"Radiology"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2203001282","article-title":"Screening Mammography With Computer-Aided Detection: Prospective Study of 12,860 Patients in a Community Breast Center","volume":"220","author":"Freer","year":"2001","journal-title":"Radiology"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2017162725","article-title":"Added Value of Computer-Aided CT Image Features for Early Lung Cancer Diagnosis With Small Pulmonary Nodules: A Matched Case-Control Study","volume":"286","author":"Huang","year":"2018","journal-title":"Radiology"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2016150063","article-title":"Low-Dose CT Screening for Lung Cancer: Computer-Aided Detection of Missed Lung Cancers","volume":"281","author":"Liang","year":"2016","journal-title":"Radiology"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.10091890","article-title":"Effect of Computer-Aided Detection for CT Colonography in a Multireader, Multicase Trial","volume":"256","author":"Dachman","year":"2010","journal-title":"Radiology"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMp1702071","article-title":"Machine Learning and Prediction in Medicine - Beyond the Peak of Inflated Expectations","volume":"376","author":"Chen","year":"2017","journal-title":"N Engl J Med"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.7797","article-title":"Unintended Consequences of Machine Learning in Medicine","volume":"318","author":"Cabitza","year":"2017","journal-title":"JAMA"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961","article-title":"Mastering the Game of Go With Deep Neural Networks and Tree Search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1002\/jmri.26766","article-title":"Deep Learning Enables Automatic Detection and Segmentation of Brain Metastases on Multisequence MRI","volume":"51","author":"Gr\u00f8vik","year":"2020","journal-title":"J Magn Reson Imaging"},{"key":"B22","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.compbiomed.2018.02.004","article-title":"Automatic Detection and Segmentation of Brain Metastases on Multimodal MR Images With a Deep Convolutional Neural Network","volume":"95","author":"Charron","year":"2018","journal-title":"Comput Biol Med"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0185844","article-title":"A Deep Convolutional Neural Network-Based Automatic Delineation Strategy for Multiple Brain Metastases Stereotactic Radiosurgery","volume":"12","author":"Liu","year":"2017","journal-title":"PloS One"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2020191479","article-title":"Computer-Aided Detection of Brain Metastases in T1-Weighted MRI for Stereotactic Radiosurgery Using Deep Learning Single-Shot Detectors","volume":"295","author":"Zhou","year":"2020","journal-title":"Radiology"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-019-0447-x","article-title":"End-To-End Lung Cancer Screening With Three-Dimensional Deep Learning on Low-Dose Chest Computed Tomography","volume":"25","author":"Ardila","year":"2019","journal-title":"Nat Med"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1016\/s1470-2045(15)70057-4","article-title":"Response Assessment Criteria for Brain Metastases: Proposal From the RANO Group","volume":"16","author":"Lin","year":"2015","journal-title":"Lancet Oncol"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1007\/s00062-019-00785-1","article-title":"Effect of MRI-Based Semiautomatic Size-Assessment in Cerebral Metastases on the RANO-BM Classification","volume":"30","author":"Bauknecht","year":"2020","journal-title":"Clin Neuroradiol"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2020200029","article-title":"Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers","volume":"2","author":"Mongan","year":"2020","journal-title":"Radiol: Artif Intell"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2019192515","article-title":"Assessing Radiology Research on Artificial Intelligence: A Brief Guide for Authors, Reviewers, and Readers-From the Radiology Editorial Board","volume":"294","author":"Bluemke","year":"2020","journal-title":"Radiology"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.2017171920","article-title":"Methodologic Guide for Evaluating Clinical Performance and Effect of Artificial Intelligence Technology for Medical Diagnosis and Prediction","volume":"286","author":"Park","year":"2018","journal-title":"Radiology"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2020.590980","article-title":"Volumetric Regression in Brain Metastases After Stereotactic Radiotherapy: Time Course, Predictors, and Significance","volume":"10","author":"Oft","year":"2020","journal-title":"Front Oncol"},{"key":"B32","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24574-4_28","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","volume-title":"Medical Image Computing and Computer-Assisted Intervention \u2013 MICCAI 2015","author":"Ronneberger","year":"2015"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.243","article-title":"Densely Connected Convolutional Networks","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Huang","year":"2017"},{"key":"B34","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","article-title":"A Survey on Image Data Augmentation for Deep Learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J Big Data"},{"key":"B35","doi-asserted-by":"crossref","DOI":"10.1109\/ISBI45749.2020.9098557","article-title":"Learning a Loss Function for Segmentation: A Feasibility Study","volume-title":"2020 IEEE 17th International Symposium on Biomedical Imaging","author":"Moltz","year":"2020"},{"key":"B36","first-page":"1801.05912","article-title":"On the Influence of Dice Loss Function in Multi-Class Organ Segmentation of Abdominal CT Using 3D Fully Convolutional Networks","author":"Shen","year":"2018","journal-title":"arXiv"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-67558-9_28","article-title":"Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations","volume-title":"Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support","author":"Sudre","year":"2017"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46723-8_49","article-title":"3D U-Net: Learning Dense Volumetric Segmentation From Sparse Annotation","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"\u00c7i\u00e7ek","year":"2016"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1109\/tvcg.2012.310","article-title":"Registration of 3D Point Clouds and Meshes: A Survey From Rigid to Nonrigid","volume":"19","author":"Tam","year":"2013","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"B40","doi-asserted-by":"publisher","first-page":"R1","DOI":"10.1088\/0031-9155\/46\/3\/201","article-title":"Medical Image Registration","volume":"46","author":"Hill","year":"2001","journal-title":"Phys Med Biol"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/s1470-2045(19)30098-1","article-title":"Automated Quantitative Tumour Response Assessment of MRI in Neuro-Oncology With Artificial Neural Networks: A Multicentre, Retrospective Study","volume":"20","author":"Kickingereder","year":"2019","journal-title":"Lancet Oncol"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1158\/1078-0432.Ccr-12-2934","article-title":"Overview: Progression-Free Survival as an Endpoint in Clinical Trials With Solid Tumors","volume":"19","author":"Korn","year":"2013","journal-title":"Clin Cancer Res"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1038\/ncponc1204","article-title":"Response Criteria for Glioma","volume":"5","author":"Sorensen","year":"2008","journal-title":"Nat Clin Pract Oncol"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1200\/jco.2017.72.7511","article-title":"Response Assessment in Neuro-Oncology Clinical Trials","volume":"35","author":"Wen","year":"2017","journal-title":"J Clin Oncol"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/now311","article-title":"Comparison of 2D (RANO) and Volumetric Methods for Assessment of Recurrent Glioblastoma Treated with Bevacizumab-a Report from the BELOB Trial","volume":"19","author":"Gahrmann","year":"2017","journal-title":"Neuro Oncol"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1093\/neuonc\/noaa232","article-title":"Brain Metastasis Detection Using Machine Learning: A Systematic Review and Meta-Analysis","volume":"23","author":"Cho","year":"2021","journal-title":"Neuro Oncol"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2018.2858826","article-title":"Focal Loss for Dense Object Detection","volume":"42","author":"Lin","year":"2020","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B48","first-page":"1804.02767","article-title":"Yolov3: An Incremental Improvement","author":"Redmon","year":"2018","journal-title":"arXiv"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1023\/a:1006308808769","article-title":"Diagnostic Accuracy of MRI Compared to CCT in Patients With Brain Metastases","volume":"44","author":"Schellinger","year":"1999","journal-title":"J Neurooncol"},{"key":"B50","article-title":"Detection of Brain Metastases: Comparison of Contrast-Enhanced MR With Unenhanced MR and Enhanced CT","volume":"11","author":"Sze","year":"1990","journal-title":"AJNR Am J Neuroradiol"}],"container-title":["Frontiers in Oncology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2021.739639\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,13]],"date-time":"2023-01-13T22:39:00Z","timestamp":1673649540000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fonc.2021.739639\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,27]]},"references-count":50,"alternative-id":["10.3389\/fonc.2021.739639"],"URL":"https:\/\/doi.org\/10.3389\/fonc.2021.739639","relation":{},"ISSN":["2234-943X"],"issn-type":[{"value":"2234-943X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,27]]},"article-number":"739639"}}