{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T07:50:57Z","timestamp":1791186657027,"version":"4.1.0"},"reference-count":29,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T00:00:00Z","timestamp":1783468800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"name":"Neosoma Inc."}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,10,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Artificial intelligence (AI)-based models have shown initial promise in imaging brain metastasis; however, many lack validation against advanced imaging-informed datasets, precluding external validity and limiting widespread adoption. To overcome these limitations, we performed comprehensive performance testing against reference standard metrics and externally validated an AI algorithm.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>As part of its FDA-clearance process, performance testing of a previously developed U-Net-based AI model was conducted on a multi-institutional cohort with reference standard established via consensus review by 3 neuroradiologists. External validation was performed on patients imaged with dual sequences (augmented) as well as an open-access dataset (UCSF-BMSR). Evaluation metrics included sensitivity, false positive (FP) rate, positive predictive value (PPV), dice similarity coefficient (DSC), 95% Hausdorff distance, normalized surface distance, and qualitative physician assessment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In the FDA performance testing cohort, the AI algorithm achieved a sensitivity of 90.0% (95% CI: 87.0%-94.0%), DSC of 0.86 (95% CI: 0.83-0.89), and average FP rate of 0.57 lesions. In the augmented and open-access external validation cohorts, a sensitivity of 81.4% (95% CI: 73.7%-89.1%) and 85.2% (95% CI: 83.0%-87.4%) with an average number of 0.22 and 1.19 FP lesions and DSCs of 0.70 (95% CI: 0.66-0.73) and 0.78 (95% CI: 0.77-0.78) were calculated, respectively. In the augmented external validation cohort, 38.5% of contours were rated as requiring major revisions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This AI algorithm demonstrated promising performance via 3 unique datasets. However, given the notable rate of contour revisions, these findings support its clinical role not as an autonomous system but as a human-in-the-loop tool requiring physician oversight.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/neuonc\/noag152","type":"journal-article","created":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T11:43:27Z","timestamp":1783424607000},"page":"2449-2460","source":"Crossref","is-referenced-by-count":0,"title":["Comprehensive performance testing and external validation of an AI algorithm to detect and segment brain metastases"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6927-0402","authenticated-orcid":false,"given":"Rupesh","family":"Kotecha","sequence":"first","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6848-4289","authenticated-orcid":false,"given":"Eyub Y","family":"Akdemir","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7972-4057","authenticated-orcid":false,"given":"Naseem Ud","family":"Din","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5203-7989","authenticated-orcid":false,"given":"Sreenija","family":"Yarlagadda","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mauricio","family":"Reyes","sequence":"additional","affiliation":[{"name":"ARTORG Center for Biomedical Engineering, University of Bern , Bern,","place":["Switzerland"]},{"name":"Department of Radiation Oncology, Inselspital, Bern University and University of Bern , Bern,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-1204-5043","authenticated-orcid":false,"given":"Francesco","family":"Nitti","sequence":"additional","affiliation":[{"name":"ARTORG Center for Biomedical Engineering, University of Bern , Bern,","place":["Switzerland"]},{"name":"Department of Radiation Oncology, Inselspital, Bern University and University of Bern , Bern,","place":["Switzerland"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1572-9011","authenticated-orcid":false,"given":"Alonso N","family":"Gutierrez","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6552-3564","authenticated-orcid":false,"given":"D Jay M","family":"Wieczorek","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongsook C","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6755-8397","authenticated-orcid":false,"given":"Ranjini","family":"Tolakanahalli","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0381-1995","authenticated-orcid":false,"given":"Evan D","family":"Bander","sequence":"additional","affiliation":[{"name":"Division of Neurosurgery, Miami Neuroscience Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Neuroscience, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0682-6853","authenticated-orcid":false,"given":"Michael W","family":"McDermott","sequence":"additional","affiliation":[{"name":"Division of Neurosurgery, Miami Neuroscience Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Neuroscience, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3794-9511","authenticated-orcid":false,"given":"Manmeet S","family":"Ahluwalia","sequence":"additional","affiliation":[{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]},{"name":"Department of Medical Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4812-5713","authenticated-orcid":false,"given":"Minesh P","family":"Mehta","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida , Miami, Florida,","place":["USA"]},{"name":"Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University , Miami, Florida,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,8]]},"reference":[{"key":"2026100502520127500_noag152-B1","doi-asserted-by":"publisher","first-page":"S33","DOI":"10.1016\/j.ijrobp.2025.06.1010","article-title":"Multicenter, phase 3 randomized trial of stereotactic radiation vs. hippocampal avoidance whole brain radiation in patients with 5-20 brain metastases: recurrence, salvage therapy, and mortality endpoints","volume":"123","author":"Aizer","year":"2025","journal-title":"Int J Radiat Oncol Biol Phys"},{"key":"2026100502520127500_noag152-B2","doi-asserted-by":"publisher","first-page":"1127","DOI":"10.1001\/jama.2026.0076","article-title":"Treatment for brain metastases with stereotactic radiation vs hippocampal-avoidance whole brain radiation: a randomized clinical trial","volume":"335","author":"Aizer","year":"2026","journal-title":"JAMA."},{"key":"2026100502520127500_noag152-B3","doi-asserted-by":"publisher","first-page":"S21","DOI":"10.1016\/j.ijrobp.2020.07.2108","article-title":"Stereotactic radiosurgery versus whole-brain radiation therapy for patients with 4-15 brain metastases: a phase III randomized controlled trial","volume":"108","author":"Li","year":"2020","journal-title":"IntJ Radiat Oncol Biol Phys"},{"key":"2026100502520127500_noag152-B4","doi-asserted-by":"publisher","first-page":"111172","DOI":"10.1016\/j.radonc.2025.111172","article-title":"Importance of 3D-TSE imaging for patients with brain metastasis treated with SRS alone","volume":"213","author":"Akdemir","year":"2025","journal-title":"Radiother Oncol."},{"key":"2026100502520127500_noag152-B5","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1016\/j.radonc.2022.05.029","article-title":"Dedicated isotropic 3-D T1 SPACE sequence imaging for radiosurgery planning improves brain metastases detection and reduces the risk of intracranial relapse","volume":"173","author":"Kutuk","year":"2022","journal-title":"Radiother Oncol."},{"key":"2026100502520127500_noag152-B6","doi-asserted-by":"publisher","first-page":"854","DOI":"10.1093\/neuonc\/noae232","article-title":"Efficacy of 3D-TSE sequence-based radiosurgery in prolonging time to distant intracranial failure: a session-wise analysis in a histology-diverse patient cohort","volume":"27","author":"Akdemir","year":"2025","journal-title":"Neuro Oncol"},{"key":"2026100502520127500_noag152-B7","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1093\/neuonc\/noz234","article-title":"Deep learning\u2013based detection and segmentation-assisted management of brain metastases","volume":"22","author":"Xue","year":"2020","journal-title":"Neuro Oncol."},{"key":"2026100502520127500_noag152-B8","doi-asserted-by":"publisher","first-page":"824","DOI":"10.1016\/j.mri.2012.02.024","article-title":"An approach for computer-aided detection of brain metastases in post-Gd T1-W MRI","volume":"30","author":"Farjam","year":"2012","journal-title":"Magn Reson Imaging."},{"key":"2026100502520127500_noag152-B9","doi-asserted-by":"publisher","first-page":"e0178265","DOI":"10.1371\/journal.pone.0178265","article-title":"Computer-aided detection of brain metastasis on 3D MR imaging: observer performance study","volume":"12","author":"Sunwoo","year":"2017","journal-title":"PLoS One."},{"key":"2026100502520127500_noag152-B10","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1007\/s11604-014-0342-9","article-title":"Additional MR contrast dosage for radiologists\u2019 diagnostic performance in detecting brain metastases: a systematic observer study at 3 T","volume":"32","author":"Togao","year":"2014","journal-title":"Jpn J Radiol."},{"key":"2026100502520127500_noag152-B11","doi-asserted-by":"publisher","first-page":"1515","DOI":"10.1080\/0284186X.2018.1473636","article-title":"Multi-institutional study of the variability in target delineation for six targets commonly treated with radiosurgery","volume":"57","author":"Sandstr\u00f6m","year":"2018","journal-title":"Acta Oncol."},{"key":"2026100502520127500_noag152-B12","doi-asserted-by":"publisher","first-page":"1560","DOI":"10.1093\/neuonc\/noab071","article-title":"Randomized multi-reader evaluation of automated detection and segmentation of brain tumors in stereotactic radiosurgery with deep neural networks","volume":"23","author":"Lu","year":"2021","journal-title":"Neuro Oncol."},{"key":"2026100502520127500_noag152-B13","doi-asserted-by":"publisher","first-page":"2140","DOI":"10.1093\/neuonc\/noae113","article-title":"Automated segmentation of brain metastases with deep learning: a multi-center, randomized crossover, multi-reader evaluation study","volume":"26","author":"Luo","year":"2024","journal-title":"Neuro Oncol."},{"key":"2026100502520127500_noag152-B14","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1007\/s11060-025-05294-5","article-title":"Evaluating the efficacy of a federated AI algorithm in assisting in the detection and segmentation of brain metastases: limitations and opportunities","volume":"176","author":"Akdemir","year":"2025","journal-title":"J Neurooncol."},{"key":"2026100502520127500_noag152-B15","doi-asserted-by":"publisher","first-page":"e230126","DOI":"10.1148\/ryai.230126","article-title":"The university of California San Francisco brain metastases stereotactic radiosurgery (UCSF-BMSR) MRI dataset","volume":"6","author":"Rudie","year":"2024","journal-title":"Radiol Artif Intell."},{"key":"2026100502520127500_noag152-B16","doi-asserted-by":"publisher","first-page":"vdac184","DOI":"10.1093\/noajnl\/vdac184","article-title":"NS-HGlio: a generalizable and repeatable HGG segmentation and volumetric measurement AI algorithm for the longitudinal MRI assessment to inform RANO in trials and clinics","volume":"5","author":"Abayazeed","year":"2023","journal-title":"Neurooncol Adv."},{"key":"2026100502520127500_noag152-B17","doi-asserted-by":"publisher","first-page":"1188","DOI":"10.1093\/neuonc\/nov095","article-title":"Consensus recommendations for a standardized brain tumor imaging protocol in clinical trials","volume":"17","author":"Ellingson","year":"2015","journal-title":"Neuro Oncol"},{"key":"2026100502520127500_noag152-B18","doi-asserted-by":"publisher","first-page":"e242416","DOI":"10.1148\/radiol.242416","article-title":"A data-centric approach to deep learning for brain metastasis analysis at MRI","volume":"315","author":"Topff","year":"2025","journal-title":"Radiology."},{"key":"2026100502520127500_noag152-B19","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1038\/s41592-023-02151-z","article-title":"Metrics reloaded: recommendations for image analysis validation","volume":"21","author":"Maier-Hein","year":"2024","journal-title":"Nat Methods."},{"key":"2026100502520127500_noag152-B20","doi-asserted-by":"publisher","first-page":"994","DOI":"10.1016\/j.ijrobp.2023.05.034","article-title":"Comprehensive evaluation of a deep learning model for automatic organs-at-risk segmentation on heterogeneous computed tomography images for abdominal radiation therapy","volume":"117","author":"Liao","year":"2023","journal-title":"Int J Radiat Oncol Biol Phys."},{"key":"2026100502520127500_noag152-B21","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1186\/s13014-020-01514-6","article-title":"Deep convolutional neural networks for automated segmentation of brain metastases trained on clinical data","volume":"15","author":"Bousabarah","year":"2020","journal-title":"Radiat Oncol."},{"key":"2026100502520127500_noag152-B22","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.radonc.2020.09.016","article-title":"MetNet: computer-aided segmentation of brain metastases in post-contrast T1-weighted magnetic resonance imaging","volume":"153","author":"Zhou","year":"2020","journal-title":"Radiother Oncol."},{"key":"2026100502520127500_noag152-B23","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.compbiomed.2018.02.004","article-title":"Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network","volume":"95","author":"Charron","year":"2018","journal-title":"Comput Biol Med."},{"key":"2026100502520127500_noag152-B24","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1038\/s41746-021-00398-4","article-title":"Handling missing MRI sequences in deep learning segmentation of brain metastases: a multicenter study","volume":"4","author":"Gr\u00f8vik","year":"2021","journal-title":"NPJ Digit Med."},{"key":"2026100502520127500_noag152-B25","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1007\/s11060-025-05365-7","article-title":"Oncological value of MRI in brain metastasis: exploring the potential of combining post-contrast T1 TSE (SPACE) and T1 GRE (MPRAGE) for stereotactic radiosurgery planning and surveillance","volume":"176","author":"Uysal","year":"2025","journal-title":"J Neurooncol."},{"key":"2026100502520127500_noag152-B26","doi-asserted-by":"publisher","first-page":"6686","DOI":"10.1007\/s00330-021-07783-3","article-title":"Robust performance of deep learning for automatic detection and segmentation of brain metastases using three-dimensional black-blood and three-dimensional gradient echo imaging","volume":"31","author":"Park","year":"2021","journal-title":"Eur Radiol."},{"key":"2026100502520127500_noag152-B27","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics11061016","article-title":"Contrast-enhanced black blood MRI sequence is superior to conventional T1 sequence in automated detection of brain metastases by convolutional neural networks","volume":"11","author":"Kottlors","year":"2021","journal-title":"Diagnostics (Basel)"},{"key":"2026100502520127500_noag152-B28","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s41747-025-00554-5","article-title":"Enhancing deep learning methods for brain metastasis detection through cross-technique annotations on SPACE MRI","volume":"9","author":"Wald","year":"2025","journal-title":"Eur Radiol Exp."},{"key":"2026100502520127500_noag152-B29","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1016\/j.ijrobp.2022.09.068","article-title":"A deep learning-based computer aided detection (CAD) system for difficult-to-detect brain metastases","volume":"115","author":"Fairchild","year":"2023","journal-title":"Int J Radiat Oncol Biol Phys."}],"container-title":["Neuro-Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/advance-article-pdf\/doi\/10.1093\/neuonc\/noag152\/68732380\/noag152.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/28\/10\/2449\/68732380\/noag152.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article-pdf\/28\/10\/2449\/68732380\/noag152.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T06:52:08Z","timestamp":1791183128000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/neuro-oncology\/article\/28\/10\/2449\/8728809"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,8]]},"references-count":29,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,7,8]]},"published-print":{"date-parts":[[2026,10,1]]}},"URL":"https:\/\/doi.org\/10.1093\/neuonc\/noag152","relation":{},"ISSN":["1522-8517","1523-5866"],"issn-type":[{"value":"1522-8517","type":"print"},{"value":"1523-5866","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,10]]},"published":{"date-parts":[[2026,7,8]]}}}