{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T15:14:32Z","timestamp":1785856472289,"version":"3.56.0"},"reference-count":35,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T00:00:00Z","timestamp":1736467200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005416","name":"Norges Forskningsr\u00e5d","doi-asserted-by":"publisher","award":["261984"],"award-info":[{"award-number":["261984"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005416","name":"Norges Forskningsr\u00e5d","doi-asserted-by":"publisher","award":["325971"],"award-info":[{"award-number":["325971"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["758657\u2010ImPRESS"],"award-info":[{"award-number":["758657\u2010ImPRESS"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008730","name":"Kreftforeningen","doi-asserted-by":"publisher","award":["3434180"],"award-info":[{"award-number":["3434180"]}],"id":[{"id":"10.13039\/100008730","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008730","name":"Kreftforeningen","doi-asserted-by":"publisher","award":["6817564"],"award-info":[{"award-number":["6817564"]}],"id":[{"id":"10.13039\/100008730","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Magnetic Resonance Imaging"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Deep learning\u2010based segmentation of brain metastases relies on large amounts of fully annotated data by domain experts. Semi\u2010supervised learning offers potential efficient methods to improve model performance without excessive annotation burden.<\/jats:p><\/jats:sec><jats:sec><jats:title>Purpose<\/jats:title><jats:p>This work tests the viability of semi\u2010supervision for brain metastases segmentation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Study Type<\/jats:title><jats:p>Retrospective.<\/jats:p><\/jats:sec><jats:sec><jats:title>Subjects<\/jats:title><jats:p>There were 156, 65, 324, and 200 labeled scans from four institutions and 519 unlabeled scans from a single institution. All subjects included in the study had diagnosed with brain metastases.<\/jats:p><\/jats:sec><jats:sec><jats:title>Field Strength\/Sequences<\/jats:title><jats:p>1.5\u2009T and 3\u2009T, 2D and 3D T1\u2010weighted pre\u2010 and post\u2010contrast, and fluid\u2010attenuated inversion recovery (FLAIR).<\/jats:p><\/jats:sec><jats:sec><jats:title>Assessment<\/jats:title><jats:p>Three semi\u2010supervision methods (mean teacher, cross\u2010pseudo supervision, and interpolation consistency training) were adapted with the U\u2010Net architecture. The three semi\u2010supervised methods were compared to their respective supervised baseline on the full and half\u2010sized training.<\/jats:p><\/jats:sec><jats:sec><jats:title>Statistical Tests<\/jats:title><jats:p>Evaluation was performed on a multinational test set from four different institutions using 5\u2010fold cross\u2010validation. Method performance was evaluated by the following: the number of false\u2010positive predictions, the number of true positive predictions, the 95th Hausdorff distance, and the Dice similarity coefficient (DSC). Significance was tested using a paired samples <jats:italic>t<\/jats:italic> test for a single fold, and across all folds within a given cohort.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Semi\u2010supervision outperformed the supervised baseline for all sites with the best\u2010performing semi\u2010supervised method achieved an on average DSC improvement of 6.3%\u2009\u00b1\u20091.6%, 8.2%\u2009\u00b1\u20093.8%, 8.6%\u2009\u00b1\u20092.6%, and 15.4%\u2009\u00b1\u20091.4%, when trained on half the dataset and 3.6%\u2009\u00b1\u20090.7%, 2.0%\u2009\u00b1\u20091.5%, 1.8%\u2009\u00b1\u20095.7%, and 4.7%\u2009\u00b1\u20091.7%, compared to the supervised baseline on four test cohorts. In addition, in three of four datasets, the semi\u2010supervised training produced equal or better results than the supervised models trained on twice the labeled data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Data Conclusion<\/jats:title><jats:p>Semi\u2010supervised learning allows for improved segmentation performance over the supervised baseline, and the improvement was particularly notable for independent external test sets when trained on small amounts of labeled data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Plain Language Summary<\/jats:title><jats:p>Artificial intelligence requires extensive datasets with large amounts of annotated data from medical experts which can be difficult to acquire due to the large workload. To compensate for this, it is possible to utilize large amounts of un\u2010annotated clinical data in addition to annotated data. However, this method has not been widely tested for the most common intracranial brain tumor, brain metastases. This study shows that this approach allows for data efficient deep learning models across multiple institutions with different clinical protocols and scanners.<\/jats:p><\/jats:sec><jats:sec><jats:title>Level of Evidence<\/jats:title><jats:p>3<\/jats:p><\/jats:sec><jats:sec><jats:title>Technical Efficacy<\/jats:title><jats:p>Stage 2<\/jats:p><\/jats:sec>","DOI":"10.1002\/jmri.29686","type":"journal-article","created":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T18:29:54Z","timestamp":1736533794000},"page":"2469-2479","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Semi\u2010Supervised Learning Allows for Improved Segmentation With Reduced Annotations of Brain Metastases Using Multicenter <scp>MRI<\/scp> Data"],"prefix":"10.1002","volume":"61","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5383-2439","authenticated-orcid":false,"given":"Jon Andr\u00e9","family":"Ottesen","sequence":"first","affiliation":[{"name":"Computational Radiology and Artificial Intelligence (CRAI) Research Group, Division of Radiology and Nuclear Medicine Oslo University Hospital  Oslo Norway"},{"name":"Department of Physics, Faculty of Mathematics and Natural Sciences University of Oslo  Oslo Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9550-8178","authenticated-orcid":false,"given":"Elizabeth","family":"Tong","sequence":"additional","affiliation":[{"name":"Department of Radiology Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kyrre Eeg","family":"Emblem","sequence":"additional","affiliation":[{"name":"Department of Physics and Computational Radiology, Division of Radiology and Nuclear Medicine Oslo University Hospital  Oslo Norway"},{"name":"Institute of Clinical Medicine, Faculty of Medicine University of Oslo  Oslo Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna","family":"Latysheva","sequence":"additional","affiliation":[{"name":"Division of Radiology and Nuclear Medicine Oslo University Hospital  Oslo Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5781-8848","authenticated-orcid":false,"given":"Greg","family":"Zaharchuk","sequence":"additional","affiliation":[{"name":"Department of Radiology Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Atle","family":"Bj\u00f8rnerud","sequence":"additional","affiliation":[{"name":"Computational Radiology and Artificial Intelligence (CRAI) Research Group, Division of Radiology and Nuclear Medicine Oslo University Hospital  Oslo Norway"},{"name":"Department of Physics, Faculty of Mathematics and Natural Sciences University of Oslo  Oslo Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9925-1162","authenticated-orcid":false,"given":"Endre","family":"Gr\u00f8vik","sequence":"additional","affiliation":[{"name":"Department of Radiology \u00c5lesund Hospital, M\u00f8re og Romsdal Hospital Trust  \u00c5lesund Norway"},{"name":"Department of Physics Norwegian University of Science and Technology  Trondheim Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,1,10]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/S11912-011-0203-Y\/TABLES\/1"},{"key":"e_1_2_8_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41572-018-0055-y"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/S11060-004-8093-6\/METRICS"},{"key":"e_1_2_8_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/NEUONC\/NOAA030"},{"key":"e_1_2_8_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.ANNONC.2021.07.016"},{"key":"e_1_2_8_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1470-2045(15)70057-4"},{"key":"e_1_2_8_8_1","doi-asserted-by":"publisher","DOI":"10.1002\/JMRI.26766"},{"key":"e_1_2_8_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.COMPBIOMED.2018.02.004"},{"key":"e_1_2_8_10_1","doi-asserted-by":"publisher","DOI":"10.1186\/S13014-020-01514-6\/TABLES\/3"},{"key":"e_1_2_8_11_1","doi-asserted-by":"publisher","DOI":"10.3389\/FNINF.2022.1056068\/BIBTEX"},{"key":"e_1_2_8_12_1","doi-asserted-by":"publisher","DOI":"10.1371\/JOURNAL.PONE.0185844"},{"key":"e_1_2_8_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/JMRI.27741"},{"key":"e_1_2_8_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/MP.15863"},{"key":"e_1_2_8_15_1","doi-asserted-by":"publisher","DOI":"10.1148\/RYAI.2021200204"},{"key":"e_1_2_8_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3220219"},{"key":"e_1_2_8_17_1","unstructured":"LeeD\u2010H.Pseudo\u2010label: The simple and efficient semi\u2010supervised learning method for deep neural networks. In:ICML 2013 Workshop: Challenges in Representation Learning.2013."},{"key":"e_1_2_8_18_1","volume-title":"Advances in neural information processing systems","author":"Tarvainen A"},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"OermannE LinkK SchnurmanZ et al.Longitudinal deep neural networks for assessing metastatic brain cancer on a massive open benchmark.2023.https:\/\/doi.org\/10.21203\/RS.3.RS-2444113\/V1.","DOI":"10.1038\/s41467-024-52414-2"},{"key":"e_1_2_8_20_1","unstructured":"MoawadAW JanasA BaidU et al.The Brain Tumor Segmentation (BraTS\u2010METS) Challenge 2023: Brain metastasis segmentation on pre\u2010treatment MRI.2023. Accessed on July 24 2023. Available from:https:\/\/arxiv.org\/abs\/2306.00838v1."},{"key":"e_1_2_8_21_1","doi-asserted-by":"publisher","DOI":"10.3390\/DIAGNOSTICS12082023"},{"key":"e_1_2_8_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.MEDIA.2022.102517"},{"key":"e_1_2_8_23_1","doi-asserted-by":"publisher","DOI":"10.7557\/18.6815"},{"key":"e_1_2_8_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2022.3161829"},{"key":"e_1_2_8_25_1","first-page":"2613","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Chen X","year":"2021"},{"key":"e_1_2_8_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.NEUNET.2021.10.008"},{"key":"e_1_2_8_27_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-021-00398-4"},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-024-03021-9"},{"key":"e_1_2_8_29_1","doi-asserted-by":"publisher","DOI":"10.1148\/RYAI.230126"},{"key":"e_1_2_8_30_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-87564-6"},{"key":"e_1_2_8_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72087-2_11"},{"key":"e_1_2_8_32_1","unstructured":"Consortium M.MONAI: Medical Open Network for AI.2022.https:\/\/doi.org\/10.5281\/ZENODO.6114127."},{"key":"e_1_2_8_33_1","unstructured":"LoshchilovI HutterF.Decoupled weight decay regularization. In:7th Int Conf Learn Represent ICLR 2019.2017. Accessed on February 15 2022. Available from:https:\/\/arxiv.org\/abs\/1711.05101v3."},{"key":"e_1_2_8_34_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/AC1835"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1093\/NEUONC\/NOZ234"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2020.2982103"}],"container-title":["Journal of Magnetic Resonance Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/jmri.29686","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,9]],"date-time":"2025-05-09T16:29:34Z","timestamp":1746808174000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/jmri.29686"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,10]]},"references-count":35,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["10.1002\/jmri.29686"],"URL":"https:\/\/doi.org\/10.1002\/jmri.29686","archive":["Portico"],"relation":{},"ISSN":["1053-1807","1522-2586"],"issn-type":[{"value":"1053-1807","type":"print"},{"value":"1522-2586","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,10]]},"assertion":[{"value":"2024-08-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-04","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}