{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T18:34:55Z","timestamp":1785868495176,"version":"3.56.0"},"reference-count":54,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T00:00:00Z","timestamp":1641859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001782","name":"University of Melbourne","doi-asserted-by":"publisher","award":["Melbourne Research Scholarship (MRS)"],"award-info":[{"award-number":["Melbourne Research Scholarship (MRS)"]}],"id":[{"id":"10.13039\/501100001782","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001217","name":"Australian Government","doi-asserted-by":"publisher","award":["Research Training Program (RTP)"],"award-info":[{"award-number":["Research Training Program (RTP)"]}],"id":[{"id":"10.13039\/501100001217","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Multi-modal three-dimensional (3-D) image segmentation is used in many medical applications, such as disease diagnosis, treatment planning, and image-guided surgery. Although multi-modal images provide information that no single image modality alone can provide, integrating such information to be used in segmentation is a challenging task. Numerous methods have been introduced to solve the problem of multi-modal medical image segmentation in recent years. In this paper, we propose a solution for the task of brain tumor segmentation. To this end, we first introduce a method of enhancing an existing magnetic resonance imaging (MRI) dataset by generating synthetic computed tomography (CT) images. Then, we discuss a process of systematic optimization of a convolutional neural network (CNN) architecture that uses this enhanced dataset, in order to customize it for our task. Using publicly available datasets, we show that the proposed method outperforms similar existing methods.<\/jats:p>","DOI":"10.3390\/s22020523","type":"journal-article","created":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T20:33:04Z","timestamp":1641933184000},"page":"523","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":41,"title":["A Deep Learning Framework for Segmenting Brain Tumors Using MRI and Synthetically Generated CT Images"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2172-7041","authenticated-orcid":false,"given":"Kh Tohidul","family":"Islam","sequence":"first","affiliation":[{"name":"Department of Surgery (Otolaryngology), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, VIC 3010, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8015-8577","authenticated-orcid":false,"given":"Sudanthi","family":"Wijewickrema","sequence":"additional","affiliation":[{"name":"Department of Surgery (Otolaryngology), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, VIC 3010, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6926-2103","authenticated-orcid":false,"given":"Stephen","family":"O\u2019Leary","sequence":"additional","affiliation":[{"name":"Department of Surgery (Otolaryngology), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne, Melbourne, VIC 3010, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,11]]},"reference":[{"key":"ref_1","first-page":"S1076633221000891","article-title":"Medical Image Analyst: A Radiology Career Focused on Comprehensive Quantitative Imaging Analytics to Improve Healthcare","volume":"29","author":"Ferreira","year":"2021","journal-title":"Acad. Radiol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1231","DOI":"10.1016\/j.aej.2020.10.046","article-title":"Efficient 3D Deep Learning Model for Medical Image Semantic Segmentation","volume":"60","author":"Alalwan","year":"2021","journal-title":"Alex. Eng. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/j.neucom.2018.11.027","article-title":"Semantic image segmentation via guidance of image classification","volume":"330","author":"Shen","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"27867","DOI":"10.1007\/s11042-020-09384-6","article-title":"Deep learning neural network for texture feature extraction in oral cancer: Enhanced loss function","volume":"79","author":"Bhandari","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10930","DOI":"10.1038\/s41598-021-90428-8","article-title":"Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images","volume":"11","author":"Ranjbarzadeh","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.neucom.2020.10.031","article-title":"Segmentation of the multimodal brain tumor image used the multi-pathway architecture method based on 3D FCN","volume":"423","author":"Sun","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tan, W., Tiwari, P., Pandey, H.M., Moreira, C., and Jaiswal, A.K. (2020). Multimodal medical image fusion algorithm in the era of big data. Neural Comput. Appl.","DOI":"10.1007\/s00521-020-05173-2"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"100004","DOI":"10.1016\/j.array.2019.100004","article-title":"A review: Deep learning for medical image segmentation using multi-modality fusion","volume":"3\u20134","author":"Zhou","year":"2019","journal-title":"Array"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1109\/TRPMS.2018.2890359","article-title":"Deep Learning-Based Image Segmentation on Multimodal Medical Imaging","volume":"3","author":"Guo","year":"2019","journal-title":"IEEE Trans. Radiat. Plasma Med Sci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Arabnia, H.R., Ferens, K., de la Fuente, D., Kozerenko, E.B., Olivas Varela, J.A., and Tinetti, F.G. (2021). Brain Tumor Segmentation Using Deep Neural Networks and Survival Prediction. Advances in Artificial Intelligence and Applied Cognitive Computing, Springer International Publishing.","DOI":"10.1007\/978-3-030-70296-0"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.1007\/s12652-020-02386-0","article-title":"Multi-modality medical image fusion technique using multi-objective differential evolution based deep neural networks","volume":"12","author":"Kaur","year":"2021","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.media.2017.10.002","article-title":"A deep learning model integrating FCNNs and CRFs for brain tumor segmentation","volume":"43","author":"Zhao","year":"2018","journal-title":"Med. Image Anal."},{"key":"ref_13","first-page":"637","article-title":"One-Pass Multi-task Convolutional Neural Networks for Efficient Brain Tumor Segmentation","volume":"Volume 11072","author":"Frangi","year":"2018","journal-title":"Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2018"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1993","DOI":"10.1109\/TMI.2014.2377694","article-title":"The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)","volume":"34","author":"Menze","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Quan, T.M., Hildebrand, D.G.C., and Jeong, W.K. (2021). FusionNet: A Deep Fully Residual Convolutional Neural Network for Image Segmentation in Connectomics. Front. Comput. Sci., 3.","DOI":"10.3389\/fcomp.2021.613981"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., and Darrell, T. (2014, January 3\u20134). Caffe: Convolutional Architecture for Fast Feature Embedding. Proceedings of the 22nd ACM International Conference on Multimedia, Orlando, FL, USA.","DOI":"10.1145\/2647868.2654889"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, G., Li, W., Ourselin, S., and Vercauteren, T. (2019). Automatic Brain Tumor Segmentation Based on Cascaded Convolutional Neural Networks With Uncertainty Estimation. Front. Comput. Neurosci., 13.","DOI":"10.3389\/fncom.2019.00056"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5471","DOI":"10.1088\/0031-9155\/60\/14\/5471","article-title":"A radiomics model from joint FDG-PET and MRI texture features for the prediction of lung metastases in soft-tissue sarcomas of the extremities","volume":"60","author":"Freeman","year":"2015","journal-title":"Phys. Med. Biol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","article-title":"The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository","volume":"26","author":"Clark","year":"2013","journal-title":"J. Digit. Imaging"},{"key":"ref_20","first-page":"154","article-title":"Segmentation of Brain Tumors Using DeepLabv3+","volume":"Volume 11384","author":"Crimi","year":"2019","journal-title":"Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1007\/978-3-030-01234-2_49","article-title":"Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation","volume":"Volume 11211","author":"Ferrari","year":"2018","journal-title":"Computer Vision\u2014ECCV 2018"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_23","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_24","unstructured":"Meade, B., Lafayette, L., Sauter, G., and Tosello, D. (2017). Spartan HPC-Cloud Hybrid: Delivering Performance and Flexibility, University of Melbourne."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1002\/mp.12155","article-title":"MR-based synthetic CT generation using a deep convolutional neural network method","volume":"44","author":"Han","year":"2017","journal-title":"Med. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"41839","DOI":"10.1109\/ACCESS.2018.2858196","article-title":"Structurally-Sensitive Multi-Scale Deep Neural Network for Low-Dose CT Denoising","volume":"6","author":"You","year":"2018","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1333","DOI":"10.3389\/fonc.2019.01333","article-title":"Synthetic CT Generation Based on T2 Weighted MRI of Nasopharyngeal Carcinoma (NPC) Using a Deep Convolutional Neural Network (DCNN)","volume":"9","author":"Wang","year":"2019","journal-title":"Front. Oncol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.phro.2020.12.007","article-title":"Synthetic computed tomography data allows for accurate absorbed dose calculations in a magnetic resonance imaging only workflow for head and neck radiotherapy","volume":"17","author":"Karlsson","year":"2021","journal-title":"Phys. Imaging Radiat. Oncol."},{"key":"ref_29","unstructured":"Loew, M.H., and Hanson, K.M. (1996). Comparison and evaluation of retrospective intermodality image registration techniques. Medical Imaging 1996: Image Processing, SPIE."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_31","unstructured":"\u00d6zg\u00fcn, \u00c7., Abdulkadir, A., Lienkamp, S.S., Brox, T., and Ronneberger, O. (2016). 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2016, Springer International Publishing."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5659","DOI":"10.1002\/mp.13247","article-title":"Technical Note: U-net-generated synthetic CT images for magnetic resonance imaging-only prostate intensity-modulated radiation therapy treatment planning","volume":"45","author":"Chen","year":"2018","journal-title":"Med. Phys."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gupta, D., Kim, M., Vineberg, K.A., and Balter, J.M. (2019). Generation of Synthetic CT Images From MRI for Treatment Planning and Patient Positioning Using a 3-Channel U-Net Trained on Sagittal Images. Front. Oncol., 9.","DOI":"10.3389\/fonc.2019.00964"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"125001","DOI":"10.1088\/1361-6560\/ab8cd2","article-title":"Abdominal synthetic CT generation from MR Dixon images using a U-net trained with \u2018semi-synthetic\u2019 CT data","volume":"65","author":"Liu","year":"2020","journal-title":"Phys. Med. Biol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1002\/acm2.13327","article-title":"Deep learning-based synthetic CT generation for MR-only radiotherapy of prostate cancer patients with 0.35T MRI linear accelerator","volume":"22","author":"Farjam","year":"2021","journal-title":"J. Appl. Clin. Med. Phys."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1038\/s41598-021-81044-7","article-title":"A deep learning based framework for the registration of three dimensional multi-modal medical images of the head","volume":"11","author":"Islam","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_37","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, Haifa, Israel."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A Stochastic Approximation Method","volume":"22","author":"Robbins","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_39","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"103248","DOI":"10.1016\/j.cviu.2021.103248","article-title":"High-level prior-based loss functions for medical image segmentation: A survey","volume":"210","author":"Jurdi","year":"2021","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sarvamangala, D.R., and Kulkarni, R.V. (2021). Convolutional neural networks in medical image understanding: A survey. Evol. Intell.","DOI":"10.1007\/s12065-020-00540-3"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Minaee, S., Boykov, Y.Y., Porikli, F., Plaza, A.J., Kehtarnavaz, N., and Terzopoulos, D. (2021). Image Segmentation Using Deep Learning: A Survey. IEEE Trans. Pattern Anal. Mach. Intell., 1.","DOI":"10.1109\/TPAMI.2021.3059968"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_44","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, inception-ResNet and the impact of residual connections on learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_48","first-page":"287","article-title":"Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge","volume":"Volume 10670","author":"Crimi","year":"2018","journal-title":"Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Salehi, S.S.M., Erdogmus, D., and Gholipour, A. (2017). Tversky Loss Function for Image Segmentation Using 3D Fully Convolutional Deep Networks. Machine Learning in Medical Imaging, Springer International Publishing.","DOI":"10.1007\/978-3-319-67389-9_44"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., and Cardoso, M.J. (2017). Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer International Publishing.","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/S1076-6332(03)00671-8","article-title":"Statistical validation of image segmentation quality based on a spatial overlap index1","volume":"11","author":"Zou","year":"2004","journal-title":"Acad. Radiol."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Csurka, G., Larlus, D., and Perronnin, F. (2013). What is a good evaluation measure for semantic segmentation? Procedings of the British Machine Vision Conference 2013, British Machine Vision Association.","DOI":"10.5244\/C.27.32"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s12204-021-2264-x","article-title":"Rethinking the Dice Loss for Deep Learning Lesion Segmentation in Medical Images","volume":"26","author":"Zhang","year":"2021","journal-title":"J. Shanghai Jiaotong Univ. (Sci.)"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1038\/s41746-020-00341-z","article-title":"Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines","volume":"3","author":"Huang","year":"2020","journal-title":"NPJ Digit. Med."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/523\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:01:57Z","timestamp":1760364117000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/2\/523"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,11]]},"references-count":54,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22020523"],"URL":"https:\/\/doi.org\/10.3390\/s22020523","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,11]]}}}