{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,13]],"date-time":"2025-09-13T16:44:05Z","timestamp":1757781845948,"version":"3.41.2"},"reference-count":84,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T00:00:00Z","timestamp":1667865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010877","name":"Science, Technology and Innovation Commission of Shenzhen Municipality","doi-asserted-by":"publisher","award":["JCYJ20190809180003689","JSGG20191129110812708","JSGG20200225150707332","ZDSYS202008201654000"],"award-info":[{"award-number":["JCYJ20190809180003689","JSGG20191129110812708","JSGG20200225150707332","ZDSYS202008201654000"]}],"id":[{"id":"10.13039\/501100010877","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Deep learning techniques have shown great potential in medical image processing, particularly through accurate and reliable image segmentation on magnetic resonance imaging (MRI) scans or computed tomography (CT) scans, which allow the localization and diagnosis of lesions. However, training these segmentation models requires a large number of manually annotated pixel-level labels, which are time-consuming and labor-intensive, in contrast to image-level labels that are easier to obtain. It is imperative to resolve this problem through weakly-supervised semantic segmentation models using image-level labels as supervision since it can significantly reduce human annotation efforts. Most of the advanced solutions exploit class activation mapping (CAM). However, the original CAMs rarely capture the precise boundaries of lesions. In this study, we propose the strategy of multi-scale inference to refine CAMs by reducing the detail loss in single-scale reasoning. For segmentation, we develop a novel model named Mixed-UNet, which has two parallel branches in the decoding phase. The results can be obtained after fusing the extracted features from two branches. We evaluate the designed Mixed-UNet against several prevalent deep learning-based segmentation approaches on our dataset collected from the local hospital and public datasets. The validation results demonstrate that our model surpasses available methods under the same supervision level in the segmentation of various lesions from brain imaging.<\/jats:p>","DOI":"10.3389\/fcomp.2022.1036934","type":"journal-article","created":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T07:27:55Z","timestamp":1667892475000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Mixed-UNet: Refined class activation mapping for weakly-supervised semantic segmentation with multi-scale inference"],"prefix":"10.3389","volume":"4","author":[{"given":"Yang","family":"Liu","sequence":"first","affiliation":[]},{"given":"Lijin","family":"Lian","sequence":"additional","affiliation":[]},{"given":"Ersi","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Lulu","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Chufan","family":"Xiao","sequence":"additional","affiliation":[]},{"given":"Xiaoyun","family":"Zhong","sequence":"additional","affiliation":[]},{"given":"Fang","family":"Li","sequence":"additional","affiliation":[]},{"given":"Bin","family":"Jiang","sequence":"additional","affiliation":[]},{"given":"Yuhan","family":"Dong","sequence":"additional","affiliation":[]},{"given":"Lan","family":"Ma","sequence":"additional","affiliation":[]},{"given":"Qiming","family":"Huang","sequence":"additional","affiliation":[]},{"given":"Ming","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Yongbing","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Dongmei","family":"Yu","sequence":"additional","affiliation":[]},{"given":"Chenggang","family":"Yan","sequence":"additional","affiliation":[]},{"given":"Peiwu","family":"Qin","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2022,11,8]]},"reference":[{"key":"B1","article-title":"Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation,","author":"Ahn","year":"2018","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B2","first-page":"4252","author":"Araslanov","year":"2020","journal-title":"Single-Stage Semantic Segmentation from Image Labels"},{"key":"B3","doi-asserted-by":"publisher","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"Segnet: a deep convolutional Encoder\u2013decoder architecture for image segmentation.","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE. Trans. Pattern. Anal. Mach. Intell."},{"journal-title":"What's the Point: Semantic Segmentation with Point Supervision","year":"2016","author":"Bearman","key":"B4"},{"key":"B5","first-page":"39","author":"Bonta","year":"2019","journal-title":"Efficient Segmentation of Medical Images Using Dilated Residual Networks Computer Aided Intervention and Diagnostics in Clinical and Medical Images"},{"journal-title":"Convolutional Simplex Projection Network (Cspn) for Weakly Supervised Semantic Segmentation","year":"2018","author":"Briq","key":"B6"},{"key":"B7","article-title":"Partially reversible u-net for memory-efficient volumetric image segmentation,","author":"Br\u00fcgger","year":"2019","journal-title":"Med Image Comput Comput Assist Interv (MICCAI)"},{"key":"B8","article-title":"Cascaded V-Net Using Roi Masks for Brain Tumor Segmentation,","author":"Casamitjana","year":"2017","journal-title":"Proceedings of the MICCAI BrainLes Workshops"},{"key":"B9","first-page":"513","author":"Chamanzar","year":"2020","journal-title":"Weakly Supervised Multi-Task Learning for Cell Detection and Segmentation"},{"key":"B10","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/s11263-020-01373-4","article-title":"Comprehensive analysis of weakly-supervised semantic segmentation in different image domains.","volume":"129","author":"Chan","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"B11","first-page":"8991","article-title":"Weakly-Supervised Semantic Segmentation Via Sub-Category Exploration,","author":"Chang","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B12","article-title":"Grad-Cam++: generalized gradient-based visual explanations for deep convolutional networks,","author":"Chattopadhay","year":"2018","journal-title":"Proceedings of the IEEE Winter Conference Application of Computer Visual (WACV)"},{"key":"B13","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected Crfs.","volume":"40","author":"Chen","year":"","journal-title":"IEEE. Trans. Pattern Anal. Mach. Intell."},{"journal-title":"Rethinking Atrous Convolution for Semantic Image Segmentation","year":"","author":"Chen","key":"B14"},{"journal-title":"Encoder\u2013decoder with Atrous Separable Convolution for Semantic Image Segmentation","year":"2018","author":"Chen","key":"B15"},{"key":"B16","doi-asserted-by":"publisher","first-page":"410","DOI":"10.1609\/aaai.v36i1.19918","article-title":"Lctr: on awakening the local continuity of transformer for weakly supervised object localization.","volume":"36","author":"Chen","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"B17","article-title":"Attention-based dropout layer for weakly supervised object localization,","author":"Choe","year":"2019","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B18","first-page":"933","author":"Cole","year":"2021","journal-title":"Multi-Label Learning from Single Positive Labels"},{"key":"B19","article-title":"Boxsup: exploiting bounding boxes to supervise convolutional networks for semantic segmentation,","author":"Dai","year":"2015","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"key":"B20","article-title":"Densely connected convolutional networks,","author":"Huang","year":"2017","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B21","article-title":"Weakly-supervised semantic segmentation network with deep seeded region growing,","author":"Huang","year":"2018","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"journal-title":"Subcellular Protein Localisation in the Human Protein Atlas Using Ensembles of Diverse Deep Architectures","year":"2012","author":"Husain","key":"B22"},{"year":"2015","author":"Ioffe","key":"B23"},{"key":"B24","doi-asserted-by":"publisher","first-page":"5875","DOI":"10.1109\/TIP.2021.3089943","article-title":"Layercam: exploring hierarchical class activation maps for localization. IEEE. Trans. Image","volume":"30","author":"Jiang","year":"2021","journal-title":"Process"},{"journal-title":"Recurseed and Certainmix for Weakly Supervised Semantic Segmentation","year":"2022","author":"Jo","key":"B25"},{"key":"B26","first-page":"639","author":"Jo","year":"2021","journal-title":"Puzzle-Cam: Improved Localization Via Matching Partial and Full Features"},{"key":"B27","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1159\/000448303","article-title":"Clinical stroke syndromes.","volume":"40","author":"Kim","year":"2016","journal-title":"Front. Neurol. Neurosci"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01376","author":"Kim","year":"2022","journal-title":"Large Loss Matters in Weakly Supervised Multi-Label. Classification"},{"journal-title":"Adam: A Method for Stochastic Optimization","year":"2014","author":"Kingma","key":"B29"},{"key":"B30","article-title":"Seed Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation","author":"Kolesnikov","year":"2016","journal-title":"Proc ECCV"},{"key":"B31","article-title":"Efficient inference in fully connected crfs with gaussian edge potentials,","author":"Kr\u00e4henb\u00fchl","year":"2011","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks.","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural. Inf. Process. Syst"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v31i1.11213","article-title":"Weakly supervised semantic segmentation using superpixel pooling network,","author":"Kwak","year":"2017","journal-title":"Proceedings of the Conference AAAI Artificial Intelligence"},{"key":"B34","first-page":"16473","author":"Lanchantin","year":"2021","journal-title":"General Multi-Label Image Classification with Transformers"},{"journal-title":"Robust Tumor Localization with Pyramid Grad-Cam","year":"2018","author":"Lee","key":"B35"},{"key":"B36","article-title":"Tell me where to look: guided attention inference network,","author":"Li","year":"2018","journal-title":"Proceedings of the IEEE Conference"},{"journal-title":"Transcam: Transformer Attention-Based Cam Refinement for Weakly Supervised Semantic Segmentation","year":"2022","author":"Li","key":"B37"},{"key":"B38","article-title":"Weakly supervised semantic segmentation based on deep learning,","author":"Liang","year":"2020","journal-title":"Proceedings of the IASTED International Conference of Model Identification Control (ICMIC)"},{"key":"B39","article-title":"Scribblesup: scribble-supervised convolutional networks for semantic segmentation,","author":"Lin","year":"2016","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B40","article-title":"Refinenet: multi-path refinement networks for high-resolution semantic segmentation,","author":"Lin","year":"2017","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B41","article-title":"Fully convolutional networks for semantic segmentation,","author":"Long","year":"2015","journal-title":"Proceedings of the IEEE Conference of Computer Visual Pattern Recognition (CVPR)"},{"key":"B42","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.zemedi.2018.11.002","article-title":"An overview of deep learning in medical imaging focusing on Mri.","volume":"29","author":"Lundervold","year":"2019","journal-title":"Z. Med. Phys."},{"journal-title":"Attention U-Net: Learning Where to Look for the Pancreas","year":"2018","author":"Oktay","key":"B43"},{"key":"B44","article-title":"Is object localization for free?-weakly-supervised learning with convolutional neural networks,","author":"Oquab","year":"2015","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B45","article-title":"Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation,","author":"Papandreou","year":"2015","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"key":"B46","article-title":"Constrained convolutional neural networks for weakly supervised segmentation,","author":"Pathak","year":"2015","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"journal-title":"Fully Convolutional Multi-Class Multiple Instance Learning","year":"2014","author":"Pathak","key":"B47"},{"key":"B48","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1016\/j.neunet.2020.07.011","article-title":"Discretely-constrained deep network for weakly supervised segmentation.","volume":"130","author":"Peng","year":"2020","journal-title":"Neural. Netw."},{"key":"B49","first-page":"49","article-title":"Deep segmentation refinement with result-dependent learning,","author":"Pham","year":"2019","journal-title":"Bildverarbeitung F\u00fcr Die Medizin"},{"key":"B50","article-title":"From image-level to pixel-level labeling with convolutional networks,","author":"Pinheiro","year":"2015","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B51","article-title":"Weakly supervised graph based semantic segmentation by learning communities of image-parts,","author":"Pourian","year":"2015","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"key":"B52","article-title":"Semantic Segmentation with Object Clique Potential,","author":"Qi","year":"2015","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"key":"B53","doi-asserted-by":"crossref","first-page":"3655","DOI":"10.1109\/TMI.2020.3002244","article-title":"Weakly supervised deep nuclei segmentation using partial points annotation in histopathology images.","author":"Qu","year":"2020","journal-title":"IEEE transactions on medical imaging"},{"key":"B54","first-page":"82","author":"Ridnik","year":"2021","journal-title":"Asymmetric Loss for Multi-Label Classification"},{"key":"B55","unstructured":"U-Net: convolutional networks for biomedical image segmentation,\n            RonnebergerO.\n            FischerP.\n            BroxT.\n          Berlin, GermanySpringer2015"},{"key":"B56","article-title":"Combining bottom-up, top-down, and smoothness cues for weakly supervised image segmentation,","author":"Roy","year":"2017","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B57","article-title":"Grad-cam: visual explanations from deep networks via gradient-based localization,","author":"Selvaraju","year":"2017","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)."},{"key":"B58","doi-asserted-by":"publisher","first-page":"3068","DOI":"10.1109\/TCYB.2019.2936503","article-title":"Visual object tracking by hierarchical attention siamese network.","volume":"50","author":"Shen","year":"2019","journal-title":"IEEE. Trans. Cybern"},{"key":"B59","article-title":"Box-driven class-wise region masking and filling rate guided loss for weakly supervised semantic segmentation,","author":"Song","year":"2019","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B60","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1007\/s10462-020-09854-1","article-title":"Deep semantic segmentation of natural and medical images: a review.","volume":"54","author":"Taghanaki","year":"2021","journal-title":"Artif. Intell. Rev."},{"key":"B61","article-title":"Learning random-walk label propagation for weakly-supervised semantic segmentation,","author":"Vernaza","year":"2017","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B62","article-title":"Weakly supervised semantic segmentation with a multi-image model,\u201d","author":"Vezhnevets","year":"2011","journal-title":"Proceedings of the IEEE International Conference of Computer Visual (ICCV)"},{"key":"B63","doi-asserted-by":"publisher","first-page":"6050","DOI":"10.1109\/TIP.2021.3091833","article-title":"Multi-scale low-discriminative feature reactivation for weakly supervised object localization.","volume":"30","author":"Wang","year":"2021","journal-title":"IEEE. Trans. Image. Process"},{"key":"B64","article-title":"Score-cam: score-weighted visual explanations for convolutional neural networks,","author":"Wang","year":"2020","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B65","doi-asserted-by":"publisher","first-page":"1736","DOI":"10.1007\/s11263-020-01293-3","article-title":"Weakly-supervised semantic segmentation by iterative affinity learning.","volume":"128","author":"Wang","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"B66","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.neucom.2019.11.019","article-title":"Deep clustering for weakly-supervised semantic segmentation in autonomous driving scenes","volume":"381","author":"Wang","year":"2019","journal-title":"Neurocomputing"},{"key":"B67","article-title":"Weakly-supervised semantic segmentation by iteratively mining common object features,","author":"Wang","year":"2018","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"journal-title":"Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation","year":"2020","author":"Wang","key":"B68"},{"key":"B69","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1002\/ana.410370214","article-title":"Acute human stroke studied by whole brain echo planar diffusion-weighted magnetic resonance imaging.","volume":"37","author":"Warach","year":"1995","journal-title":"Ann. Neurol."},{"key":"B70","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1016\/j.patcog.2016.01.015","article-title":"Learning to segment with image-level annotations. Pattern","volume":"59","author":"Wei","year":"","journal-title":"Recognit."},{"key":"B71","doi-asserted-by":"publisher","first-page":"1901","DOI":"10.1109\/TPAMI.2015.2491929","article-title":"Hcp: a flexible cnn framework for multi-label image classification. IEEE. Trans. Pattern. Anal. Mach","volume":"38","author":"Wei","year":"","journal-title":"Intell."},{"key":"B72","article-title":"Revisiting dilated convolution: a simple approach for weakly-and semi-supervised semantic segmentation,","author":"Wei","year":"2018","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B73","doi-asserted-by":"publisher","first-page":"44247","DOI":"10.1109\/ACCESS.2019.2908991","article-title":"Nas-Unet: neural architecture search for medical image segmentation. IEEE","volume":"7","author":"Weng","year":"2019","journal-title":"Access"},{"journal-title":"Group Normalization","year":"2018","author":"Wu","key":"B74"},{"journal-title":"Contrastive Learning of Class-Agnostic Activation Map for Weakly Supervised Object Localization and Semantic Segmentation","year":"2022","author":"Xie","key":"B75"},{"key":"B76","article-title":"Learning to segment under various forms of weak supervision,","author":"Xu","year":"2015","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B77","first-page":"13706","article-title":"Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels.","author":"Xu","year":"2020","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition"},{"key":"B78","doi-asserted-by":"publisher","first-page":"638182","DOI":"10.3389\/fonc.2021.638182","article-title":"Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis. Front","volume":"11","author":"Yang","year":"2021","journal-title":"Oncol"},{"journal-title":"Multi-Scale Context Aggregation by Dilated Convolutions","year":"2015","author":"Yu","key":"B79"},{"key":"B80","first-page":"2340","article-title":"Re-labeling imagenet: from single to multi-labels, from global to localized labels.","author":"Yun","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B81","doi-asserted-by":"crossref","first-page":"12765","DOI":"10.1609\/aaai.v34i07.6971","article-title":"Reliability does matter: An end-to-end weakly supervised semantic segmentation approach.","author":"Zhang","year":"2020","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B82","article-title":"Adversarial complementary learning for weakly supervised object localization,","author":"Zhang","year":"2018","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B83","article-title":"Pyramid scene parsing network,","author":"Zhao","year":"2017","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"},{"key":"B84","article-title":"Learning deep features for discriminative localization,","author":"Zhou","year":"2016","journal-title":"Proceedings of the IEEE Conference Computer Visual Pattern Recognition (CVPR)"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1036934\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,8]],"date-time":"2022-11-08T07:29:15Z","timestamp":1667892555000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2022.1036934\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,8]]},"references-count":84,"alternative-id":["10.3389\/fcomp.2022.1036934"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2022.1036934","relation":{},"ISSN":["2624-9898"],"issn-type":[{"type":"electronic","value":"2624-9898"}],"subject":[],"published":{"date-parts":[[2022,11,8]]},"article-number":"1036934"}}