{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:20:39Z","timestamp":1740108039324,"version":"3.37.3"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"27","license":[{"start":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T00:00:00Z","timestamp":1689724800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T00:00:00Z","timestamp":1689724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"crossref","award":["DP150104251"],"award-info":[{"award-number":["DP150104251"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001801","name":"University of Western Australia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001801","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Weakly supervised semantic segmentation (WSSS) commonly relies on Class Activation Mapping (CAM) to produce pseudo semantic labels using image-level annotations. However, because CAM maps often form sparse object regions with poor boundaries, they cannot provide sufficient segmentation supervision. Because off-the-shelf saliency maps can provide rich object boundaries that can be leveraged to improve semantic segmentation, we propose to jointly learn semantic segmentation and class-agnostic masks by using image-level annotations and off-the-shelf saliency maps as supervision. We also propose a cross-task label refinement mechanism, which takes advantage of the learned class-agnostic masks and semantic segmentation masks, to refine the pseudo labels and provide more accurate supervision to both tasks. Moreover, we introduce a new normalization method for CAM to generate more complete class-specific localization maps. The improved CAM maps complement our learned class-agnostic masks, leading to high-quality pseudo semantic segmentation labels. Extensive experiments demonstrate the effectiveness of the proposed approach, with state-of-the-art WSSS results established on PASCAL VOC 2012 and MS COCO.<\/jats:p>","DOI":"10.1007\/s00521-023-08826-0","type":"journal-article","created":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T09:03:26Z","timestamp":1689757406000},"page":"20189-20205","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Learning class-agnostic masks with cross-task refinement for weakly supervised semantic segmentation"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1759-2941","authenticated-orcid":false,"given":"Lian","family":"Xu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammed","family":"Bennamoun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Farid","family":"Boussaid","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wanli","family":"Ouyang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dan","family":"Xu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,7,19]]},"reference":[{"issue":"4","key":"8826_CR1","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","volume":"40","author":"L-C Chen","year":"2018","unstructured":"Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. PAMI 40(4):834\u2013848","journal-title":"PAMI"},{"key":"8826_CR2","doi-asserted-by":"crossref","unstructured":"Zhang L, Xu D, Arnab A, HS Torr P (2020) Dynamic graph message passing networks. In: CVPR","DOI":"10.1109\/CVPR42600.2020.00378"},{"key":"8826_CR3","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3043781","author":"D Xu","year":"2020","unstructured":"Xu D, Alameda-Pineda X, Ouyang W, Ricci E, Wang X, Sebe N (2020) Probabilistic graph attention network with conditional kernels for pixel-wise prediction. TPAMI. https:\/\/doi.org\/10.1109\/TPAMI.2020.3043781","journal-title":"TPAMI"},{"issue":"7","key":"8826_CR4","doi-asserted-by":"publisher","first-page":"1949","DOI":"10.1007\/s00521-019-04491-4","volume":"32","author":"L Zhang","year":"2020","unstructured":"Zhang L, Sheng Z, Li Y, Sun Q, Zhao Y, Feng D (2020) Image object detection and semantic segmentation based on convolutional neural network. Neural Comput Appl 32(7):1949\u20131958","journal-title":"Neural Comput Appl"},{"issue":"5","key":"8826_CR5","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1007\/s00521-017-3158-6","volume":"29","author":"F Jiang","year":"2018","unstructured":"Jiang F, Grigorev A, Rho S, Tian Z, Fu Y, Jifara W, Adil K, Liu S (2018) Medical image semantic segmentation based on deep learning. Neural Comput Appl 29(5):1257\u20131265","journal-title":"Neural Comput Appl"},{"issue":"17","key":"8826_CR6","doi-asserted-by":"publisher","first-page":"10737","DOI":"10.1007\/s00521-020-04870-2","volume":"33","author":"T Meraj","year":"2021","unstructured":"Meraj T, Rauf HT, Zahoor S, Hassan A, Lali MI, Ali L, Bukhari SAC, Shoaib U (2021) Lung nodules detection using semantic segmentation and classification with optimal features. Neural Comput Appl 33(17):10737\u201310750","journal-title":"Neural Comput Appl"},{"key":"8826_CR7","doi-asserted-by":"crossref","unstructured":"Bearman A, Russakovsky O, Ferrari V, Fei-Fei L (2016) What\u2019s the point: semantic segmentation with point supervision. In: ECCV","DOI":"10.1007\/978-3-319-46478-7_34"},{"key":"8826_CR8","doi-asserted-by":"crossref","unstructured":"Lin D, Dai J, Jia J, He K, Sun J (2016) Scribblesup: scribble-supervised convolutional networks for semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR.2016.344"},{"key":"8826_CR9","doi-asserted-by":"crossref","unstructured":"Tang M, Djelouah A, Perazzi F, Boykov Y, Schroers C (2018) Normalized cut loss for weakly-supervised cnn segmentation. In: CVPR","DOI":"10.1109\/CVPR.2018.00195"},{"key":"8826_CR10","doi-asserted-by":"crossref","unstructured":"Hu R, Doll\u00e1r P, He K, Darrell T, Girshick R (2018) Learning to segment every thing. In: CVPR","DOI":"10.1109\/CVPR.2018.00445"},{"key":"8826_CR11","doi-asserted-by":"crossref","unstructured":"Song C, Huang Y, Ouyang W, Wang L (2019) Box-driven class-wise region masking and filling rate guided loss for weakly supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR.2019.00325"},{"issue":"2","key":"8826_CR12","first-page":"470","volume":"16","author":"L Zhang","year":"2014","unstructured":"Zhang L, Gao Y, Xia Y, Lu K, Shen J, Ji R (2014) Representative discovery of structure cues for weakly-supervised image segmentation. TMM 16(2):470\u2013479","journal-title":"TMM"},{"key":"8826_CR13","doi-asserted-by":"crossref","unstructured":"Zhang T, Lin G, Cai J, Shen T, Shen, C, Kot AC (2019) Decoupled spatial neural attention for weakly supervised semantic segmentation. TMM","DOI":"10.1109\/TMM.2019.2914870"},{"key":"8826_CR14","first-page":"1035","volume":"23","author":"L Zhou","year":"2021","unstructured":"Zhou L, Gong C, Liu Z, Fu K (2021) Sal: selection and attention losses for weakly supervised semantic segmentation. TMM 23:1035\u20131048","journal-title":"TMM"},{"key":"8826_CR15","doi-asserted-by":"crossref","unstructured":"Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: CVPR","DOI":"10.1109\/CVPR.2016.319"},{"key":"8826_CR16","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: ICCV","DOI":"10.1109\/ICCV.2017.74"},{"key":"8826_CR17","doi-asserted-by":"crossref","unstructured":"Huang Z, Wang X, Wang J, Liu W, Wang J (2018) Weakly-supervised semantic segmentation network with deep seeded region growing. In: CVPR","DOI":"10.1109\/CVPR.2018.00733"},{"key":"8826_CR18","doi-asserted-by":"crossref","unstructured":"Fan J, Zhang Z, Song C, Tan T (2020) Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR42600.2020.00434"},{"key":"8826_CR19","doi-asserted-by":"crossref","unstructured":"Sun G, Wang W, Dai J, Van Gool L (2020) Mining cross-image semantics for weakly supervised semantic segmentation. In: ECCV","DOI":"10.1007\/978-3-030-58536-5_21"},{"key":"8826_CR20","doi-asserted-by":"crossref","unstructured":"Wei Y, Xiao H, Shi H, Jie Z, Feng J, Huang TS (2018) Revisiting dilated convolution: a simple approach for weakly-and semi-supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR.2018.00759"},{"key":"8826_CR21","doi-asserted-by":"crossref","unstructured":"Jiang, P.-T., Hou, Q., Cao, Y., Cheng, M.-M., Wei, Y., Xiong, H.-K.: Integral object mining via online attention accumulation. In: ICCV (2019)","DOI":"10.1109\/ICCV.2019.00216"},{"key":"8826_CR22","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.neucom.2020.09.045","volume":"421","author":"L Xu","year":"2021","unstructured":"Xu L, Xue H, Bennamoun M, Boussaid F, Sohel F (2021) Atrous convolutional feature network for weakly supervised semantic segmentation. Neurocomputing 421:115\u2013126","journal-title":"Neurocomputing"},{"key":"8826_CR23","doi-asserted-by":"crossref","unstructured":"Wei Y, Feng J, Liang X., Cheng MM, Zhao Y, Yan S (2017) Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: CVPR","DOI":"10.1109\/CVPR.2017.687"},{"key":"8826_CR24","unstructured":"Hou Q, Jiang P, Wei Y, Cheng MM (2018) Self-erasing network for integral object attention. In: NeurIPS"},{"key":"8826_CR25","doi-asserted-by":"crossref","unstructured":"Li K, Wu Z, Peng KC, Ernst J, Fu Y Tell me where to look: Guided attention inference network. In: CVPR (2018)","DOI":"10.1109\/CVPR.2018.00960"},{"key":"8826_CR26","doi-asserted-by":"crossref","unstructured":"Wang Y, Zhang J, Kan M, Shan S, Chen X (2020) Self-supervised equivariant attention mechanism for weakly supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR42600.2020.01229"},{"key":"8826_CR27","doi-asserted-by":"crossref","unstructured":"Chang YT, Wang Q, Hung WC, Piramuthu R, Tsai YH, Yang MH (2020) Weakly-supervised semantic segmentation via sub-category exploration. In: CVPR","DOI":"10.1109\/CVPR42600.2020.00901"},{"key":"8826_CR28","doi-asserted-by":"crossref","unstructured":"Fan J, Zhang Z, Tan T, Song C, Xiao J (2020) Cian: Cross-image affinity net for weakly supervised semantic segmentation. In: AAAI","DOI":"10.1609\/aaai.v34i07.6705"},{"key":"8826_CR29","unstructured":"Zhang D, Zhang H, Tang J, Hua X, Sun Q(2020) Causal intervention for weakly-supervised semantic segmentation. In: NeurIPS"},{"key":"8826_CR30","doi-asserted-by":"crossref","unstructured":"Wang X, You S, Li X, Ma H (2018) Weakly-supervised semantic segmentation by iteratively mining common object features. In: CVPR","DOI":"10.1109\/CVPR.2018.00147"},{"issue":"6","key":"8826_CR31","doi-asserted-by":"publisher","first-page":"1736","DOI":"10.1007\/s11263-020-01293-3","volume":"128","author":"X Wang","year":"2020","unstructured":"Wang X, Liu S, Ma H, Yang M-H (2020) Weakly-supervised semantic segmentation by iterative affinity learning. IJCV 128(6):1736\u20131749","journal-title":"IJCV"},{"key":"8826_CR32","doi-asserted-by":"crossref","unstructured":"Araslanov N, Roth S (2020) Single-stage semantic segmentation from image labels. In: CVPR","DOI":"10.1109\/CVPR42600.2020.00431"},{"key":"8826_CR33","doi-asserted-by":"crossref","unstructured":"Lee J, Kim E, Lee S, Lee J, Yoon S (2019) Ficklenet: Weakly and semi-supervised segmentation using stochastic inference. In: CVPR","DOI":"10.1109\/CVPR.2019.00541"},{"key":"8826_CR34","doi-asserted-by":"crossref","unstructured":"Xu Y, Xu D, Hong X, Ouyang W, Ji R, Zhao G (2019) Structured modeling of joint deep feature and prediction refinement for salient object detection. In: ICCV","DOI":"10.1109\/ICCV.2019.00389"},{"key":"8826_CR35","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.patcog.2019.01.006","volume":"90","author":"Z Wu","year":"2019","unstructured":"Wu Z, Shen C, Van Den Hengel A (2019) Wider or deeper: revisiting the resnet model for visual recognition. Pattern Recogn 90:119\u2013133","journal-title":"Pattern Recogn"},{"key":"8826_CR36","doi-asserted-by":"crossref","unstructured":"Ahn J, Kwak S(2018) Learning pixel-level semantic affinity with image-level supervision for weakly supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR.2018.00523"},{"key":"8826_CR37","doi-asserted-by":"crossref","unstructured":"Zhang J, Yu X, Li A, Song P, Liu B, Dai Y (2020) Weakly-supervised salient object detection via scribble annotations. In: CVPR","DOI":"10.1109\/CVPR42600.2020.01256"},{"issue":"4","key":"8826_CR38","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1109\/TPAMI.2018.2815688","volume":"41","author":"Q Hou","year":"2019","unstructured":"Hou Q, Cheng M, Hu X, Borji A, Tu Z, Torr P (2019) Deeply supervised salient object detection with short connections. PAMI 41(4):815\u2013828","journal-title":"PAMI"},{"issue":"2","key":"8826_CR39","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","volume":"88","author":"M Everingham","year":"2010","unstructured":"Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. IJCV 88(2):303\u2013338","journal-title":"IJCV"},{"key":"8826_CR40","doi-asserted-by":"crossref","unstructured":"Lin TY, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Doll\u00e1r P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: ECCV","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"8826_CR41","unstructured":"Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2015) Semantic image segmentation with deep convolutional nets and fully connected crfs. In: ICLR"},{"key":"8826_CR42","doi-asserted-by":"crossref","unstructured":"Kolesnikov A, Lampert CH (2016) Seed, expand and constrain: Three principles for weakly-supervised image segmentation. In: ECCV","DOI":"10.1007\/978-3-319-46493-0_42"},{"key":"8826_CR43","doi-asserted-by":"crossref","unstructured":"Hariharan B, Arbel\u00e1ez P, Bourdev L, Maji S, Malik J (2011) Semantic contours from inverse detectors. In: ICCV","DOI":"10.1109\/ICCV.2011.6126343"},{"key":"8826_CR44","doi-asserted-by":"crossref","unstructured":"Xu D, Ouyang W, Wang X, Sebe N (2018) Pad-net: Multi-tasks guided prediciton-and-distillation network for simultaneous depth estimation and scene parsing. In: CVPR","DOI":"10.1109\/CVPR.2018.00077"},{"key":"8826_CR45","doi-asserted-by":"crossref","unstructured":"Zhang B, Xiao J, Wei Y, Sun M, Huang K (2020) Reliability does matter: an end-to-end weakly supervised semantic segmentation approach. In: AAAI","DOI":"10.1609\/aaai.v34i07.6971"},{"key":"8826_CR46","doi-asserted-by":"crossref","unstructured":"Luo W, Yang M (2020)Learning saliency-free model with generic features for weakly-supervised semantic segmentation. In: AAAI","DOI":"10.1609\/aaai.v34i07.6842"},{"key":"8826_CR47","doi-asserted-by":"crossref","unstructured":"Zhang T, Lin G, Liu W, Cai J, Kot A (2020) Splitting vs. merging: Mining object regions with discrepancy and intersection loss for weakly supervised semantic segmentation. In: ECCV","DOI":"10.1007\/978-3-030-58542-6_40"},{"key":"8826_CR48","doi-asserted-by":"crossref","unstructured":"Yao Y, Chen T, Xie GS, Zhang C, Shen F, Wu Q, Tang Z, Zhang J (2021) Non-salient region object mining for weakly supervised semantic segmentation. In: CVPR","DOI":"10.1109\/CVPR46437.2021.00265"},{"key":"8826_CR49","doi-asserted-by":"crossref","unstructured":"Xu L, Ouyang W, Bennamoun M, Boussaid F, Sohel F, Xu D (2021) Leveraging auxiliary tasks with affinity learning for weakly supervised semantic segmentation. In: ICCV","DOI":"10.1109\/ICCV48922.2021.00690"},{"key":"8826_CR50","doi-asserted-by":"crossref","unstructured":"Al-Huda Z, Peng B, Yang Y, Algburi RNA, Ahmad M, Khurshid F, Moghalles K (2021) Weakly supervised semantic segmentation by iteratively refining optimal segmentation with deep cues guidance. Neural Comput Appl, pp 1\u201326","DOI":"10.1007\/s00521-020-05669-x"},{"key":"8826_CR51","unstructured":"Paszke A, Gross S, Chintala S, Chanan G (2017) Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration"},{"key":"8826_CR52","doi-asserted-by":"crossref","unstructured":"Liu N, Han J (2016) Dhsnet: Deep hierarchical saliency network for salient object detection. In: CVPR","DOI":"10.1109\/CVPR.2016.80"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08826-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-023-08826-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08826-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,30]],"date-time":"2023-08-30T00:12:14Z","timestamp":1693354334000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-023-08826-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,19]]},"references-count":52,"journal-issue":{"issue":"27","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["8826"],"URL":"https:\/\/doi.org\/10.1007\/s00521-023-08826-0","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"type":"print","value":"0941-0643"},{"type":"electronic","value":"1433-3058"}],"subject":[],"published":{"date-parts":[[2023,7,19]]},"assertion":[{"value":"1 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 June 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}