{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T14:49:36Z","timestamp":1772722176646,"version":"3.50.1"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T00:00:00Z","timestamp":1729468800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Panoptic segmentation plays a crucial role in enabling robots to comprehend their surroundings, providing fine-grained scene understanding information for robots' intelligent tasks. Although existing methods have made some progress, they are prone to fail in areas with weak textures, small objects, etc. Inspired by biological vision research, we propose a cascaded contour-enhanced panoptic segmentation network called CCPSNet, attempting to enhance the discriminability of instances through structural knowledge. To acquire the scene structure, a cascade contour detection stream is designed, which extracts comprehensive scene contours using channel regulation structural perception module and coarse-to-fine cascade strategy. Furthermore, the contour-guided multi-scale feature enhancement stream is developed to boost the discrimination ability for small objects and weak textures. The stream integrates contour information and multi-scale context features through structural-aware feature modulation module and inverse aggregation technique. Experimental results show that our method improves accuracy on the Cityscapes (61.2 PQ) and COCO (43.5 PQ) datasets while also demonstrating robustness in challenging simulated real-world complex scenarios faced by robots, such as dirty cameras and rainy conditions. The proposed network promises to help the robot perceive the real scene. In future work, an unsupervised training strategy for the network could be explored to reduce the training cost.<\/jats:p>","DOI":"10.3389\/fnbot.2024.1489021","type":"journal-article","created":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T05:10:21Z","timestamp":1729487421000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Cascade contour-enhanced panoptic segmentation for robotic vision perception"],"prefix":"10.3389","volume":"18","author":[{"given":"Yue","family":"Xu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Runze","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongchen","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lili","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaolin","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiamao","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,10,21]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1398703","DOI":"10.3389\/fnbot.2024.1398703","article-title":"Remote intelligent perception system for multi-object detection","volume":"18","author":"Alazeb","year":"2024","journal-title":"Front. Neurorobot"},{"key":"B2","first-page":"213","article-title":"\u201cEnd-to-end object detection with transformers,\u201d","volume-title":"European conference on computer vision","author":"Carion","year":"2020"},{"key":"B3","doi-asserted-by":"publisher","first-page":"103736","DOI":"10.1016\/j.jvcir.2022.103736","article-title":"Se-psnet: Silhouette-based enhancement feature for panoptic segmentation network","volume":"90","author":"Chang","year":"2023","journal-title":"J. Vis. Commun. Image Represent"},{"key":"B4","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1109\/TPAMI.2017.2699184","article-title":"Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","volume":"40","author":"Chen","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B5","doi-asserted-by":"publisher","first-page":"2288","DOI":"10.1109\/TCSVT.2020.3020257","article-title":"Spatialflow: bridging all tasks for panoptic segmentation","volume":"31","author":"Chen","year":"2020","journal-title":"IEEE Trans. Circ. Syst. Video Technol"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00385","article-title":"\u201cBanet: bidirectional aggregation network with occlusion handling for panoptic segmentation,\u201d","author":"Chen","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01249","article-title":"\u201cPanoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation,\u201d","author":"Cheng","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B8","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1016\/j.ins.2020.09.058","article-title":"Analysis of activation maps through global pooling measurements for texture classification","volume":"555","author":"Condori","year":"2021","journal-title":"Inf. Sci"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.350","article-title":"\u201cThe cityscapes dataset for semantic urban scene understanding,\u201d","author":"Cordts","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B10","article-title":"Panoptic segmentation with a joint semantic and instance segmentation network","author":"De Geus","year":"2018","journal-title":"arXiv preprint arXiv:1809.02110"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00073","article-title":"\u201cSsap: single-shot instance segmentation with affinity pyramid,\u201d","author":"Gao","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B12","doi-asserted-by":"publisher","first-page":"6013","DOI":"10.1109\/TIP.2021.3090522","article-title":"Learning category-and instance-aware pixel embedding for fast panoptic segmentation","volume":"30","author":"Gao","year":"2021","journal-title":"IEEE Trans. Image Proc"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322","article-title":"\u201cMask R-CNN,\u201d","author":"He","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01647","article-title":"\u201cLpsnet: a lightweight solution for fast panoptic segmentation,\u201d","author":"Hong","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01709","article-title":"\u201cYou only segment once: towards real-time panoptic segmentation,\u201d","author":"Hu","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B17","unstructured":"Jung\n              A. B.\n            \n            \n              Wada\n              K.\n            \n            \n              Crall\n              J.\n            \n            \n              Tanaka\n              S.\n            \n            \n              Graving\n              J.\n            \n            \n              Reinders\n              C.\n            \n          \n          Imgaug\n          \n          2020"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00656","article-title":"\u201cPanoptic feature pyramid networks,\u201d","author":"Kirillov","year":"","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00963","article-title":"\u201cPanoptic segmentation,\u201d","author":"Kirillov","year":"","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01073","article-title":"\u201cLearning instance occlusion for panoptic segmentation,\u201d","author":"Lazarow","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B21","article-title":"Learning to fuse things and stuff","author":"Li","year":"2018","journal-title":"arXiv preprint arXiv:1812.01192"},{"key":"B22","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1007\/978-3-030-58520-4_26","article-title":"\u201cImproving semantic segmentation via decoupled body and edge supervision,\u201d","volume-title":"Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XVII 16","author":"Li","year":"2020"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00719","article-title":"\u201cAttention-guided unified network for panoptic segmentation,\u201d","author":"Li","year":"2019","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1487","DOI":"10.1109\/TIP.2023.3234499","article-title":"IDNet: information decomposition network for fast panoptic segmentation","volume":"33","author":"Lin","year":"2023","journal-title":"IEEE Trans. Image Proc"},{"key":"B25","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"\u201cMicrosoft coco: common objects in context,\u201d","volume-title":"Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13","author":"Lin","year":"2014"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00633","article-title":"\u201cAn end-to-end network for panoptic segmentation,\u201d","author":"Liu","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B27","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3389\/fnbot.2018.00064","article-title":"Faster R-cnn for robust pedestrian detection using semantic segmentation network","volume":"12","author":"Liu","year":"2018","journal-title":"Front. Neurorobot"},{"key":"B28","article-title":"Perceptual video quality assessment: a survey","author":"Min","year":"2024","journal-title":"arXiv preprint arXiv:2402.03413"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2996463","article-title":"Fixation prediction through multimodal analysis","volume":"13","author":"Min","year":"2016","journal-title":"ACM Trans. Multim. Comput. Commun. Applic"},{"key":"B30","doi-asserted-by":"publisher","first-page":"3805","DOI":"10.1109\/TIP.2020.2966082","article-title":"A multimodal saliency model for videos with high audio-visual correspondence","volume":"29","author":"Min","year":"2020","journal-title":"IEEE Trans. Image Proc"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1551","DOI":"10.1007\/s11263-021-01445-z","article-title":"Efficientps: efficient panoptic segmentation","volume":"129","author":"Mohan","year":"2021","journal-title":"Int. J. Comput. Vis"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00745","article-title":"\u201cAdaptis: adaptive instance selection network,\u201d","author":"Sofiiuk","year":"2019","journal-title":"Proceedings of the IEEE\/CVF international conference on computer vision"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/WACV56688.2023.00395","article-title":"\u201cPRN: panoptic refinement network,\u201d","author":"Sun","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00533","article-title":"\u201cGated-SCNN: gated shape cnns for semantic segmentation,\u201d","author":"Takikawa","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01079","article-title":"\u201cEfficientdet: scalable and efficient object detection,\u201d","author":"Tan","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00542","article-title":"\u201cMax-deeplab: end-to-end panoptic segmentation with mask transformers,\u201d","author":"Wang","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B37","first-page":"108","article-title":"\u201cAxial-deeplab: stand-alone axial-attention for panoptic segmentation,\u201d","volume-title":"European Conference on Computer Vision","author":"Wang","year":"2020"},{"key":"B38","first-page":"17721","article-title":"Solov2: Dynamic and fast instance segmentation","volume":"33","author":"Wang","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.164","article-title":"\u201cHolistically-nested edge detection,\u201d","author":"Xie","year":"2015","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00902","article-title":"\u201cUpsnet: a unified panoptic segmentation network,\u201d","author":"Xiong","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-88007-1_7","article-title":"\u201cContour-aware panoptic segmentation network,\u201d","author":"Xu","year":"2021","journal-title":"Pattern Recognition and Computer Vision: 4th Chinese Conference, PRCV 2021, Beijing, China, October 29-November 1, 2021, Proceedings, Part II"},{"key":"B42","doi-asserted-by":"publisher","first-page":"978225","DOI":"10.3389\/fnbot.2022.978225","article-title":"Dual-flow network with attention for autonomous driving","volume":"16","author":"Yang","year":"2023","journal-title":"Front. Neurorobot"},{"key":"B43","article-title":"Deeperlab: single-shot image parser","author":"Yang","year":"2019","journal-title":"arXiv preprint arXiv:1902.05093"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1204418","DOI":"10.3389\/fnbot.2023.1204418","article-title":"Based on cross-scale fusion attention mechanism network for semantic segmentation for street scenes","volume":"17","author":"Ye","year":"2023","journal-title":"Front. Neurorobot"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00259","article-title":"\u201cCMT-deeplab: clustering mask transformers for panoptic segmentation,\u201d","author":"Yu","year":"","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B46","first-page":"288","article-title":"\u201cK-means mask transformer,\u201d","volume-title":"European Conference on Computer Vision","author":"Yu","year":""},{"key":"B47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11432-019-2757-1","article-title":"Perceptual image quality assessment: a survey","volume":"63","author":"Zhai","year":"2020","journal-title":"Sci. China Inform. Sci"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1075520","DOI":"10.3389\/fnbot.2022.1075520","article-title":"A lightweight multi-dimension dynamic convolutional network for real-time semantic segmentation","volume":"16","author":"Zhang","year":"2022","journal-title":"Front. Neurorobot"},{"key":"B49","doi-asserted-by":"publisher","first-page":"1119231","DOI":"10.3389\/fnbot.2023.1119231","article-title":"Rethinking 1D convolution for lightweight semantic segmentation","volume":"17","author":"Zhang","year":"2023","journal-title":"Front. Neurorobot"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.660","article-title":"\u201cPyramid scene parsing network,\u201d","author":"Zhao","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01368","article-title":"\u201cJoint semantic segmentation and boundary detection using iterative pyramid contexts,\u201d","author":"Zhen","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B52","doi-asserted-by":"publisher","first-page":"6594","DOI":"10.1523\/JNEUROSCI.20-17-06594.2000","article-title":"Coding of border ownership in monkey visual cortex","volume":"20","author":"Zhou","year":"2000","journal-title":"J. Neurosci"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1489021\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,21]],"date-time":"2024-10-21T05:10:26Z","timestamp":1729487426000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2024.1489021\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,21]]},"references-count":52,"alternative-id":["10.3389\/fnbot.2024.1489021"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2024.1489021","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,21]]},"article-number":"1489021"}}