{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T12:59:41Z","timestamp":1740142781441,"version":"3.37.3"},"reference-count":60,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62090020","61672499"],"award-info":[{"award-number":["62090020","61672499"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association of Chinese Academy of Sciences","doi-asserted-by":"publisher","award":["2013073"],"award-info":[{"award-number":["2013073"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Strategic Priority Research Program of Chinese Academy of Sciences","award":["XDC05030200"],"award-info":[{"award-number":["XDC05030200"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,6,24]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The accurate representation of objects holds pivotal significance in the realm of panoptic segmentation. Presently, prevalent object representation methodologies, including box-based, keypoint-based and query-based techniques, encounter a challenge known as the \u2018representation confusion\u2019 issue in specific scenarios, often resulting in the mislabeling of instances. In response, this paper introduces Convex Object Representation (COR), a straightforward yet highly effective approach to address this problem. COR leverages a CNN-based Euclidean Distance Transform to convert the target instance into a convex heatmap. Simultaneously, it offers a parallel embedding method for encoding the object. Subsequently, COR characterizes objects based on the distinctive embedding vectors of their convex vertices. This paper seamlessly integrates COR into a state-of-the-art query-based panoptic segmentation framework. Experimental findings validate that COR successfully mitigates the representation confusion predicament, enhancing segmentation accuracy. The COR-augmented methods exhibit notable improvements of +1.3 and +0.7 points in PQ on the Cityscapes validation and MS COCO panoptic 2017 validation datasets, respectively.<\/jats:p>","DOI":"10.1093\/comjnl\/bxad119","type":"journal-article","created":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T22:23:53Z","timestamp":1703197433000},"page":"2009-2019","source":"Crossref","is-referenced-by-count":0,"title":["Panoptic Segmentation with Convex Object Representation"],"prefix":"10.1093","volume":"67","author":[{"given":"Zhicheng","family":"Yao","sequence":"first","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences , No. 6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190 , China"},{"name":"University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408 , China"}]},{"given":"Sa","family":"Wang","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences , No. 6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190 , China"},{"name":"University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408 , China"}]},{"given":"Jinbin","family":"Zhu","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences , No. 6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190 , China"}]},{"given":"Yungang","family":"Bao","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences , No. 6 Kexueyuan South Road Zhongguancun, Haidian District, Beijing 100190 , China"},{"name":"University of Chinese Academy of Sciences , No. 1 Yanqihu East Rd, Huairou District, Beijing 101408 , China"}]}],"member":"286","published-online":{"date-parts":[[2023,12,20]]},"reference":[{"key":"2024062414104995000_ref1","first-page":"9396","article-title":"Panoptic segmentation","volume-title":"CVPR","author":"Kirillov","year":"2019"},{"key":"2024062414104995000_ref2","first-page":"6165","article-title":"An end-to-end network for panoptic segmentation","volume-title":"CVPR","author":"Liu","year":"2019"},{"key":"2024062414104995000_ref3","first-page":"8810","article-title":"Upsnet: A unified panoptic segmentation network","volume-title":"CVPR","author":"Xiong","year":"2019"},{"key":"2024062414104995000_ref4","first-page":"10326","article-title":"K-net: Towards unified image segmentation","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Zhang","year":"2021"},{"article-title":"Masked-attention mask transformer for universal image segmentation","year":"2021","author":"Cheng","key":"2024062414104995000_ref5"},{"key":"2024062414104995000_ref6","first-page":"12472","article-title":"Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation","volume-title":"CVPR","author":"Cheng","year":"2020"},{"key":"2024062414104995000_ref7","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1109\/TPAMI.2018.2844175","article-title":"Mask r-cnn","volume":"42","author":"He","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2024062414104995000_ref8","first-page":"17864","article-title":"Per-pixel classification is not all you need for semantic segmentation","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Cheng","year":"2021"},{"key":"2024062414104995000_ref9","first-page":"736","article-title":"Rich feature hierarchies for accurate object detection and semantic segmentation","volume-title":"CVPR","author":"Girshick","year":"2014"},{"key":"2024062414104995000_ref10","first-page":"3431","article-title":"Fully convolutional networks for semantic segmentation","volume-title":"CVPR","author":"Long","year":"2014"},{"key":"2024062414104995000_ref11","first-page":"9626","article-title":"FCOS: Fully convolutional one-stage object detection","volume-title":"ICCV","author":"Tian","year":"2019"},{"article-title":"Deep watershed transform for instance segmentation","year":"2016","author":"Bai","key":"2024062414104995000_ref12"},{"article-title":"Towards bounding-box free panoptic segmentation","year":"2020","author":"Bonde","key":"2024062414104995000_ref13"},{"key":"2024062414104995000_ref14","article-title":"Combined object categorization and segmentation with an implicit shape model","volume-title":"ECCV\u201904 Workshop on Statistical Learning in Computer Vision","author":"Leibe","year":"2004"},{"key":"2024062414104995000_ref15","first-page":"642","article-title":"Ssap: Single-shot instance segmentation with affinity pyramid","volume-title":"ICCV","author":"Gao","year":"2019"},{"key":"2024062414104995000_ref16","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-01264-9_45","article-title":"Cornernet: Detecting objects as paired keypoints","volume-title":"ECCV","author":"Law","year":"2018"},{"article-title":"Deeperlab: single-shot image parser","year":"2019","author":"Yang","key":"2024062414104995000_ref17"},{"key":"2024062414104995000_ref18","first-page":"6568","article-title":"Centernet: Keypoint triplets for object detection","volume-title":"ICCV","author":"Duan","year":"2019"},{"key":"2024062414104995000_ref19","first-page":"1799","article-title":"Joint training of a convolutional network and a graphical model for human pose estimation","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Tompson","year":"2014"},{"key":"2024062414104995000_ref20","first-page":"6000","article-title":"Attention is all you need","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Vaswani","year":"2017"},{"article-title":"Proposal-free network for instance-level object segmentation","year":"2015","author":"Liang","key":"2024062414104995000_ref21"},{"key":"2024062414104995000_ref22","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-45886-1_2","article-title":"Pixel-level encoding and depth layering for instance-level semantic labeling","volume-title":"the 38th Annual Symposium of the German Association for Pattern Recognition","author":"Uhrig","year":"2016"},{"article-title":"Associative embedding: end-to-end learning for joint detection and grouping","year":"2017","author":"Newell","key":"2024062414104995000_ref23"},{"key":"2024062414104995000_ref24","first-page":"12190","article-title":"Polarmask: Single shot instance segmentation with polar representation","volume-title":"CVPR","author":"Xie","year":"2019"},{"key":"2024062414104995000_ref25","first-page":"78","article-title":"Dppd: Deformable polar polygon object detection","volume-title":"CVPR Workshops","author":"Zheng","year":"2023"},{"key":"2024062414104995000_ref26","first-page":"62","article-title":"Keypoints-aware object detection","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Jaiswal","year":"2021"},{"key":"2024062414104995000_ref27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-01264-9_17","article-title":"Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model","volume-title":"ECCV","author":"Papandreou","year":"2018"},{"key":"2024062414104995000_ref28","first-page":"850","article-title":"Extremenet: Bottom-up object detection by grouping extreme and center points","volume-title":"CVPR","author":"Zhou","year":"2019"},{"key":"2024062414104995000_ref29","first-page":"213","article-title":"End-to-end object detection with transformers","volume-title":"ECCV","author":"Carion","year":"2020"},{"key":"2024062414104995000_ref30","first-page":"2550","article-title":"Cmt-deeplab: Clustering mask transformers for panoptic segmentation","volume-title":"CVPR","author":"Yu","year":"2022"},{"key":"2024062414104995000_ref31","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV51070.2023.00371","article-title":"Segment anything","author":"Kirillov","year":"2023"},{"key":"2024062414104995000_ref32","first-page":"8748","article-title":"Learning transferable visual models from natural language supervision","volume-title":"ICML","author":"Radford","year":"2021"},{"article-title":"Llama: open and efficient foundation language models","year":"2023","author":"Touvron","key":"2024062414104995000_ref33"},{"key":"2024062414104995000_ref34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1322432.1322434","article-title":"2d euclidean distance transform algorithms: a comparative survey","volume":"40","author":"Fabbri","year":"2008","journal-title":"ACM Comput. Surv"},{"article-title":"Panonet: real-time panoptic segmentation through position-sensitive feature embedding","year":"2020","author":"Chen","key":"2024062414104995000_ref35"},{"article-title":"Deformable convnets v2: more deformable, better results","year":"2018","author":"Zhu","key":"2024062414104995000_ref36"},{"article-title":"Semantic instance segmentation with a discriminative loss function","year":"2017","author":"Brabandere","key":"2024062414104995000_ref37"},{"key":"2024062414104995000_ref38","first-page":"3213","article-title":"The cityscapes dataset for semantic urban scene understanding","volume-title":"CVPR","author":"Cordts","year":"2016"},{"key":"2024062414104995000_ref39","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"Microsoft coco: Common objects in context","volume-title":"ECCV","author":"Lin","year":"2014"},{"key":"2024062414104995000_ref40","first-page":"6392","article-title":"Panoptic feature pyramid networks","volume-title":"CVPR","author":"Kirillov","year":"2019"},{"key":"2024062414104995000_ref41","first-page":"8520","article-title":"Real-time panoptic segmentation from dense detections","volume-title":"CVPR","author":"Hou","year":"2020"},{"key":"2024062414104995000_ref42","first-page":"7019","article-title":"Attention-guided unified network for panoptic segmentation","volume-title":"CVPR","author":"Li","year":"2019"},{"key":"2024062414104995000_ref43","first-page":"8269","article-title":"Seamless scene segmentation","volume-title":"CVPR","author":"Porzi","year":"2019"},{"key":"2024062414104995000_ref44","doi-asserted-by":"crossref","first-page":"1551","DOI":"10.1007\/s11263-021-01445-z","article-title":"Efficientps: efficient panoptic segmentation","volume":"129","author":"Mohan","year":"2020","journal-title":"Int. J. Comput. Vis."},{"article-title":"Learning to fuse things and stuff","year":"2018","author":"Li","key":"2024062414104995000_ref45"},{"key":"2024062414104995000_ref46","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.01333","article-title":"Unifying training and inference for panoptic segmentation","author":"Li","year":"2020"},{"key":"2024062414104995000_ref47","first-page":"15635","article-title":"Combinatorial optimization for panoptic segmentation: A fully differentiable approach","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Abbas","year":"2021"},{"key":"2024062414104995000_ref48","doi-asserted-by":"crossref","DOI":"10.1109\/ICCV.2019.00745","article-title":"Adaptis: adaptive instance selection network","author":"Sofiiuk","year":"2019"},{"article-title":"Fully convolutional networks for panoptic segmentation","year":"2020","author":"Li","key":"2024062414104995000_ref49"},{"key":"2024062414104995000_ref50","first-page":"2477","article-title":"Refine: Prediction fusion network for panoptic segmentation","volume-title":"Proc. of the AAAI Conf. on Artificial Intelligence","author":"Ren","year":"2021"},{"article-title":"Max-deeplab: end-to-end panoptic segmentation with mask transformers","year":"2020","author":"Wang","key":"2024062414104995000_ref51"},{"article-title":"Panoptic segformer","year":"2021","author":"Li","key":"2024062414104995000_ref52"},{"article-title":"Detectron2","year":"2019","author":"Wu","key":"2024062414104995000_ref53"},{"key":"2024062414104995000_ref54","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"CVPR","author":"He","year":"2016"},{"key":"2024062414104995000_ref55","article-title":"Adam: A method for stochastic optimization","volume-title":"ICLR","author":"Kingma","year":"2015"},{"key":"2024062414104995000_ref56","doi-asserted-by":"crossref","first-page":"3349","DOI":"10.1109\/TPAMI.2020.2983686","article-title":"Deep high-resolution representation learning for visual recognition","volume":"43","author":"Wang","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2024062414104995000_ref57","first-page":"9992","article-title":"Swin transformer: Hierarchical vision transformer using shifted windows","volume-title":"ICCV","author":"Liu","year":"2021"},{"key":"2024062414104995000_ref58","first-page":"6890","article-title":"Instances as queries","volume-title":"ICCV","author":"Fang","year":"2021"},{"key":"2024062414104995000_ref59","first-page":"8026","article-title":"Pytorch: An imperative style, high-performance deep learning library","volume-title":"Conf. and Workshop on Neural Information Processing Systems","author":"Paszke","year":"2019"},{"author":"The opencv library","key":"2024062414104995000_ref60"}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/67\/6\/2009\/58309225\/bxad119.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/67\/6\/2009\/58309225\/bxad119.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,24]],"date-time":"2024-06-24T14:43:54Z","timestamp":1719240234000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/67\/6\/2009\/7484595"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,20]]},"references-count":60,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,12,20]]},"published-print":{"date-parts":[[2024,6,24]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxad119","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"type":"print","value":"0010-4620"},{"type":"electronic","value":"1460-2067"}],"subject":[],"published-other":{"date-parts":[[2024,6]]},"published":{"date-parts":[[2023,12,20]]}}}