{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T04:46:47Z","timestamp":1773809207732,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2019,8,31]],"date-time":"2019-08-31T00:00:00Z","timestamp":1567209600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Young Top-notch Talent Program of Chinese Academy of Sciences","award":["QYZDB-SSW-JSC015"],"award-info":[{"award-number":["QYZDB-SSW-JSC015"]}]},{"name":"National Key R8D Program of China","award":["2017YFB0502900"],"award-info":[{"award-number":["2017YFB0502900"]}]},{"name":"CAS \u201cLight of West China\u201d Program","award":["XAB2017B15"],"award-info":[{"award-number":["XAB2017B15"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61772510 and 61702498"],"award-info":[{"award-number":["61772510 and 61702498"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Key Research Program of Frontier Sciences, CAS","award":["QYZDY-SSW-JSC044"],"award-info":[{"award-number":["QYZDY-SSW-JSC044"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"<jats:p>Recently, progress on semantic image segmentation is substantial, benefiting from the rapid development of Convolutional Neural Networks. Semantic image segmentation approaches proposed lately have been mostly based on Fully convolutional Networks (FCNs). However, these FCN-based methods use large receptive fields and too many pooling layers to depict the discriminative semantic information of the images. Specifically, on one hand, convolutional kernel with large receptive field smooth the detailed edges, since too much contexture information is used to depict the \u201ccenter pixel.\u201d However, the pooling layer increases the receptive field through zooming out the latest feature maps, which loses many detailed information of the image, especially in the deeper layers of the network. These operations often cause low spatial resolution inside deep layers, which leads to spatially fragmented prediction. To address this problem, we exploit the inherent multi-scale and pyramidal hierarchy of deep convolutional networks to extract the feature maps with different resolutions and take full advantages of these feature maps via a gradually stacked fusing way. Specifically, for two adjacent convolutional layers, we upsample the features from deeper layer with stride of 2 and then stack them on the features from shallower layer. Then, a convolutional layer with kernels of 1\u00d7 1 is followed to fuse these stacked features. The fused feature preserves the spatial structure information of the image; meanwhile, it owns strong discriminative capability for pixel classification. Additionally, to further preserve the spatial structure information and regional connectivity of the predicted category label map, we propose a novel loss term for the network. In detail, two graph model-based spatial affinity matrixes are proposed, which are used to depict the pixel-level relationships in the input image and predicted category label map respectively, and then their cosine distance is backward propagated to the network. The proposed architecture, called spatial structure preserving feature pyramid network, significantly improves the spatial resolution of the predicted category label map for semantic image segmentation. The proposed method achieves state-of-the-art results on three public and challenging datasets for semantic image segmentation.<\/jats:p>","DOI":"10.1145\/3321512","type":"journal-article","created":{"date-parts":[[2019,9,3]],"date-time":"2019-09-03T12:47:00Z","timestamp":1567514820000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":31,"title":["Spatial Structure Preserving Feature Pyramid Network for Semantic Image Segmentation"],"prefix":"10.1145","volume":"15","author":[{"given":"Yuan","family":"Yuan","sequence":"first","affiliation":[{"name":"Center for OPTical Imagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Shaanxi, China"}]},{"given":"Jie","family":"Fang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, China and University of Chinese Academy of Sciences, Beijing, China"}]},{"given":"Xiaoqiang","family":"Lu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Shaanxi, China"}]},{"given":"Yachuang","family":"Feng","sequence":"additional","affiliation":[{"name":"Key Laboratory of Spectral Imaging Technology CAS, Xi'an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Shaanxi, China"}]}],"member":"320","published-online":{"date-parts":[[2019,8,31]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.95"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2831627"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.314"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2017.7952376"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"e_1_2_1_6_1","volume-title":"Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587","author":"Chen Liang-Chieh","year":"2017"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.191"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.177"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.isprsjprs.2017.02.011"},{"key":"e_1_2_1_11_1","first-page":"815","article-title":"DeCAF: A deep convolutional activation feature for generic visual recognition","volume":"50","author":"Donahue Jeff","year":"2013","journal-title":"Comput. Sci."},{"key":"e_1_2_1_12_1","unstructured":"David Eigen Christian Puhrsch and Rob Fergus. 2014. Depth map prediction from a single image using a multi-scale deep network. Comput. Sci. (2014) 2366--2374.   David Eigen Christian Puhrsch and Rob Fergus. 2014. Depth map prediction from a single image using a multi-scale deep network. Comput. Sci. (2014) 2366--2374."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.167"},{"key":"e_1_2_1_14_1","volume-title":"Neural Network Nearest Neighbor Fields for Image Transforms","author":"Ganin Yaroslav"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the European Conference on Computer Vision. 519--534","author":"Ghiasi Golnaz"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.79"},{"key":"e_1_2_1_17_1","volume-title":"Learning rich features from RGB-D images for object detection and segmentation. 8695","author":"Gupta Saurabh","year":"2014"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126343"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 447--456","author":"Hariharan B."},{"key":"e_1_2_1_20_1","volume-title":"Margret Keuper, and Mario Fritz.","author":"He Yang","year":"2017"},{"key":"e_1_2_1_21_1","volume-title":"Speedup of deep learning ensembles for semantic segmentation using a model compression technique. Computer Vision and Image Understanding","author":"Holliday Andrew","year":"2017"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Sina Honari Jason Yosinski Pascal Vincent and Christopher Pal. 2016. Recombinator networks: Learning coarse-to-fine feature aggregation. In Computer Vision and Pattern Recognition. 5743--5752.  Sina Honari Jason Yosinski Pascal Vincent and Christopher Pal. 2016. Recombinator networks: Learning coarse-to-fine feature aggregation. In Computer Vision and Pattern Recognition. 5743--5752.","DOI":"10.1109\/CVPR.2016.619"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.349"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Conference on Machine Learning. 597--606","author":"Hong Seunghoon","year":"2015"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2261--2269","author":"Huang Gao"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2016.90"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2798282"},{"key":"e_1_2_1_29_1","unstructured":"Ronald Kemker Carl Salvaggio and Christopher Kanan. 2017. High-resolution multispectral dataset for semantic segmentation. (2017).  Ronald Kemker Carl Salvaggio and Christopher Kanan. 2017. High-resolution multispectral dataset for semantic segmentation. (2017)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.98"},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the International Conference on Neural Information Processing Systems. 1097--1105","author":"Krizhevsky Alex"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.348"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.106"},{"key":"e_1_2_1_34_1","volume-title":"Cheng Yang Fu, and Alexander C","author":"Liu Wei","year":"2016"},{"key":"e_1_2_1_35_1","volume-title":"Berg","author":"Liu Wei","year":"2015"},{"key":"e_1_2_1_36_1","first-page":"1337","article-title":"Fully convolutional networks for semantic segmentation","volume":"79","author":"Long Jonathan","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.178"},{"key":"e_1_2_1_39_1","unstructured":"Adam Paszke Abhishek Chaurasia Sangpil Kim and Eugenio Culurciello. 2016. ENet: A deep neural network architecture for real-time semantic segmentation. (2016).  Adam Paszke Abhishek Chaurasia Sangpil Kim and Eugenio Culurciello. 2016. ENet: A deep neural network architecture for real-time semantic segmentation. (2016)."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46448-0_5"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2017.2695523"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_1_43_1","volume-title":"OverFeat: Integrated recognition, localization and detection using convolutional networks. eprint Arxiv","author":"Sermanet Pierre","year":"2013"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3889--3898","author":"Sevillalara Laura"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2812600"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33715-4_54"},{"key":"e_1_2_1_47_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014).  Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Nasim Souly Concetto Spampinato and Mubarak Shah. 2017. Semi and weakly supervised semantic segmentation using generative adversarial network. (2017).  Nasim Souly Concetto Spampinato and Mubarak Shah. 2017. Semi and weakly supervised semantic segmentation using generative adversarial network. (2017).","DOI":"10.1109\/ICCV.2017.606"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Christian Szegedy Wei Liu Yangqing Jia Pierre Sermanet Scott Reed Dragomir Anguelov Dumitru Erhan Vincent Vanhoucke and Andrew Rabinovich. 2015. Going deeper with convolutions. In Computer Vision and Pattern Recognition. 1--9.  Christian Szegedy Wei Liu Yangqing Jia Pierre Sermanet Scott Reed Dragomir Anguelov Dumitru Erhan Vincent Vanhoucke and Andrew Rabinovich. 2015. Going deeper with convolutions. In Computer Vision and Pattern Recognition. 1--9.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-012-0574-z"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2018.00163"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2830098"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.59"},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the IEEE Sponsored International Conference on Knowledge and Smart Technologies.","author":"Yasrab Robail","year":"2017"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIVC.2016.7571265"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321512","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3321512","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:54:38Z","timestamp":1750204478000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3321512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,31]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3321512"],"URL":"https:\/\/doi.org\/10.1145\/3321512","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,31]]},"assertion":[{"value":"2018-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-08-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}