{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T21:51:00Z","timestamp":1788904260860,"version":"build-2803163510"},"reference-count":40,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,22]],"date-time":"2021-12-22T00:00:00Z","timestamp":1640131200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2020YFC1523003"],"award-info":[{"award-number":["2020YFC1523003"]}]},{"name":"National Key R&amp;D Program of Hubei province","award":["2020BAB125"],"award-info":[{"award-number":["2020BAB125"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Although great success has been achieved in instance segmentation, accurate segmentation of instances remains difficult, especially at object edges. This problem is more prominent for instance segmentation in remote sensing imagery due to the diverse scales, variable illumination, smaller objects, and complex backgrounds. We find that most current instance segmentation networks do not consider the segmentation difficulty of different instances and different regions within the instance. In this paper, we study this problem and propose an ensemble method to segment instances from remote sensing images, considering the enhancement of hard-to-segment instances and instance edges. First, we apply a pixel-level Dice metric that reliably describes the segmentation quality of each instance to achieve online hard instance learning. Instances with low Dice values are studied with emphasis. Second, we generate a penalty map based on the analysis of boundary shapes to not only enhance the edges of objects but also discriminatively strengthen the edges of different shapes. That is, different areas of an object, such as internal areas, flat edges, and sharp edges, are distinguished and discriminatively weighed. Finally, the hard-to-segment instance learning and the shape-penalty map are integrated for precise instance segmentation. To evaluate the effectiveness and generalization ability of the proposed method, we train with the classic instance segmentation network Mask R-CNN and conduct experiments on two different types of remote sensing datasets: the iSAID-Reduce100 and the JKGW_WHU datasets, which have extremely different feature distributions and spatial resolutions. The comprehensive experimental results show that the proposed method improved the segmentation results by 2.78% and 1.77% in mask AP on the iSAID-Reduce100 and JKGW_WHU datasets, respectively. We also test other state-of-the-art (SOTA) methods that focus on inaccurate edges. Experiments demonstrate that our method outperforms these methods.<\/jats:p>","DOI":"10.3390\/rs14010023","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T02:02:57Z","timestamp":1640224977000},"page":"23","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Instance Segmentation in Very High Resolution Remote Sensing Imagery Based on Hard-to-Segment Instance Learning and Boundary Shape Analysis"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8233-3778","authenticated-orcid":false,"given":"Yiping","family":"Gong","sequence":"first","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fan","family":"Zhang","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangyang","family":"Jia","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8464-2558","authenticated-orcid":false,"given":"Zhu","family":"Mao","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xianfeng","family":"Huang","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"},{"name":"Institute of Yangtze River Civilization and Archaeology, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deren","family":"Li","sequence":"additional","affiliation":[{"name":"The State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,22]]},"reference":[{"key":"ref_1","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1016\/j.patcog.2017.05.001","article-title":"Deep patch learning for weakly supervised object classification and discovery","volume":"71","author":"Tang","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kwon, S. (2020). CLSTM: Deep Feature-Based Speech Emotion Recognition Using the Hierarchical ConvLSTM Network. Mathematics, 8.","DOI":"10.3390\/math8122133"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23\u201328). Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015, January 7\u201313). Fast R-CNN. Proceedings of the International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Shrivastava, A., Gupta, A., and Girshick, R.B. (2016, January 27\u201330). Training Region-Based Object Detectors with Online Hard Example Mining. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.89"},{"key":"ref_10","unstructured":"Dai, J., Li, Y., He, K., and Sun, J. (2016). R-FCN: Object Detection via Region-based Fully Convolutional Networks. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015\u201418th International Conference Munich, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_13","unstructured":"Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., and Yuille, A.L. (2016). DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2016, January 27\u201330). Instance-Aware Semantic Segmentation via Multi-task Network Cascades. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.343"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Dollar, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Huang, Z., Huang, L., Gong, Y., Huang, C., and Wang, X. (2019, January 15\u201320). Mask scoring r-cnn. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00657"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"534","DOI":"10.1007\/978-3-319-46466-4_32","article-title":"Instance-Sensitive Fully Convolutional Networks","volume":"Volume 9910","author":"Leibe","year":"2016","journal-title":"Proceedings of the Computer Vision\u2014ECCV 2016\u201414th European Conference"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Arnab, A., and Torr, P.H.S. (2017, January 21\u201326). Pixelwise Instance Segmentation with a Dynamically Instantiated Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.100"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Hermans, A., Papandreou, G., Schroff, F., Wang, P., and Adam, H. (2018, January 18\u201323). Masklab: Instance segmentation by refining object detection with semantic and direction features. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00422"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Takikawa, T., Acuna, D., Jampani, V., and Fidler, S. (November, January 27). Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00533"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chen, L., Barron, J.T., Papandreou, G., Murphy, K., and Yuille, A.L. (2016, January 27\u201330). Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.492"},{"key":"ref_23","unstructured":"Kervadec, H., Bouchtiba, J., Desrosiers, C., Granger, E., Dolz, J., and Ayed, I.B. (2019, January 8\u201310). Boundary loss for highly unbalanced segmentation. Proceedings of the International Conference on Medical Imaging with Deep Learning, PMLR, London, UK."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Hayder, Z., He, X., and Salzmann, M. (2017, January 21\u201326). Boundary-aware instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.70"},{"key":"ref_25","unstructured":"Chen, Z., Zhou, H., Xie, X., and Lai, J. (2019). Contour Loss: Boundary-Aware Learning for Salient Object Segmentation. arXiv."},{"key":"ref_26","unstructured":"Caliv\u00e1, F., Iriondo, C., Martinez, A.M., Majumdar, S., and Pedoia, V. (2019). Distance Map Loss Penalty Term for Semantic Segmentation. arXiv."},{"key":"ref_27","unstructured":"Pinheiro, P.O., Collobert, R., and Doll\u00e1r, P. (2015). Learning to segment object candidates. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, Y., Qi, H., Dai, J., Ji, X., and Wei, Y. (2017, January 21\u201326). Fully Convolutional Instance-Aware Semantic Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.472"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Levinkov, E., Andres, B., Savchynskyy, B., and Rother, C. (2017, January 21\u201326). InstanceCut: From Edges to Instances with MultiCut. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.774"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Arbel\u00e1ez, P.A., Pont-Tuset, J., Barron, J.T., Marqu\u00e9s, F., and Malik, J. (2014, January 23\u201328). Multiscale Combinatorial Grouping. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.49"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dai, J., He, K., and Sun, J. (2015, January 7\u201312). Convolutional feature masking for joint object and stuff segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299025"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Malisiewicz, T., Gupta, A., and Efros, A.A. (2011, January 6\u201313). Ensemble of exemplar-SVMs for object detection and beyond. Proceedings of the IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126229"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1627","DOI":"10.1109\/TPAMI.2009.167","article-title":"Object Detection with Discriminatively Trained Part-Based Models","volume":"32","author":"Felzenszwalb","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Doll\u00e1r, P., Tu, Z., Perona, P., and Belongie, S.J. (2009, January 7\u201310). Integral Channel Features. Proceedings of the British Machine Vision Conference, BMVC 2009, London, UK.","DOI":"10.5244\/C.23.91"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cheng, T., Wang, X., Huang, L., and Liu, W. (2020, January 23\u201328). Boundary-preserving mask r-cnn. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58568-6_39"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Milletari, F., Navab, N., and Ahmadi, S.A. (2016, January 25\u201328). V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.79"},{"key":"ref_37","unstructured":"Wu, S., and Li, X. (2019). Iou-balanced loss functions for single-stage object detection. arXiv."},{"key":"ref_38","unstructured":"Waqas Zamir, S., Arora, A., Gupta, A., Khan, S., Sun, G., Shahbaz Khan, F., Zhu, F., Shao, L., Xia, G.S., and Bai, X. (2019, January 15\u201320). isaid: A large-scale dataset for instance segmentation in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA."},{"key":"ref_39","unstructured":"Abdulla, W. (2019, June 20). Mask R-CNN for Object Detection and Instance Segmentation on Keras and TensorFlow. Available online: https:\/\/github.com\/matterport\/Mask_RCNN."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., and Guadarrama, S. (2017, January 21\u201326). Speed\/accuracy trade-offs for modern convolutional object detectors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.351"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/23\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:51:03Z","timestamp":1760169063000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/1\/23"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,22]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["rs14010023"],"URL":"https:\/\/doi.org\/10.3390\/rs14010023","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,22]]}}}