{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T04:11:59Z","timestamp":1769573519255,"version":"3.49.0"},"reference-count":49,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2020,8,24]],"date-time":"2020-08-24T00:00:00Z","timestamp":1598227200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Detecting objects such as aircraft and ships is a fundamental research area in remote sensing analytics. Owing to the prosperity and development of CNNs, many previous methodologies have been proposed for object detection within remote sensing images. Despite the advance, using the object detection datasets with a more complex structure, i.e., datasets with hierarchically multi-labeled objects, is limited to the existing detection models. Especially in remote sensing images, since objects are obtained from bird\u2019s-eye view, the objects are captured with restricted visual features and not always guaranteed to be labeled up to fine categories. We propose a hierarchical multi-label object detection framework applicable to hierarchically partial-annotated datasets. In the framework, an object detection pipeline called Decoupled Hierarchical Classification Refinement (DHCR) fuses the results of two networks: (1) an object detection network with multiple classifiers, and (2) a hierarchical sibling classification network for supporting hierarchical multi-label classification. Our framework additionally introduces a region proposal method for efficient detection on vain areas of the remote sensing images, called clustering-guided cropping strategy. Thorough experiments validate the effectiveness of our framework on our own object detection datasets constructed with remote sensing images from WorldView-3 and SkySat satellites. Under our proposed framework, DHCR-based detections significantly improve the performance of respective baseline models and we achieve state-of-the-art results on the datasets.<\/jats:p>","DOI":"10.3390\/rs12172734","type":"journal-article","created":{"date-parts":[[2020,8,24]],"date-time":"2020-08-24T10:37:32Z","timestamp":1598265452000},"page":"2734","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Hierarchical Multi-Label Object Detection Framework for Remote Sensing Images"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2331-2296","authenticated-orcid":false,"given":"Su-Jin","family":"Shin","sequence":"first","affiliation":[{"name":"Agency for Defense Development, Institute of Defense Advanced Technology Research, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seyeob","family":"Kim","sequence":"additional","affiliation":[{"name":"Agency for Defense Development, Institute of Defense Advanced Technology Research, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Youngjung","family":"Kim","sequence":"additional","affiliation":[{"name":"Agency for Defense Development, Institute of Defense Advanced Technology Research, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sungho","family":"Kim","sequence":"additional","affiliation":[{"name":"Agency for Defense Development, Institute of Defense Advanced Technology Research, Daejeon 34186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,8,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.image.2019.02.002","article-title":"Automatic detection of passable roads after floods in remote sensed and social media data","volume":"74","author":"Ahmad","year":"2019","journal-title":"Signal Process. Image Commun."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fu, K., Li, Y., Sun, H., Yang, X., Xu, G., Li, Y., and Sun, X. (2018). A ship rotation detection model in remote sensing images based on feature fusion pyramid network and deep reinforcement learning. Remote Sens., 10.","DOI":"10.3390\/rs10121922"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Safonova, A., Tabik, S., Alcaraz-Segura, D., Rubtsov, A., Maglinets, Y., and Herrera, F. (2019). Detection of fir trees (Abies sibirica) damaged by the bark beetle in unmanned aerial vehicle images with deep learning. Remote Sens., 11.","DOI":"10.3390\/rs11060643"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2327","DOI":"10.1109\/JSTARS.2013.2242846","article-title":"Airborne vehicle detection in dense urban areas using HoG features and disparity maps","volume":"6","author":"Tuermer","year":"2013","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhong, J., Lei, T., and Yao, G. (2017). Robust vehicle detection in aerial images based on cascaded convolutional neural networks. Sensors, 17.","DOI":"10.3390\/s17122720"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Munir, N., Awrangjeb, M., and Stantic, B. (2019, January 2\u20134). An Automated Method for Individual Wire Extraction from Power Line Corridor using LiDAR Data. Proceedings of the 2019 Digital Image Computing: Techniques and Applications (DICTA), Perth, Australia.","DOI":"10.1109\/DICTA47822.2019.8946085"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"89","DOI":"10.5194\/isprs-annals-IV-4-W4-89-2017","article-title":"A new mask for automatic building detection from high density point cloud data and multispectral imagery","volume":"4","author":"Awrangjeb","year":"2017","journal-title":"ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5553","DOI":"10.1109\/TGRS.2016.2569141","article-title":"Weakly supervised learning based on coupled convolutional neural networks for aircraft detection","volume":"54","author":"Zhang","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Xia, G.S., Bai, X., Ding, J., Zhu, Z., Belongie, S., Luo, J., Datcu, M., Pelillo, M., and Zhang, L. (2018, January 18\u201322). DOTA: A large-scale dataset for object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00418"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1074","DOI":"10.1109\/LGRS.2016.2565705","article-title":"Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds","volume":"13","author":"Liu","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7405","DOI":"10.1109\/TGRS.2016.2601622","article-title":"Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images","volume":"54","author":"Cheng","year":"2016","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_12","unstructured":"(2020, July 01). DigitalGlobe. Available online: http:\/\/www.digitalglobe.com\/."},{"key":"ref_13","unstructured":"(2020, July 01). Planet Labs. Available online: https:\/\/www.planet.com\/."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lin, Q., Zhao, J., Tong, Q., Zhang, G., Yuan, Z., and Fu, G. (2019, January 8\u201312). Cropping Region Proposal Network Based Framework for Efficient Object Detection on Large Scale Remote Sensing Images. Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME), Shanghai, China.","DOI":"10.1109\/ICME.2019.00265"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1109\/34.1000236","article-title":"Mean shift: A robust approach toward feature space analysis","volume":"24","author":"Comaniciu","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_16","unstructured":"Yang, F., Fan, H., Chu, P., Blasch, E., and Ling, H. (November, January 27). Clustered object detection in aerial images. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yan, Z., Zhang, H., Piramuthu, R., Jagadeesh, V., DeCoste, D., Di, W., and Yu, Y. (2015, January 7\u201313). HD-CNN: Hierarchical deep convolutional neural networks for large scale visual recognition. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.314"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ouyang, W., Wang, X., Zhang, C., and Yang, X. (2016, January 27\u201330). Factors in Finetuning deep model for object detection with long-tail distribution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.100"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Cheng, B., Wei, Y., Shi, H., Feris, R., Xiong, J., and Huang, T. (2018, January 8\u201314). Revisiting rcnn: On awakening the classification power of faster rcnn. Proceedings of the European Conference on Computer Vision (ECCV), Mubich, Germnay.","DOI":"10.1007\/978-3-030-01267-0_28"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ammour, N., Alhichri, H., Bazi, Y., Benjdira, B., Alajlan, N., and Zuair, M. (2017). Deep learning approach for car detection in UAV imagery. Remote Sens., 9.","DOI":"10.3390\/rs9040312"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Audebert, N., Le Saux, B., and Lef\u00e8vre, S. (2017). Segment-before-detect: Vehicle detection and classification through semantic segmentation of aerial images. Remote Sens., 9.","DOI":"10.3390\/rs9040368"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xu, Y., Zhu, M., Li, S., Feng, H., Ma, S., and Che, J. (2018). End-to-end airport detection in remote sensing images combining cascade region proposal networks and multi-threshold detection networks. Remote Sens., 10.","DOI":"10.3390\/rs10101516"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, F., Ren, R., Van de Voorde, T., Xu, W., Zhou, G., and Zhou, Y. (2018). Fast automatic airport detection in remote sensing images using convolutional neural networks. Remote Sens., 10.","DOI":"10.3390\/rs10030443"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yang, X., Sun, H., Fu, K., Yang, J., Sun, X., Yan, M., and Guo, Z. (2018). Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks. Remote Sens., 10.","DOI":"10.3390\/rs10010132"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, Z., Zhang, T., and Ouyang, C. (2018). End-to-end airplane detection using transfer learning in remote sensing images. Remote Sens., 10.","DOI":"10.3390\/rs10010139"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, W., Fu, H., Yu, L., and Cracknell, A. (2017). Deep learning based oil palm tree detection and counting for high-resolution remote sensing images. Remote Sens., 9.","DOI":"10.3390\/rs9010022"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5512","DOI":"10.1109\/TGRS.2019.2899955","article-title":"R2-CNN: Fast Tiny Object Detection in Large-scale Remote Sensing Images","volume":"57","author":"Pang","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, C., Yang, T., Zhu, S., Chen, C., and Guan, S. (2020, January 13\u201319). Density Map Guided Object Detection in Aerial Images. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00103"},{"key":"ref_31","unstructured":"Redmon, J., and Farhadi, A. (July, January USA). YOLO9000: Better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI."},{"key":"ref_32","unstructured":"Zhu, X., and Bain, M. (2017). B-CNN: Branch convolutional neural network for hierarchical classification. arXiv."},{"key":"ref_33","unstructured":"Parag, T., and Wang, H. (2020). Multilayer Dense Connections for Hierarchical Concept Classification. arXiv."},{"key":"ref_34","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (July, January 26). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_35","unstructured":"Redmon, J., and Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2592","DOI":"10.1109\/TII.2019.2937905","article-title":"Intelligent Embedded Vision for Summarization of Multiview Videos in IIoT","volume":"16","author":"Hussain","year":"2019","journal-title":"IEEE Trans. Ind. Informatics"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1109\/TII.2019.2929228","article-title":"Cloud-assisted multiview video summarization using CNN and bidirectional LSTM","volume":"16","author":"Hussain","year":"2019","journal-title":"IEEE Trans. Ind. Informatics"},{"key":"ref_38","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Tan, J., Wang, C., Li, B., Li, Q., Ouyang, W., Yin, C., and Yan, J. (2020, January 13\u201319). Equalization Loss for Long-Tailed Object Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01168"},{"key":"ref_40","unstructured":"Ruder, S. (2016). An overview of gradient descent optimization algorithms. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"50839","DOI":"10.1109\/ACCESS.2018.2869884","article-title":"Position detection and direction prediction for arbitrary-oriented ships via multitask rotation region convolutional neural network","volume":"6","author":"Yang","year":"2018","journal-title":"IEEE Access"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ding, J., Xue, N., Long, Y., Xia, G.S., and Lu, Q. (2019, January 15\u201321). Learning RoI transformer for oriented object detection in aerial images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00296"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"3111","DOI":"10.1109\/TMM.2018.2818020","article-title":"Arbitrary-oriented scene text detection via rotation proposals","volume":"20","author":"Ma","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Zhu, X., Wang, X., Yang, S., Li, W., Wang, H., Fu, P., and Luo, Z. (2017). R2cnn: Rotational region cnn for orientation robust scene text detection. arXiv.","DOI":"10.1109\/ICPR.2018.8545598"},{"key":"ref_45","unstructured":"Yang, X., Yang, J., Yan, J., Zhang, Y., Zhang, T., Guo, Z., Sun, X., and Fu, K. (November, January 27). Scrdet: Towards more robust detection for small, cluttered and rotated objects. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., He, K., and Doll\u00e1r, P. (2017, January 22\u201329). Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Koo, J., Seo, J., Jeon, S., Choe, J., and Jeon, T. (2018, January 6\u20139). RBox-CNN: Rotated bounding box based CNN for ship detection in remote sensing image. Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, WA, USA.","DOI":"10.1145\/3274895.3274915"},{"key":"ref_48","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 16), Savannah, GA, USA."},{"key":"ref_49","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/17\/2734\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:05:41Z","timestamp":1760177141000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/12\/17\/2734"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,24]]},"references-count":49,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["rs12172734"],"URL":"https:\/\/doi.org\/10.3390\/rs12172734","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,24]]}}}