{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T21:45:40Z","timestamp":1779399940478,"version":"3.53.1"},"reference-count":56,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T00:00:00Z","timestamp":1691539200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62076127"],"award-info":[{"award-number":["62076127"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Rock detection on the surface of celestial bodies is critical in the deep space environment for obstacle avoidance and path planning of space probes. However, in the remote and complex deep environment, rocks have the characteristics of irregular shape, being similar to the background, sparse pixel characteristics, and being easy for light and dust to affect. Most existing methods face significant challenges to attain high accuracy and low computational complexity in rock detection. In this paper, we propose a novel semantic segmentation network based on a hybrid framework combining CNN and transformer for deep space rock images, namely RockSeg. The network includes a multiscale low-level feature fusion (MSF) module and an efficient backbone network for feature extraction to achieve the effective segmentation of the rocks. Firstly, in the network encoder, we propose a new backbone network (Resnet-T) that combines the part of the Resnet backbone and the transformer block with a multi-headed attention mechanism to capture the global context information. Additionally, a simple and efficient multiscale feature fusion module is designed to fuse low-level features at different scales to generate richer and more detailed feature maps. In the network decoder, these feature maps are integrated with the output feature maps to obtain more precise semantic segmentation results. Finally, we conduct experiments on two deep space rock datasets: the MoonData and MarsData datasets. The experimental results demonstrate that the proposed model outperforms state-of-the-art rock detection algorithms under the conditions of low computational complexity and fast inference speed.<\/jats:p>","DOI":"10.3390\/rs15163935","type":"journal-article","created":{"date-parts":[[2023,8,9]],"date-time":"2023-08-09T10:21:50Z","timestamp":1691576510000},"page":"3935","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["RockSeg: A Novel Semantic Segmentation Network Based on a Hybrid Framework Combining a Convolutional Neural Network and Transformer for Deep Space Rock Images"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9506-3762","authenticated-orcid":false,"given":"Lili","family":"Fan","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiabin","family":"Yuan","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuewei","family":"Niu","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keke","family":"Zha","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weiqi","family":"Ma","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MAES.2021.3115897","article-title":"NASA Space Robotics Challenge 2 Qualification Round: An Approach to Autonomous Lunar Rover Operations","volume":"36","author":"Kilic","year":"2021","journal-title":"IEEE Aerosp. Electron. Syst. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kuang, B., Wisniewski, M., Rana, Z.A., and Zhao, Y. (2021). Rock Segmentation in the Navigation Vision of the Planetary Rovers. Mathematics, 9.","DOI":"10.3390\/math9233048"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.actaastro.2021.12.030","article-title":"Autonomous navigation for deep space small satellites: Scientific and technological advances","volume":"193","author":"Turan","year":"2022","journal-title":"Acta Astronaut."},{"key":"ref_4","unstructured":"Furl\u00e1n, F., Rubio, E., Sossa, H., and Ponce, V. Rock detection in a Mars-like environment using a CNN. Proceedings of the Mexican Conference on Pattern Recognition."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3334492","article-title":"RockFormer: A U-shaped Transformer Network for Martian Rock Segmentation","volume":"61","author":"Liu","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Brockers, R., Delaune, J., Proen\u00e7a, P., Schoppmann, P., Domnik, M., Kubiak, G., and Tzanetos, T. (2021, January 6\u201313). Autonomous safe landing site detection for a future mars science helicopter. Proceedings of the 2021 IEEE Aerospace Conference (50100), Big Sky, MT, USA.","DOI":"10.1109\/AERO50100.2021.9438289"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Fan, L., Yuan, J., Zha, K., and Wang, X. (2022). ELCD: Efficient Lunar Crater Detection Based on Attention Mechanisms and Multiscale Feature Fusion Networks from Digital Elevation Models. Remote Sens., 14.","DOI":"10.3390\/rs14205225"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ebadi, K., Coble, K., Kogan, D., Atha, D., Schwartz, R., Padgett, C., and Vander Hook, J. (2022, January 5\u201312). Semantic mapping in unstructured environments: Toward autonomous localization of planetary robotic explorers. Proceedings of the 2022 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO53065.2022.9843550"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1002\/rob.22054","article-title":"On the role of feature and signal selection for terrain learning in planetary exploration robots","volume":"39","author":"Ugenti","year":"2022","journal-title":"J. Field Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.actaastro.2020.03.007","article-title":"Path planning for asteroid hopping rovers with pre-trained deep reinforcement learning architectures","volume":"171","author":"Jiang","year":"2020","journal-title":"Acta Astronaut."},{"key":"ref_11","first-page":"1","article-title":"Semi-Supervised Learning for Mars Imagery Classification and Segmentation","volume":"19","author":"Wang","year":"2023","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_12","unstructured":"Gui, C., and Li, Z. (2013). Emerging Technologies for Information Systems, Computing, and Management, Springer."},{"key":"ref_13","first-page":"329","article-title":"Rockster: Onboard rock segmentation through edge regrouping","volume":"13","author":"Burl","year":"2016","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1029\/2017JE005496","article-title":"Analysis of rock abundance on lunar surface from orbital and descent images using automatic rock detection","volume":"123","author":"Li","year":"2018","journal-title":"J. Geophys. Res. Planets"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1016\/j.asr.2017.04.028","article-title":"Autonomous rock detection on mars through region contrast","volume":"60","author":"Xiao","year":"2017","journal-title":"Adv. Space Res."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xiao, X., Cui, H., Yao, M., Fu, Y., and Qi, W. (2018, January 8\u201313). Auto rock detection via sparse-based background modeling for mars rover. Proceedings of the 2018 IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, Brazil.","DOI":"10.1109\/CEC.2018.8477665"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3335","DOI":"10.1109\/TNNLS.2021.3131206","article-title":"A kernel-based multi-featured rock modeling and detection framework for a mars rover","volume":"34","author":"Xiao","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Goh, E., Ward, I.R., Vincent, G., Pak, K., Chen, J., and Wilson, B. (2023, January 4\u201311). Self-supervised Distillation for Computer Vision Onboard Planetary Robots. Proceedings of the 2023 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO55745.2023.10115598"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"105371","DOI":"10.1016\/j.pss.2021.105371","article-title":"Terrain classification-based rover traverse planner with kinematic constraints for Mars exploration","volume":"209","author":"Huang","year":"2021","journal-title":"Planet. Space Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.neucom.2019.02.003","article-title":"Survey on semantic segmentation using deep learning techniques","volume":"338","author":"Lateef","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_21","first-page":"1","article-title":"A stepwise domain adaptive segmentation network with covariate shift alleviation for remote sensing imagery","volume":"60","author":"Li","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_22","unstructured":"Yu, F., and Koltun, V. (2015). Multi-scale context aggregation by dilated convolutions. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","volume":"37","author":"He","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-decoder with atrous separable convolution for semantic image segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yuan, Y., Chen, X., and Wang, J. (2020, January 23\u201328). Object-contextual representations for semantic segmentation. Proceedings of the Computer Vision\u2014ECCV 2020: 16th European Conference, Glasgow, UK. Proceedings, Part VI 16.","DOI":"10.1007\/978-3-030-58539-6_11"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/j.neucom.2020.05.070","article-title":"A survey on U-shaped networks in medical image segmentations","volume":"409","author":"Liu","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_29","unstructured":"Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., and Wang, M. Swin-unet: Unet-like pure transformer for medical image segmentation. Proceedings of the European Conference on Computer Vision."},{"key":"ref_30","first-page":"755","article-title":"Automatic Rocks Segmentation Based on Deep Learning for Planetary Rover Images","volume":"18","author":"Li","year":"2021","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_31","unstructured":"Ma, W., Jiabin, Y., Zha, k., and Fan, L. (2023). Computer Science, China Academic Journal Electronic Publish House."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15\u201320). Deep high-resolution representation learning for human pose estimation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dai, Y., Zheng, T., Xue, C., and Zhou, L. (2022). SegMarsViT: Lightweight Mars Terrain Segmentation Network for Autonomous Driving in Planetary Exploration. Remote Sens., 14.","DOI":"10.3390\/rs14246297"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/MRA.2021.3134875","article-title":"Design, testing, and evolution of mars rover testbeds: European space agency planetary exploration","volume":"29","author":"Azkarate","year":"2022","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"107099","DOI":"10.1016\/j.cmpb.2022.107099","article-title":"Efficient combination of CNN and transformer for dual-teacher uncertainty-guided semi-supervised medical image segmentation","volume":"226","author":"Xiao","year":"2022","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"119024","DOI":"10.1016\/j.eswa.2022.119024","article-title":"CSwin-PNet: A CNN-Swin Transformer combined pyramid network for breast lesion segmentation in ultrasound images","volume":"213","author":"Yang","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_37","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3297373","article-title":"Self-supervised Pre-training via Multi-modality Images with Transformer for Change Detection","volume":"61","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3297373","article-title":"Asymmetric cross-attention hierarchical network based on CNN and transformer for bitemporal remote sensing images change detection","volume":"61","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_40","unstructured":"Luo, H., Wang, P., Xu, Y., Ding, F., Zhou, Y., Wang, F., Li, H., and Jin, R. (2021). Self-supervised pre-training for transformer-based person re-identification. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Fang, J., Lin, H., Chen, X., and Zeng, K. (2022, January 19\u201320). A hybrid network of cnn and transformer for lightweight image super-resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00119"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wagstaff, K., Lu, Y., Stanboli, A., Grimes, K., Gowda, T., and Padams, J. (2018, January 2\u20137). Deep mars: Cnn classification of mars imagery for the pds imaging atlas. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11404"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1007\/s12145-019-00433-9","article-title":"Autonomous Martian rock image classification based on transfer deep learning methods","volume":"13","author":"Li","year":"2020","journal-title":"Earth Sci. Inform."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1016\/j.actaastro.2022.08.002","article-title":"A hybrid attention semantic segmentation network for unstructured terrain on Mars","volume":"204","author":"Liu","year":"2023","journal-title":"Acta Astronaut."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1007\/s12145-022-00890-9","article-title":"MRISNet: Deep-learning-based Martian instance segmentation against blur","volume":"16","author":"Liu","year":"2023","journal-title":"Earth Sci. Inform."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Panambur, T., Chakraborty, D., Meyer, M., Milliken, R., Learned-Miller, E., and Parente, M. (2022, January 19\u201320). Self-supervised learning to guide scientifically relevant categorization of martian terrain images. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPRW56347.2022.00138"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Goh, E., Chen, J., and Wilson, B. (2022, January 5\u201312). Mars terrain segmentation with less labels. Proceedings of the 2022 IEEE Aerospace Conference (AERO), Big Sky, MT, USA.","DOI":"10.1109\/AERO53065.2022.9843245"},{"key":"ref_48","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_49","unstructured":"Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., and Liu, T. (2020, January 13\u201318). On layer normalization in the transformer architecture. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"6169","DOI":"10.1109\/TGRS.2020.3026051","article-title":"MAP-Net: Multiple attending path neural network for building footprint extraction from remote sensed imagery","volume":"59","author":"Zhu","year":"2020","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.neucom.2022.07.071","article-title":"Multi-level features fusion via cross-layer guided attention for hyperspectral pansharpening","volume":"506","author":"Hou","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Bock, S., and Wei\u00df, M. (2019, January 14\u201319). A proof of local convergence for the Adam optimizer. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.","DOI":"10.1109\/IJCNN.2019.8852239"},{"key":"ref_53","unstructured":"Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., and Liu, W. (November, January 27). Ccnet: Criss-cross attention for semantic segmentation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republich of Korea."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., and Lu, H. (2019, January 15\u201320). Dual attention network for scene segmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00326"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.jrmge.2021.05.004","article-title":"Real-time prediction of rock mass classification based on TBM operation big data and stacking technique of ensemble learning","volume":"14","author":"Hou","year":"2022","journal-title":"J. Rock Mech. Geotech. Eng."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1016\/j.jrmge.2021.12.011","article-title":"Slope stability prediction using ensemble learning techniques: A case study in Yunyang County, Chongqing, China","volume":"14","author":"Zhang","year":"2022","journal-title":"J. Rock Mech. Geotech. Eng."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/16\/3935\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:28:21Z","timestamp":1760128101000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/16\/3935"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,9]]},"references-count":56,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["rs15163935"],"URL":"https:\/\/doi.org\/10.3390\/rs15163935","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,9]]}}}