{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T10:56:21Z","timestamp":1787828181332,"version":"build-2784847793"},"reference-count":52,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2025,11,1]],"date-time":"2025-11-01T00:00:00Z","timestamp":1761955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key R&D Program of Shandong Province, China","award":["2024TSGC0658"],"award-info":[{"award-number":["2024TSGC0658"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U23A20346"],"award-info":[{"award-number":["U23A20346"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62403162"],"award-info":[{"award-number":["62403162"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Accurate pose estimation of non-cooperative space objects is crucial for applications such as satellite maintenance, space debris removal, and on-orbit assembly. However, monocular pose estimation methods face significant challenges in environments with limited visibility. Different from the traditional pose estimation methods that use images from a single band as input, we propose a novel deep learning-based pose estimation framework for non-cooperative space objects by fusing visible and infrared images. First, we introduce an image fusion subnetwork that integrates multi-scale features from visible and infrared images into a unified embedding space, preserving the detailed features of visible images and the intensity information of infrared images. Subsequently, we design a robust pose estimation subnetwork that leverages the rich information from the fused images to achieve accurate pose estimation. By combining these two subnetworks, we construct the Visible and Infrared Fused Pose Estimation Framework (VIPE) for non-cooperative space objects. Additionally, we present a Bimodal-Vision Pose Estimation (BVPE) dataset, comprising 3,630 visible-infrared image pairs, to facilitate research in this domain. Extensive experiments on the BVPE dataset demonstrate that VIPE significantly outperforms existing monocular pose estimation methods, particularly in complex space environments, providing more reliable and accurate pose estimation results.<\/jats:p>","DOI":"10.3390\/s25216664","type":"journal-article","created":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T18:21:46Z","timestamp":1762194106000},"page":"6664","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["VIPE: Visible and Infrared Fused Pose Estimation Framework for Space Noncooperative Objects"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0307-5641","authenticated-orcid":false,"given":"Zhao","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9106-9542","authenticated-orcid":false,"given":"Dong","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-7826-1681","authenticated-orcid":false,"given":"Yuhui","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weizhao","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guanghui","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8687-5606","authenticated-orcid":false,"given":"Yuekan","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Mechanical & Electronic Engineering, Shandong University of Science and Technology, Qingdao 266590, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1016\/j.asr.2022.09.027","article-title":"Pose measurement and motion estimation of non-cooperative satellite based on spatial circle feature","volume":"71","author":"Zhang","year":"2023","journal-title":"Adv. Space Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1016\/j.actaastro.2024.09.055","article-title":"Service spacecraft for space debris removal","volume":"225","author":"Svotina","year":"2024","journal-title":"Acta Astronaut."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.1016\/j.asr.2024.05.011","article-title":"PVSPE: A pyramid vision multitask transformer network for spacecraft pose estimation","volume":"74","author":"Yang","year":"2024","journal-title":"Adv. Space Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101055","DOI":"10.1016\/j.paerosci.2024.101055","article-title":"Robotic manipulators for in-orbit servicing and active debris removal: Review and comparison","volume":"151","author":"Rybus","year":"2024","journal-title":"Prog. Aerosp. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4902","DOI":"10.1109\/TAES.2022.3211246","article-title":"Space Noncooperative Object Active Tracking With Deep Reinforcement Learning","volume":"58","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ye, R., Wang, L., Ren, Y., Wang, Y., Chen, X., and Liu, Y. (2023). FilterformerPose: Satellite Pose Estimation Using Filterformer. Sensors, 23.","DOI":"10.3390\/s23208633"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.actaastro.2023.08.001","article-title":"A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects","volume":"212","author":"Pauly","year":"2023","journal-title":"Acta Astronaut."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.paerosci.2017.07.001","article-title":"A review of cooperative and uncooperative spacecraft pose determination techniques for close-proximity operations","volume":"93","author":"Opromolla","year":"2017","journal-title":"Prog. Aerosp. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.actaastro.2021.07.023","article-title":"2D vision-based tracking algorithm for general space non-cooperative objects","volume":"188","author":"Zhou","year":"2021","journal-title":"Acta Astronaut."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Han, H., Kim, H., and Bang, H. (2022). Monocular Pose Estimation of an Uncooperative Spacecraft Using Convexity Defect Features. Sensors, 22.","DOI":"10.3390\/s22218541"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1504\/IJSPACESE.2014.060600","article-title":"Pose estimation of an uncooperative spacecraft from actual space imagery","volume":"2","author":"Amico","year":"2014","journal-title":"Int. J. Space Sci. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3036","DOI":"10.1109\/TAES.2014.120757","article-title":"Relative pose estimation for cylinder-shaped spacecrafts using single image","volume":"50","author":"Liu","year":"2014","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1414","DOI":"10.2514\/1.A34124","article-title":"Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous","volume":"55","author":"Sharma","year":"2018","journal-title":"J. Spacecr. Rocket."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rondao, D., and Aouf, N. (2018, January 8\u201312). Multi-View Monocular Pose Estimation for Spacecraft Relative Navigation. Proceedings of the 2018 AIAA Guidance, Navigation, and Control Conference, Kissimmee, FL, USA.","DOI":"10.2514\/6.2018-2100"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive Image Features from Scale-Invariant Keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cviu.2007.09.014","article-title":"SURF: Speeded up robust features","volume":"110","author":"Bay","year":"2008","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, S., Yang, W., Wang, W., Mai, J., Liang, J., and Zhang, X. (2024). Spacecraft Homography Pose Estimation with Single-Stage Deep Convolutional Neural Network. Sensors, 24.","DOI":"10.3390\/s24061828"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., and Hu, Q. (2020, January 13\u201319). ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.01155"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4083","DOI":"10.1109\/TAES.2020.2989063","article-title":"Satellite Pose Estimation Challenge: Dataset, Competition Design, and Results","volume":"56","author":"Kisantal","year":"2020","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Park, T.H., Martens, M., Lecuyer, G., Izzo, D., and D\u2019Amico, S. (2022, January 5\u201312). SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap. Proceedings of the 2022 IEEE Aerospace Conference (AERO), Big Sky, MT, USA.","DOI":"10.1109\/AERO53065.2022.9843439"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10491","DOI":"10.4249\/scholarpedia.10491","article-title":"Scale Invariant Feature Transform","volume":"7","author":"Lindeberg","year":"2012","journal-title":"Scholarpedia"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_23","unstructured":"Park, T.H., Sharma, S., and D\u2019Amico, S. (2019). Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, B., Cao, J., Parra, A., and Chin, T.J. (2019, January 27\u201328). Satellite Pose Estimation with Deep Landmark Regression and Nonlinear Pose Refinement. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00343"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, K., Xiao, B., Liu, D., and Wang, J. (2019, January 15\u201320). Deep High-Resolution Representation Learning for Human Pose Estimation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00584"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Pasqualetto Cassinis, L., Fonod, R., Gill, E., Ahrns, I., and Gil Fernandez, J. (2020, January 6\u201310). CNN-Based Pose Estimation System for Close-Proximity Operations Around Uncooperative Spacecraft. Proceedings of the AIAA Scitech 2020 Forum, Orlando, FL, USA.","DOI":"10.2514\/6.2020-1457"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"216283","DOI":"10.1109\/ACCESS.2020.3041415","article-title":"Fast and Accurate Spacecraft Pose Estimation From Single Shot Space Imagery Using Box Reliability and Keypoints Existence Judgments","volume":"8","author":"Huo","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4279","DOI":"10.1109\/TAES.2022.3161605","article-title":"Revisiting Monocular Satellite Pose Estimation With Transformer","volume":"58","author":"Wang","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2555","DOI":"10.1109\/TAES.2023.3332075","article-title":"DTSE-SpaceNet: Deformable-Transformer-Based Single-Stage End-to-End Network for 6-D Pose Estimation in Space","volume":"60","author":"Liu","year":"2024","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1016\/j.actaastro.2024.08.027","article-title":"A comprehensive study on PnP-based pipeline for pose estimation of noncooperative satellite","volume":"224","author":"Yu","year":"2024","journal-title":"Acta Astronaut."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sharma, S., Beierle, C., and D\u2019Amico, S. (2018, January 3\u201310). Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks. Proceedings of the 2018 IEEE Aerospace Conference, Big Sky, MT, USA.","DOI":"10.1109\/AERO.2018.8396425"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Proenca, P.F., and Gao, Y. (August, January 31). Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197244"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Garcia, A., Musallam, M.A., Gaudilliere, V., Ghorbel, E., Al Ismaeil, K., Perez, M., and Aouada, D. (2021, January 19\u201325). LSPnet: A 2D Localization-oriented Spacecraft Pose Estimation Neural Network. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00233"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"24579","DOI":"10.1109\/JSEN.2021.3115844","article-title":"Non-Model-Based Monocular Pose Estimation Network for Uncooperative Spacecraft Using Convolutional Neural Network","volume":"21","author":"Huang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sharma, S., and D\u2019Amico, S. (2019). Pose Estimation for Non-Cooperative Rendezvous Using Neural Networks. arXiv.","DOI":"10.1109\/AERO.2018.8396425"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5726","DOI":"10.1016\/j.asr.2023.03.036","article-title":"Robust multi-task learning and online refinement for spacecraft pose estimation across domain gap","volume":"73","author":"Park","year":"2024","journal-title":"Adv. Space Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2228","DOI":"10.1177\/09544100221143565","article-title":"An image fusion algorithm based on iterative wavelet transform for space non-cooperative targets","volume":"237","author":"Zhang","year":"2023","journal-title":"Proc. Inst. Mech. Eng. Part G J. Aerosp. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"057006","DOI":"10.1117\/1.OE.52.5.057006","article-title":"Dictionary learning method for joint sparse representation-based image fusion","volume":"52","author":"Zhang","year":"2013","journal-title":"Opt. Eng."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.inffus.2019.07.011","article-title":"IFCNN: A general image fusion framework based on convolutional neural network","volume":"54","author":"Zhang","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1109\/TMM.2020.2997127","article-title":"AttentionFGAN: Infrared and Visible Image Fusion Using Attention-Based Generative Adversarial Networks","volume":"23","author":"Li","year":"2021","journal-title":"IEEE Trans. Multimed."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1109\/TIP.2018.2887342","article-title":"DenseFuse: A Fusion Approach to Infrared and Visible Images","volume":"28","author":"Li","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","first-page":"937","article-title":"ChiNet: Deep Recurrent Convolutional Learning for Multimodal Spacecraft Pose Estimation","volume":"59","author":"Rondao","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"5030914","DOI":"10.1109\/TIM.2023.3325520","article-title":"DCTNet: A Heterogeneous Dual-Branch Multi-Cascade Network for Infrared and Visible Image Fusion","volume":"72","author":"Li","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/TPAMI.2020.3012548","article-title":"U2Fusion: A Unified Unsupervised Image Fusion Network","volume":"44","author":"Xu","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","unstructured":"Hogan, M., Rondao, D., Aouf, N., and Dubois-Matra, O. (2021). Using Convolutional Neural Networks for Relative Pose Estimation of a Non-Cooperative Spacecraft with Thermal Infrared Imagery. arXiv."},{"key":"ref_47","first-page":"5021811","article-title":"DFTI: Dual-Branch Fusion Network Based on Transformer and Inception for Space Noncooperative Objects","volume":"73","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Yu, F., Koltun, V., and Funkhouser, T. (2017, January 21\u201326). Dilated Residual Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.75"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"023522","DOI":"10.1117\/1.2945910","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classification","volume":"2","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2959","DOI":"10.1109\/26.477498","article-title":"Image quality measures and their performance","volume":"43","author":"Eskicioglu","year":"1995","journal-title":"IEEE Trans. Commun."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/el:20000267","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Electron. Lett."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Xiang, Y., Schmidt, T., Narayanan, V., and Fox, D. (2017). PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes. arXiv.","DOI":"10.15607\/RSS.2018.XIV.019"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/21\/6664\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,4]],"date-time":"2025-11-04T05:14:50Z","timestamp":1762233290000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/21\/6664"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,1]]},"references-count":52,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["s25216664"],"URL":"https:\/\/doi.org\/10.3390\/s25216664","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,1]]}}}