{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T00:00:09Z","timestamp":1762300809747,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Natural Science Foundation Project, China","award":["12150007","2019B030302001"],"award-info":[{"award-number":["12150007","2019B030302001"]}]},{"name":"the Guangdong Major Project of Basic and Applied Basic Research, China","award":["12150007","2019B030302001"],"award-info":[{"award-number":["12150007","2019B030302001"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The Solar System Boundary Exploration (SSBE) mission is the focal point for future far-reaching space exploration. Due to the SSBE having many scientific difficulties that need to be studied, such as a super long space exploratory distance, a super long flight time in orbit, and a significant communication data delay between the ground and the probe, the probe must have sufficient intelligence to realize intelligent autonomous navigation. Traditional navigation schemes have been unable to provide high-accuracy autonomous intelligent navigation for the probe independent of the ground. Therefore, high-accuracy intelligent astronomical integrated navigation would provide new methods and technologies for the navigation of the SSBE probe. The probe of the SSBE is disturbed by multiple sources of solar light pressure and a complex, unknown environment during its long cruise operation while in orbit. In order to ensure the high-accuracy position state and velocity state error estimation for the probe in the cruise phase, an autonomous intelligent integrated navigation scheme based on the X-ray pulsar\/solar and target planetary Doppler velocity measurements is proposed. The reinforcement Q-learning method is introduced, and the reward mechanism is designed for trial-and-error tuning of state and observation noise error covariance parameters. The federated extended Kalman filter (FEKF) based on the Q-learning (QLFEKF) navigation algorithm is proposed to achieve high-accuracy state estimations of the autonomous intelligence navigation system for the SSBE probe cruise phase. The main advantage of the QLFEKF is that Q-learning combined with the conventional federated filtering method could optimize the state parameters in real-time and obtain high position and velocity state estimation (PVSE) accuracy. Compared with the conventional FEKF integrated navigation algorithm, the PVSE navigation accuracy of the federated filter integrated based the Q-learning navigation algorithm is improved by 55.84% and 37.04%, respectively, demonstrating the higher accuracy and greater capability of the raised autonomous intelligent integrated navigation algorithm. The simulation results show that the intelligent integrated navigation algorithm based on QLFEKF has higher navigation accuracy and is able to satisfy the demands of autonomous high accuracy for the SSBE cruise phase.<\/jats:p>","DOI":"10.3390\/rs16234465","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T07:28:01Z","timestamp":1732778881000},"page":"4465","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Intelligent QLFEKF Integrated Navigation for the SSBE Cruise Phase Based on X-Ray Pulsar\/Solar and Target Planetary Doppler Information Fusion"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5955-9085","authenticated-orcid":false,"given":"Wenjian","family":"Tao","sequence":"first","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1448-1581","authenticated-orcid":false,"given":"Jinxiu","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]},{"given":"Jianing","family":"Song","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Beijing Institute of Technology, Zhuhai 519088, China"}]},{"given":"Qin","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]},{"given":"Zebin","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]},{"given":"Hui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]},{"given":"Jikun","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Yancheng Institute of Technology, Yancheng 224051, China"}]},{"given":"Jihe","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Sun Yat-sen University, Shenzhen 518107, China"},{"name":"Shenzhen Key Laboratory of Intelligent Microsatellite Constellation, Shenzhen 518107, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1360\/N112018-00273","article-title":"Exploring the solar system boundary","volume":"49","author":"Wu","year":"2019","journal-title":"Sci. Sin. Inform."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1126\/science.188.4187.445","article-title":"Pioneer 10 and pioneer 11","volume":"4187","author":"Hall","year":"1975","journal-title":"Science"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/s11214-009-9590-x","article-title":"Simulation of ambiguity effects in Doppler tracking of Pioneer probes","volume":"151","author":"Courty","year":"2010","journal-title":"Space Sci. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Capova, K.A. (2021). Introducing Humans to the Extraterrestrials: The Pioneering Missions of the Pioneer and Voyager Probes. Front. Hum. Dyn., 3.","DOI":"10.3389\/fhumd.2021.714616"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1126\/science.1235451","article-title":"Magnetic field observations as Voyager 1 entered the heliosheath depletion region","volume":"341","author":"Burlaga","year":"2013","journal-title":"Science"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1038\/s41550-019-0928-3","article-title":"Cosmic ray measurements from Voyager 2 as it crossed into interstellar space","volume":"3","author":"Stone","year":"2019","journal-title":"Nat. Astron."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s11214-008-9374-8","article-title":"The new horizons spacecraft","volume":"140","author":"Fountain","year":"2008","journal-title":"Space Sci. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1007\/s11214-007-9242-y","article-title":"New Horizons mission design","volume":"140","author":"Guo","year":"2008","journal-title":"Space Sci. Rev."},{"key":"ref_9","first-page":"62","article-title":"Laser communication proposal for solar system boundary exploration","volume":"43","author":"Liu","year":"2022","journal-title":"J. Telem. Track. Command"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s40747-023-01123-2","article-title":"Gravity assist space pruning and global optimization of spacecraft trajectories for solar system boundary exploration","volume":"10","author":"Song","year":"2023","journal-title":"Complex Intell. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zheng, W., and Wang, Y. (2020). X-ray Pulsar\u2013Based Navigation: Theory and Applications, Springer.","DOI":"10.1007\/978-981-15-3293-1"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"012127","DOI":"10.1088\/1742-6596\/2224\/1\/012127","article-title":"X-Ray Pulsar-based Navigation Scheme for Solar System Boundary Exploration","volume":"2224","author":"Wang","year":"2020","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"49","DOI":"10.2514\/1.13331","article-title":"Spacecraft Navigation Using X-Ray Pulsars","volume":"29","author":"Sheikh","year":"2006","journal-title":"J. Guid. Control Dyn."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1109\/TCST.2010.2068049","article-title":"Relative navigation between two spacecraft using X-ray pulsars","volume":"19","author":"Emadzadeh","year":"2010","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1007\/s10686-016-9496-z","article-title":"Towards practical autonomous deep-space navigation using X-Ray pulsar timing","volume":"42","author":"Shemar","year":"2016","journal-title":"Exp. Astron."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1017\/S0373463315000727","article-title":"Pulse phase estimation of X-ray pulsar with the aid of vehicle orbital dynamics","volume":"69","author":"Wang","year":"2016","journal-title":"J. Navig."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.sigpro.2012.07.002","article-title":"Fast near-maximum likelihood phase estimation of X-ray pulsars","volume":"93","author":"Rinauro","year":"2013","journal-title":"Signal Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"L35","DOI":"10.1086\/180473","article-title":"Time-of-Arrival Observations of Eleven Pulsars","volume":"159","author":"Reichley","year":"1970","journal-title":"Astrophys. J."},{"key":"ref_19","unstructured":"Downs, G.S. (2024, June 12). Interplanetary Navigation Using Pulsating Radio Sources. NASA Technical Report N74-34150. 1974; pp. 1\u201312, Available online: https:\/\/ntrs.nasa.gov\/api\/citations\/19740026037\/downloads\/19740026037.pdf."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2149","DOI":"10.1109\/TAES.2021.3068432","article-title":"Estimator for deep-space position and attitude using X-ray pulsars","volume":"57","author":"Runnels","year":"2021","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gao, J., Fang, H., and Su, J. (2022). Differential X-Ray Pulsar Navigation Method Based on Pulse Arrival Time Difference. China Satellite Navigation Conference (CSNC 2022) Proceedings: Volume III, Springer Nature.","DOI":"10.1007\/978-981-19-2576-4_49"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"78075","DOI":"10.1109\/ACCESS.2021.3079500","article-title":"Accuracy analysis of spectral velocimetry for the solar Doppler difference navigation","volume":"9","author":"Huang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yim, J.R., Crassidis, J.L., and Junkins, J.L. (2000, January 14\u201317). Autonomous orbit navigation of interplanetary spacecraft. Proceedings of the Astrodynamics Specialist Conference, Denver, CO, USA.","DOI":"10.2514\/6.2000-3936"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1109\/TAES.2017.2651558","article-title":"A novel differential Doppler measurement-aided autonomous celestial navigation method for spacecraft during approach phase","volume":"53","author":"Ning","year":"2017","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1049\/iet-rsn.2011.0021","article-title":"Doppler\/XNAV\u2013integrated navigation system using small-area X-ray sensor","volume":"5","author":"Liu","year":"2011","journal-title":"IET Radar Sonar Navig."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1007\/s42405-020-00253-y","article-title":"Solar TDOA\/Doppler difference joint observation navigation for the approach phase of mars exploration","volume":"9","author":"Pan","year":"2020","journal-title":"Int. J. Aeronaut. Space Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1016\/j.asr.2016.02.001","article-title":"X-ray pulsars\/Doppler integrated navigation for Mars final approach","volume":"57","author":"Cui","year":"2016","journal-title":"Adv. Space Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1186\/s43020-021-00053-z","article-title":"Distributed Kalman filter for UWB\/INS integrated pedestrian localization under colored measurement noise","volume":"2","author":"Xu","year":"2021","journal-title":"Satell. Navig."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1186\/s43020-023-00115-4","article-title":"Resilient timekeeping algorithm with multi-observation fusion Kalman filter","volume":"4","author":"Wang","year":"2023","journal-title":"Satell. Navig."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1109\/9.847726","article-title":"A new method for the nonlinear transformation of means and covariances in filters and estimators","volume":"45","author":"Julier","year":"2000","journal-title":"IEEE Trans. Autom. Control."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xin, S.J., Wang, X.M., Zhang, J.L., Zhou, K., and Chen, Y.F. (2023). A Comparative Study of Factor Graph Optimization-Based and Extended Kalman Filter-Based PPP-B2b\/INS Integrated Navigation. Remote Sens., 15.","DOI":"10.3390\/rs15215144"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yin, Z.H., Yang, J.C., Ma, Y., Wang, S.L., Chai, D.S., and Cui, H.N. (2023). A Robust Adaptive Extended Kalman Filter Based on an Improved Measurement Noise Covariance Matrix for the Monitoring and Isolation of Abnormal Disturbances in GNSS\/INS Vehicle Navigation. Remote Sens., 15.","DOI":"10.3390\/rs15174125"},{"key":"ref_33","first-page":"964","article-title":"Doppler velocity measurement based on double measurement model and its integrated navigation","volume":"38","author":"Kang","year":"2017","journal-title":"J. Astronaut."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1109\/JSTSP.2009.2028380","article-title":"Augmentation of XNAV system to an ultraviolet sensor-based satellite navigation system","volume":"3","author":"Qiao","year":"2009","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"194","DOI":"10.11728\/cjss2013.02.194","article-title":"Autonomous orbit and attitude determination including time prediction based on XNAV and ultraviolet sensor","volume":"33","author":"Yang","year":"2013","journal-title":"Chin. J. Space Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.ast.2014.11.019","article-title":"X-ray pulsar\/Doppler difference integrated navigation for deep space exploration with unstable solar spectrum","volume":"41","author":"Liu","year":"2015","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ou, J.J., Guo, X., Lou, W.J., and Zhu, M. (2021). Quadrotor autonomous navigation in semi-known environments based on deep reinforcement learning. Remote Sens., 13.","DOI":"10.3390\/rs13214330"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Xiong, K., Zhao, Q., and Yuan, L. (2024). Calibration Method for Relativistic Navigation System Using Parallel Q-Learning Extended Kalman Filter. Sensors, 24.","DOI":"10.3390\/s24196186"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1002\/asjc.2336","article-title":"Q-learning for noise covariance adaptation in extended KALMAN filter","volume":"23","author":"Xiong","year":"2021","journal-title":"Asian J. Control"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2248","DOI":"10.1177\/0954410020927522","article-title":"Integrated celestial navigation for spacecraft using interferometer and Earth sensor","volume":"234","author":"Xiong","year":"2020","journal-title":"Proc. Inst. Mech. Eng. Part G J. Aerosp. Eng."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Nemati, M.H., Kankashvar, M.R., and Bolandi, H. (2022, January 17\u201319). Unscented Kalman Filter adaptive noise covariance selection for satellite formation flying with Q_leaming. Proceedings of the 2022 30th International Conference on Electrical Engineering (ICEE), IEEE, Tehran, Iran.","DOI":"10.1109\/ICEE55646.2022.9827301"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1177\/09544100231219818","article-title":"Spacecraft autonomous navigation using line-of-sight directions of non-cooperative targets by improved Q-learning based extended Kalman filter","volume":"238","author":"Xiong","year":"2024","journal-title":"Proc. Inst. Mech. Eng. Part G J. Aerosp. Eng."},{"key":"ref_43","first-page":"1017","article-title":"Research on Autonomous Navigation Method of Deep Space Cruise Phase Based on the Sun Observation","volume":"31","author":"Chang","year":"2010","journal-title":"Yuhang Xuebao\/J. Astronaut."},{"key":"ref_44","first-page":"214","article-title":"Research on Celestial Navigation for Mars Missions during the Interplanetary Cruising","volume":"3","author":"Chen","year":"2016","journal-title":"J. Deep Space Explor."},{"key":"ref_45","first-page":"952","article-title":"Research on Autonomous Navigation Method for the Cruise Phase of Mars Exploration","volume":"41","author":"Song","year":"2016","journal-title":"Geomat. Inf. Sci. Wuhan Univ."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TAES.2014.130463","article-title":"Novel algorithm for X-ray pulsar navigation against Doppler effects","volume":"51","author":"Liu","year":"2015","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"4484","DOI":"10.1109\/TSP.2010.2050479","article-title":"On modeling and pulse phase estimation of X-ray pulsars","volume":"58","author":"Emadzadeh","year":"2010","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/j.2161-4296.2011.tb01799.x","article-title":"Spacecraft navigation and timing using X-ray pulsars","volume":"58","author":"Sheikh","year":"2011","journal-title":"Navigation"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"675","DOI":"10.3969\/j.issn.1004-4132.2010.04.022","article-title":"Pulsar\/CNS integrated navigation based on federated UKF","volume":"21","author":"Liu","year":"2010","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"105321","DOI":"10.1016\/j.engappai.2022.105321","article-title":"Autonomous unmanned aerial vehicle navigation using reinforcement learning: A systematic review","volume":"115","author":"AlMahamid","year":"2022","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"5023","DOI":"10.1007\/s10462-022-10299-x","article-title":"Deep multiagent reinforcement learning: Challenges and directions","volume":"56","author":"Wong","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1843","DOI":"10.1177\/0954410014561705","article-title":"X-ray pulsar-based navigation system\/Sun measurement integrated navigation method for deep space explorer","volume":"229","author":"Wang","year":"2015","journal-title":"Proc. Inst. Mech. Eng. Part G J. Aerosp. Eng."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1016\/j.asr.2020.10.019","article-title":"Pre-correction X-ray pulsar navigation algorithm based on asynchronous overlapping observation method","volume":"67","author":"Xu","year":"2021","journal-title":"Adv. Space Res."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/23\/4465\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:41:24Z","timestamp":1760114484000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/23\/4465"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":54,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["rs16234465"],"URL":"https:\/\/doi.org\/10.3390\/rs16234465","relation":{},"ISSN":["2072-4292"],"issn-type":[{"type":"electronic","value":"2072-4292"}],"subject":[],"published":{"date-parts":[[2024,11,28]]}}}