{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T05:01:50Z","timestamp":1774933310818,"version":"3.50.1"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T00:00:00Z","timestamp":1702252800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T00:00:00Z","timestamp":1702252800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12150007"],"award-info":[{"award-number":["12150007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021171","name":"Basic and Applied Basic Research Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["2020A1515110169"],"award-info":[{"award-number":["2020A1515110169"]}],"id":[{"id":"10.13039\/501100021171","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Basic Scientific Research Project","award":["JCKY2020903B002"],"award-info":[{"award-number":["JCKY2020903B002"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>With the continuous advancement of deep space exploration missions, the solar system boundary exploration mission is established as one of the China's most important deep space scientific exploration missions. The mission of the solar system boundary exploration has many challenges such as ultra-remote detection distance, ultra-long operation time, and ultra-long communication delay. Therefore, the problem of high-precision autonomous navigation needs to be solved urgently. This paper designs an autonomous intelligent navigation method based on X-ray pulsars in the cruise phase, which estimate the motion state of the probe in real time. The proposed navigation method employs the <jats:italic>Q<\/jats:italic>-learning Extended Kalman filter (QLEKF) to improve navigation accuracy during long periods of self-determining running. The QLEKF selects automatically the error covariance matrix parameter of the process noise and the measurement noise by the reward mechanism of reinforcement learning. Compared to the traditional EKF and AEKF, the QLEKF improves the estimation accuracy of position and velocity. Finally, the simulation result demonstrates the effectiveness and the superiority of the intelligent navigation algorithm based on QLEKF, which can satisfy the high-precision navigation requirements in the cruise phase of the solar system boundary exploration.<\/jats:p>","DOI":"10.1007\/s40747-023-01286-y","type":"journal-article","created":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T08:01:47Z","timestamp":1702281707000},"page":"2653-2672","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Intelligent navigation for the cruise phase of solar system boundary exploration based on Q-learning EKF"],"prefix":"10.1007","volume":"10","author":[{"given":"Wenjian","family":"Tao","sequence":"first","affiliation":[]},{"given":"Jinxiu","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Hang","family":"Hu","sequence":"additional","affiliation":[]},{"given":"Juzheng","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Huijie","family":"Sun","sequence":"additional","affiliation":[]},{"given":"Zhankui","family":"Zeng","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0623-0395","authenticated-orcid":false,"given":"Jianing","family":"Song","sequence":"additional","affiliation":[]},{"given":"Jihe","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,12,11]]},"reference":[{"issue":"10","key":"1286_CR1","first-page":"469","volume":"37","author":"P Dyal","year":"1984","unstructured":"Dyal P, Fimmel RO (1984) Exploring beyond the planets\u2014the Pioneer 10 and 11 missions. J Br Interplanet Soc 37(10):469\u2013479","journal-title":"J Br Interplanet Soc"},{"key":"1286_CR2","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.1126\/science.1117569","volume":"309","author":"RB Decker","year":"2005","unstructured":"Decker RB, Krimigis SM, Roelof EC et al (2005) Voyager 1 in the foreshock, termination shock, and heliosheath. Science 309:2020\u20132024","journal-title":"Science"},{"issue":"2","key":"1286_CR3","doi-asserted-by":"publisher","first-page":"1125","DOI":"10.1088\/0004-637X\/692\/2\/1125","volume":"692","author":"LF Burlaga","year":"2009","unstructured":"Burlaga LF, Ness NF, Acu\u00f1a MH et al (2009) Observations of the heliosheath and solar wind near the termination shock by Voyager 2. Astrophys J 692(2):1125\u20131130","journal-title":"Astrophys J"},{"key":"1286_CR4","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1007\/s11214-008-9374-8","volume":"140","author":"GH Fountain","year":"2008","unstructured":"Fountain GH, Kusnierkiewicz DY, Hersman CB et al (2008) The new horizons spacecraft. Space Sci Rev 140:23\u201347","journal-title":"Space Sci Rev"},{"key":"1286_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1360\/N112018-00273","volume":"49","author":"WR Wu","year":"2019","unstructured":"Wu WR, Yu DY, Huang JC et al (2019) Exploring the solar system boundary (in Chinese). Sci Sin Inf 49:1\u201316","journal-title":"Sci Sin Inf"},{"key":"1286_CR6","first-page":"284","volume":"42","author":"BY Tian","year":"2021","unstructured":"Tian BY, Wang YD, Zhang XY et al (2021) Flight mission planning for solar system boundary exploration. J Astronaut 42:284\u2013294","journal-title":"J Astronaut"},{"key":"1286_CR7","first-page":"01123","volume":"2023","author":"YQ Song","year":"2023","unstructured":"Song YQ, Wu WR, Hu H et al (2023) Gravity assist space pruning and global optimization of spacecraft trajectories for solar system boundary exploration. Complex Intell Syst 2023:01123\u201301132","journal-title":"Complex Intell Syst"},{"key":"1286_CR8","unstructured":"Wang C, Li H, Guo XC et al (2020) Scientific objectives for the exploration of the boundary of solar system. J Deep Space Explor 7(6):517\u2013524, 535"},{"key":"1286_CR9","doi-asserted-by":"publisher","first-page":"112424","DOI":"10.1109\/ACCESS.2019.2934501","volume":"7","author":"X Ma","year":"2019","unstructured":"Ma X, Ning XL, Chen X et al (2019) Geometric coplanar constraints-aided autonomous celestial navigation for spacecraft in deep space exploration. IEEE Access 7:112424\u2013112434","journal-title":"IEEE Access"},{"key":"1286_CR10","doi-asserted-by":"publisher","first-page":"5245136","DOI":"10.34133\/2021\/5245136","volume":"2021","author":"Y Xia","year":"2021","unstructured":"Xia Y, Li J, Zhai R et al (2021) Application prospect of fission-powered spacecraft in solar system exploration missions. Space Sci Technol 2021:5245136","journal-title":"Space Sci Technol"},{"issue":"1","key":"1286_CR11","doi-asserted-by":"publisher","first-page":"012127","DOI":"10.1088\/1742-6596\/2224\/1\/012127","volume":"2224","author":"YS Wang","year":"2022","unstructured":"Wang YS, Wang YD, Zheng W et al (2022) X-ray pulsar-based navigation scheme for solar system boundary exploration. J Phys Conf Ser 2224(1):012127","journal-title":"J Phys Conf Ser"},{"issue":"1","key":"1286_CR12","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1360\/SSI-2019-0242","volume":"50","author":"WR Wu","year":"2020","unstructured":"Wu WR, Li HT, Li Z et al (2020) Status and prospect of China\u2019s deep space TT&C network. Sci Sin Inf 50(1):93\u2013114","journal-title":"Sci Sin Inf"},{"key":"1286_CR13","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1016\/j.paerosci.2013.06.003","volume":"63","author":"X Ma","year":"2013","unstructured":"Ma X, Fang JC, Ning XL (2013) An overview of the autonomous navigation for a gravity-assist interplanetary spacecraft. Prog Aerosp Sci 63:56\u201366","journal-title":"Prog Aerosp Sci"},{"key":"1286_CR14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/978-981-15-3293-1","volume-title":"X-ray pulsar-based navigation: theory and applications, Chap. 1","author":"W Zheng","year":"2020","unstructured":"Zheng W, Wang YD (2020) X-ray pulsar-based navigation: theory and applications, Chap. 1. Springer, Singapore, pp 1\u20133"},{"issue":"2","key":"1286_CR15","first-page":"80","volume":"1","author":"P Reichley","year":"1971","unstructured":"Reichley P, Downs G, Morris G (1971) Use of pulsar signals as clocks. NASA Jet Propuls Lab Q Tech Rev 1(2):80\u201386","journal-title":"NASA Jet Propuls Lab Q Tech Rev"},{"key":"1286_CR16","unstructured":"Chester TJ, Butman SA (1981) Navigation using X-ray pulsars. NASA Technical Report 22-25"},{"key":"1286_CR17","unstructured":"Ray Paul S, Sheikh Suneel I, Graven Paul H et al (2008) Deep space navigation using celestial X-ray sources. In: Proceedings of the 2008 national technical meeting of The Institute of Navigation, San Diego, CA, pp 101\u2013109"},{"issue":"1","key":"1286_CR18","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1016\/j.asr.2020.10.019","volume":"67","author":"Q Xu","year":"2021","unstructured":"Xu Q, Fan XH, Zhao AG et al (2021) Pre-correction X-ray pulsar navigation algorithm based on asynchronous overlapping observation method. Adv Space Res 67(1):583\u2013596","journal-title":"Adv Space Res"},{"issue":"2","key":"1286_CR19","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1017\/S0373463315000727","volume":"69","author":"YQ Wang","year":"2016","unstructured":"Wang YQ, Zheng W (2016) Pulse phase estimation of X-ray pulsar with the aid of vehicle orbital dynamics. J Navig 69(2):414\u2013432","journal-title":"J Navig"},{"issue":"1","key":"1286_CR20","first-page":"57","volume":"142","author":"X Ning","year":"2017","unstructured":"Ning X, Yang YQ, Gui MZ et al (2017) Pulsar navigation using time of arrival (TOA) and time differential TOA (TDTOA). Acta Astronaut 142(1):57\u201363","journal-title":"Acta Astronaut"},{"key":"1286_CR21","doi-asserted-by":"publisher","first-page":"6325","DOI":"10.4028\/www.scientific.net\/AMR.433-440.6325","volume":"433\u2013440","author":"B Yang","year":"2012","unstructured":"Yang B, Liu LK (2012) Deep space navigation using X-ray pulsars. Adv Mater Res 433\u2013440:6325\u20136331","journal-title":"Adv Mater Res"},{"issue":"5","key":"1286_CR22","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1016\/j.asr.2012.10.009","volume":"51","author":"EH Wei","year":"2013","unstructured":"Wei EH, Jin SG, Zhang Q et al (2013) Autonomous navigation of Mars probe using X-ray pulsars: modeling and results. Adv Space Res 51(5):849\u2013857","journal-title":"Adv Space Res"},{"issue":"1","key":"1286_CR23","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.ast.2013.12.008","volume":"32","author":"DZ Feng","year":"2014","unstructured":"Feng DZ, Guo HH, Xin W et al (2014) Autonomous orbit determination and its error analysis for deep space using X-ray pulsar. Aerosp Sci Technol 32(1):35\u201341","journal-title":"Aerosp Sci Technol"},{"issue":"4","key":"1286_CR24","doi-asserted-by":"publisher","first-page":"2149","DOI":"10.1109\/TAES.2021.3068432","volume":"57","author":"JT Runnels","year":"2021","unstructured":"Runnels JT, Gebre-Egziabher D (2021) Estimator for deep-space position and attitude using X-ray pulsars. IEEE Trans Aerosp Electron Syst 57(4):2149\u20132166","journal-title":"IEEE Trans Aerosp Electron Syst"},{"key":"1286_CR25","doi-asserted-by":"publisher","first-page":"9823609","DOI":"10.34133\/2021\/9823609","volume":"2021","author":"HH Zhang","year":"2021","unstructured":"Zhang HH, Li J, Wang ZG et al (2021) Guidance navigation and control for Chang\u2019E-5 powered descent. Space Sci Technol 2021:9823609","journal-title":"Space Sci Technol"},{"key":"1286_CR26","first-page":"9846185","volume":"2021","author":"XY Huang","year":"2021","unstructured":"Huang XY, Li MD, Wang XL et al (2021) The Tianwen-1 guidance, navigation, and control for mars entry, descent, and landing. Space Sci Technol 2021:9846185","journal-title":"Space Sci Technol"},{"issue":"4","key":"1286_CR27","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1007\/s11424-017-5151-7","volume":"30","author":"YG Jiang","year":"2017","unstructured":"Jiang YG, Huang Y, Xue WC et al (2017) On designing consistent extended Kalman filter. J Syst Sci Complex 30(4):751\u2013764","journal-title":"J Syst Sci Complex"},{"key":"1286_CR28","doi-asserted-by":"publisher","first-page":"9854601","DOI":"10.34133\/2022\/9854601","volume":"2022","author":"BJ Yang","year":"2022","unstructured":"Yang BJ, Huang H, Gao L (2022) Centered error entropy-based sigma-point Kalman filter for spacecraft state estimation with non-Gaussian noise. Space Sci Technol 2022:9854601","journal-title":"Space Sci Technol"},{"issue":"4","key":"1286_CR29","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1016\/j.cja.2015.05.001","volume":"28","author":"QP Song","year":"2015","unstructured":"Song QP, Liu RK (2015) Weighted adaptive filtering algorithm for carrier tracking of deep space signal. Chin J Aeronaut 28(4):1236\u20131244","journal-title":"Chin J Aeronaut"},{"key":"1286_CR30","doi-asserted-by":"crossref","unstructured":"Fu K, Zhang D, Tang P et al (2015) Adaptive extended Kalman filter for a red shift navigation system. In: 2015 34th Chinese control conference (CCC). IEEE, pp 5194\u20135199","DOI":"10.1109\/ChiCC.2015.7260449"},{"issue":"4","key":"1286_CR31","doi-asserted-by":"publisher","first-page":"1803","DOI":"10.1002\/asjc.2336","volume":"23","author":"K Xiong","year":"2021","unstructured":"Xiong K, Wei CL, Zhang HY (2021) Q-learning for noise covariance adaptation in extended KALMAN filter. Asian J Control 23(4):1803\u20131816","journal-title":"Asian J Control"},{"issue":"2","key":"1286_CR32","doi-asserted-by":"publisher","first-page":"178","DOI":"10.1287\/ijoc.1080.0305","volume":"21","author":"A Gosavi","year":"2009","unstructured":"Gosavi A (2009) Reinforcement learning: a tutorial survey and recent advances. INFORMS J Comput 21(2):178\u2013192","journal-title":"INFORMS J Comput"},{"issue":"2018","key":"1286_CR33","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1016\/j.procs.2018.10.178","volume":"141","author":"D Kim","year":"2018","unstructured":"Kim D, Lee T, Kim S et al (2018) Adaptive packet scheduling in IoT environment based on Q-learning. Procedia Comput Sci 141(2018):247\u2013254","journal-title":"Procedia Comput Sci"},{"key":"1286_CR34","doi-asserted-by":"crossref","unstructured":"Dai X, Nateghi V, Fourati H et al (2022) Q-learning-based noise covariance adaptation in Kalman filter for MARG sensors attitude estimation. In: 2022 IEEE international symposium on inertial sensors and systems (INERTIAL). IEEE, 2022","DOI":"10.1109\/INERTIAL53425.2022.9787752"},{"issue":"16","key":"1286_CR35","doi-asserted-by":"publisher","first-page":"2248","DOI":"10.1177\/0954410020927522","volume":"234","author":"K Xiong","year":"2020","unstructured":"Xiong K, Wei CW (2020) Integrated celestial navigation for spacecraft using interferometer and earth sensor. Proc Inst Mech Eng Part G J Aerosp Eng 234(16):2248\u20132262","journal-title":"Proc Inst Mech Eng Part G J Aerosp Eng"},{"issue":"6","key":"1286_CR36","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1108\/AEAT-05-2021-0139","volume":"94","author":"K Xiong","year":"2022","unstructured":"Xiong K, Wei CL, Zhou P (2022) Integrated autonomous optical navigation using Q-Learning extended Kalman filter. Aircr Eng Aerosp Technol 94(6):848\u2013861","journal-title":"Aircr Eng Aerosp Technol"},{"issue":"18","key":"1286_CR37","doi-asserted-by":"publisher","first-page":"6992","DOI":"10.3390\/s22186992","volume":"22","author":"K Xiong","year":"2022","unstructured":"Xiong K, Zhou P, Wei CL (2022) Autonomous navigation of unmanned aircraft using space target LOS measurements and QLEKF. Sensors 22(18):6992","journal-title":"Sensors"},{"issue":"1","key":"1286_CR38","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1109\/TAES.2014.130463","volume":"51","author":"J Liu","year":"2015","unstructured":"Liu J, Fang JC, Kang ZW et al (2015) Novel algorithm for X-ray pulsar navigation against Doppler effects. IEEE Trans Aerosp Electron Syst 51(1):228\u2013241","journal-title":"IEEE Trans Aerosp Electron Syst"},{"issue":"9","key":"1286_CR39","doi-asserted-by":"publisher","first-page":"4484","DOI":"10.1109\/TSP.2010.2050479","volume":"58","author":"AA Emadzadeh","year":"2010","unstructured":"Emadzadeh AA, Speyer JL (2010) On modeling and pulse phase estimation of X-ray pulsars. IEEE Trans Signal Process 58(9):4484\u20134495","journal-title":"IEEE Trans Signal Process"},{"key":"1286_CR40","doi-asserted-by":"publisher","first-page":"161141","DOI":"10.1109\/ACCESS.2019.2950531","volume":"7","author":"LR Shen","year":"2019","unstructured":"Shen LR, Sun HF, Li XP et al (2019) A robust filtering method for X-ray pulsar navigation in the situation of strong noises and large state model errors. IEEE Access 7:161141\u2013161151","journal-title":"IEEE Access"},{"issue":"1","key":"1286_CR41","doi-asserted-by":"publisher","first-page":"49","DOI":"10.2514\/1.13331","volume":"29","author":"SI Sheikh","year":"2006","unstructured":"Sheikh SI, Pines DJ, Ray PS et al (2006) Spacecraft navigation using X-ray pulsars. J Guid Control Dyn 29(1):49\u201363","journal-title":"J Guid Control Dyn"},{"issue":"1","key":"1286_CR42","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1017\/S0373463316000357","volume":"70","author":"PB Ma","year":"2017","unstructured":"Ma PB, Wang TS, Jiang FH et al (2017) Autonomous navigation of Mars probes by single X-ray pulsar measurement and optical data of viewing Martian moons. J Navig 70(1):18\u201332","journal-title":"J Navig"},{"key":"1286_CR43","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins CJCH, Dayan P (1992) Q-learning. Mach Learn 8:279\u2013292","journal-title":"Mach Learn"},{"key":"1286_CR44","doi-asserted-by":"publisher","first-page":"110368","DOI":"10.1016\/j.knosys.2023.110368","volume":"265","author":"FQ Zhao","year":"2023","unstructured":"Zhao FQ, Wang QY, Wang L (2023) An inverse reinforcement learning framework with the Q-learning mechanism for the metaheuristic algorithm. Knowl Based Syst 265:110368","journal-title":"Knowl Based Syst"},{"key":"1286_CR45","doi-asserted-by":"crossref","unstructured":"Dai X, Fourati H, Prieur C (2022) A dynamic grid-based Q-learning for noise covariance adaptation in EKF and its application in navigation. In: 2022 IEEE 61st conference on decision and control (CDC), pp 4984\u20134989","DOI":"10.1109\/CDC51059.2022.9993410"},{"issue":"7","key":"1286_CR46","doi-asserted-by":"publisher","first-page":"1059","DOI":"10.3390\/sym12071059","volume":"12","author":"XB Ren","year":"2020","unstructured":"Ren XB, Nie GG, Li LY (2020) An improved augmented algorithm for direction error in XPNAV. Symmetry 12(7):1059","journal-title":"Symmetry"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01286-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01286-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01286-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,30]],"date-time":"2024-03-30T15:33:56Z","timestamp":1711812836000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01286-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,11]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["1286"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01286-y","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,11]]},"assertion":[{"value":"2 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 November 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 December 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declared that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}