{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,27]],"date-time":"2026-04-27T10:52:25Z","timestamp":1777287145329,"version":"3.51.4"},"reference-count":44,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T00:00:00Z","timestamp":1727136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62394354"],"award-info":[{"award-number":["62394354"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>For the relativistic navigation system where the position and velocity of the spacecraft are determined through the observation of the relativistic perturbations including stellar aberration and starlight gravitational deflection, a novel parallel Q-learning extended Kalman filter (PQEKF) is presented to implement the measurement bias calibration. The relativistic perturbations are extracted from the inter-star angle measurement achieved with a group of high-accuracy star sensors on the spacecraft. Inter-star angle measurement bias caused by the misalignment of the star sensors is one of the main error sources in the relativistic navigation system. In order to suppress the unfavorable effect of measurement bias on navigation performance, the PQEKF is developed to estimate the position and velocity, together with the calibration parameters, where the Q-learning approach is adopted to fine tune the process noise covariance matrix of the filter automatically. The high performance of the presented method is illustrated via numerical simulations in the scenario of medium Earth orbit (MEO) satellite navigation. The simulation results show that, for the considered MEO satellite and the presented PQEKF algorithm, in the case that the inter-star angle measurement accuracy is about 1 mas, after calibration, the positioning accuracy of the relativistic navigation system is less than 300 m.<\/jats:p>","DOI":"10.3390\/s24196186","type":"journal-article","created":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T04:01:24Z","timestamp":1727236884000},"page":"6186","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Calibration Method for Relativistic Navigation System Using Parallel Q-Learning Extended Kalman Filter"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9546-602X","authenticated-orcid":false,"given":"Kai","family":"Xiong","sequence":"first","affiliation":[{"name":"Science and Technology on Space Intelligent Control Laboratory, Beijing Institute of Control Engineering, Beijing 100094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qin","family":"Zhao","sequence":"additional","affiliation":[{"name":"Science and Technology on Space Intelligent Control Laboratory, Beijing Institute of Control Engineering, Beijing 100094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Yuan","sequence":"additional","affiliation":[{"name":"China Academy of Space Technology, Beijing 100094, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"107291","DOI":"10.1016\/j.ast.2021.107291","article-title":"Constraint Navigation Filter for Space Vehicle Autonomous Positioning with Deficient GNSS Measurements","volume":"120","author":"Huang","year":"2022","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1007\/s40295-020-00244-x","article-title":"Radiometric Autonomous Navigation Fused with Optical for Deep Space Exploration","volume":"68","author":"Ely","year":"2021","journal-title":"J. Astronaut. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"107237","DOI":"10.1016\/j.ast.2021.107237","article-title":"Reduction of GNSS-Denied Inertial Navigation Errors for Fixed Wing Autonomous Unmanned Air Vehicles","volume":"120","author":"Gallo","year":"2022","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1108\/AEAT-03-2022-0063","article-title":"INS\/CNS\/DNS\/XNAV Deep Integrated Navigation in a Highly Dynamic Environment","volume":"95","author":"Hu","year":"2023","journal-title":"Aircr. Eng. Aerosp. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3649662","DOI":"10.1155\/2022\/3649662","article-title":"A New Method to Improve the Measurement Accuracy of Autonomous Astronomical Navigation","volume":"2022","author":"Yang","year":"2022","journal-title":"J. Math."},{"key":"ref_6","first-page":"9","article-title":"Development Situation and Trend of Space Intelligent Navigation Technology","volume":"48","author":"Wang","year":"2022","journal-title":"Aerosp. Control Appl."},{"key":"ref_7","first-page":"25","article-title":"Observability Analysis of Satellite Autonomous Orbit Determination with Modeling and Measurement Errors","volume":"43","author":"Zhou","year":"2023","journal-title":"Chin. Space Sci. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"192","DOI":"10.2514\/1.G000872","article-title":"Optical Navigation Using Planet\u2019s Centroid and Apparent Diameter in Image","volume":"38","author":"Christian","year":"2015","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1016\/j.asr.2020.10.048","article-title":"Guidepost-based Autonomous Orbit Determination Method for GEO Satellite","volume":"67","author":"Hou","year":"2021","journal-title":"Adv. Space Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.actaastro.2021.12.030","article-title":"Autonomous navigation for deep space small satellites: Scientific and technological advances","volume":"193","author":"Turan","year":"2022","journal-title":"Acta Astronaut."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"49","DOI":"10.2514\/1.13331","article-title":"Spacecraft Navigation Using X-Ray Pulsars","volume":"29","author":"Sheikh","year":"2006","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1850","DOI":"10.2514\/1.G006204","article-title":"Use of Statistical Linearization for Nonlinear Least-Squares Problems in Pulsar Navigation","volume":"46","author":"Wang","year":"2023","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_13","first-page":"1501","article-title":"Deep Space Navigation by Optical Pulsars","volume":"46","author":"Zoccarato","year":"2023","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_14","first-page":"64","article-title":"A Study of the Navigation Technology and Application Based on Astronomical Spectral Velocity Measurement","volume":"19","author":"Zhang","year":"2020","journal-title":"Navig. Control"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1897","DOI":"10.1049\/iet-rsn.2020.0259","article-title":"Modelling and analysis of celestial Doppler difference velocimetry navigation considering solar characteristics","volume":"14","author":"Liu","year":"2020","journal-title":"IET Radar Sonar Navig."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"17480","DOI":"10.1109\/JSEN.2023.3288540","article-title":"A Novel Sun Direction\/Solar Disk Velocity Difference Integrated Navigation Method Against Installation Error of Spectrometer Array","volume":"23","author":"Gui","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Christian, J.A. (2019). StarNAV: Autonomous Optical Navigation of a Spacecraft by the Relativistic Perturbation of Starlight. Sensors, 19.","DOI":"10.3390\/s19194064"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"074502","DOI":"10.1088\/1538-3873\/ac0774","article-title":"Lost in Space? Relativistic Interstellar Navigation using an Astrometric Star Catalog","volume":"133","year":"2021","journal-title":"Publ. Astron. Soc. Pac."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1016\/j.actaastro.2021.12.007","article-title":"Navigation and star identification for an interstellar mission","volume":"192","author":"McKee","year":"2022","journal-title":"Acta Astronaut."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1580","DOI":"10.1086\/367593","article-title":"A Practical Relativistic Model for Microarcsecond Astrometry in Space","volume":"125","author":"Klioner","year":"2003","journal-title":"Astron. J."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1016\/j.actaastro.2022.04.027","article-title":"StarNAV with a wide field-of-view optical sensor","volume":"197","author":"McKee","year":"2022","journal-title":"Acta Astron."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.2514\/1.G005340","article-title":"Autonomous Navigation of Relativistic Spacecraft in Interstellar Space","volume":"44","author":"Yucalan","year":"2021","journal-title":"J. Guid. Control Dyn."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2248","DOI":"10.1177\/0954410020927522","article-title":"Integrated Celestial Navigation for Spacecraft Using Interferometer and Earth Sensor","volume":"234","author":"Xiong","year":"2020","journal-title":"Proc. Inst. Mech. Eng. Part G: J. Aerosp. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1108\/AEAT-05-2021-0139","article-title":"Integrated Autonomous Optical Navigation Using Q-Learning Extended Kalman Filter","volume":"94","author":"Xiong","year":"2022","journal-title":"Aircr. Eng. Aerosp. Technol."},{"key":"ref_25","first-page":"126","article-title":"Celestial angle measurement navigation for Mars probe considering relativistic effect","volume":"10","author":"Gui","year":"2023","journal-title":"J. Deep Space Explor."},{"key":"ref_26","first-page":"159","article-title":"An autonomous navigation method for spacecraft in cislunar space using stellar aberration observation","volume":"10","author":"Liu","year":"2023","journal-title":"J. Deep Space Explor."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"159371","DOI":"10.1109\/ACCESS.2020.3016277","article-title":"ANN Based Learning to Kalman Filter Algorithm for Indoor Environment Prediction in Smart Greenhouse","volume":"8","author":"Ullah","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2022.3197775","article-title":"A Hybrid Model and Learning-Based Adaptive Navigation Filter","volume":"71","author":"Or","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"032205","DOI":"10.1007\/s11432-016-0405-2","article-title":"Recursive Adaptive Filter Using Current Innovation for Celestial Navigation During the Mars Approach Phase","volume":"60","author":"Ning","year":"2017","journal-title":"Sci. China-Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.dsp.2015.09.004","article-title":"Robust unscented Kalman filter with adaptation of process and measurement noise covariances","volume":"48","author":"Li","year":"2016","journal-title":"Digit. Signal Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1729881420925357","DOI":"10.1177\/1729881420925357","article-title":"Autonomous Navigation Control Based on Improved Adaptive Filtering for Agricultural Robot","volume":"17","author":"Jia","year":"2020","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Xiong, K., Zhou, P., and Wei, C. (2022). Autonomous Navigation of Unmanned Aircraft Using Space Target LOS Measurements and QLEKF. Sensors, 22.","DOI":"10.3390\/s22186992"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2653","DOI":"10.1007\/s40747-023-01286-y","article-title":"Intelligent Navigation for the Cruise Phase of Solar System Boundary Exploration Based on Q-learning EKF","volume":"2","author":"Tao","year":"2024","journal-title":"Complex Intell. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1002\/asjc.2336","article-title":"Q-learning for noise covariance adaptation in extended Kalman filter","volume":"23","author":"Xiong","year":"2021","journal-title":"Asian J. Control."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3044","DOI":"10.1080\/00207721.2021.1919337","article-title":"SARSA in extended Kalman Filter for complex urban environments positioning","volume":"52","author":"Chen","year":"2021","journal-title":"Int. J. Syst. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1007\/s42154-022-00195-z","article-title":"Approximate optimal filter design for vehicle system through Actor-Critic reinforcement learning","volume":"5","author":"Yin","year":"2022","journal-title":"Automot. Innov."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"12","DOI":"10.2514\/1.22452","article-title":"Survey of Nonlinear Attitude Estimation Methods","volume":"30","author":"Crassidis","year":"2007","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"107731","DOI":"10.1016\/j.knosys.2021.107731","article-title":"Constrained Evolutionary Optimization Based on Reinforcement Learning Using the Objective Function and Constraints","volume":"237","author":"Hu","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"133653","DOI":"10.1109\/ACCESS.2019.2941229","article-title":"Q-learning Algorithms: A Comprehensive Classification and Applications","volume":"7","author":"Jang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.automatica.2019.01.018","article-title":"Data-driven approximate Q-learning stabilization with optimality error bound analysis","volume":"103","author":"Li","year":"2019","journal-title":"Automatica"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1109\/TII.2016.2617464","article-title":"Decoupled Visual Servoing with Fuzzy Q-learning","volume":"14","author":"Shi","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"137982","DOI":"10.1109\/ACCESS.2019.2942330","article-title":"UAV-Based Interference Source Localization: A Multi-model Q-learning Approach","volume":"7","author":"Wu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103712","DOI":"10.1016\/j.engappai.2020.103712","article-title":"Regenerative Braking System Modeling by Fuzzy Q-Learning","volume":"93","author":"Maia","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1109\/TCYB.2016.2542923","article-title":"Discrete-time Deterministic Q-learning: A Novel Convergence Analysis","volume":"47","author":"Wei","year":"2017","journal-title":"IEEE Trans. Cybern."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/19\/6186\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:01:58Z","timestamp":1760112118000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/19\/6186"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,24]]},"references-count":44,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["s24196186"],"URL":"https:\/\/doi.org\/10.3390\/s24196186","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,24]]}}}