{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T22:33:22Z","timestamp":1790202802635,"version":"4.1.0"},"reference-count":32,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T00:00:00Z","timestamp":1586995200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11572019"],"award-info":[{"award-number":["11572019"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009101","name":"Shanghai Academy of Spaceflight Technology","doi-asserted-by":"publisher","award":["SAST2019084"],"award-info":[{"award-number":["SAST2019084"]}],"id":[{"id":"10.13039\/100009101","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In order to help the pursuer find its advantaged control policy in a one-to-one game in space, this paper proposes an innovative pre-trained fuzzy reinforcement learning algorithm, which is conducted in the x, y, and z channels separately. Compared with the previous algorithms applied in ground games, this is the first time reinforcement learning has been introduced to help the pursuer in space optimize its control policy. The known part of the environment is utilized to help the pursuer pre-train its consequent set before learning. An actor-critic framework is built in each moving channel of the pursuer. The consequent set of the pursuer is updated through the gradient descent method in fuzzy inference systems. The numerical experimental results validate the effectiveness of the proposed algorithm in improving the game ability of the pursuer.<\/jats:p>","DOI":"10.3390\/s20082253","type":"journal-article","created":{"date-parts":[[2020,4,16]],"date-time":"2020-04-16T13:01:39Z","timestamp":1587042099000},"page":"2253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["A Pre-Trained Fuzzy Reinforcement Learning Method for the Pursuing Satellite in a One-to-One Game in Space"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2305-2666","authenticated-orcid":false,"given":"Xiao","family":"Wang","sequence":"first","affiliation":[{"name":"School of Astronautics, Beihang University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peng","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Astronautics, Beihang University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yushan","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Astronautics, Beihang University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue","family":"Sun","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Aerospace Intelligent Control Technology, Shanghai Aerospace Control Technology Institute, Shanghai 201109, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s42064-018-0053-6","article-title":"A survey on artificial intelligence trends in spacecraft guidance dynamics and control","volume":"3","author":"Izzo","year":"2019","journal-title":"Astrodynamics"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Pan, H., and Kapila, V. (2001, January 4\u20137). Adaptive nonlinear control for spacecraft formation flying with coupled translational and attitude dynamics. Proceedings of the IEEE Conference on Decision and Control, Orlando, FL, USA.","DOI":"10.1115\/IMECE2001\/DSC-24580"},{"key":"ref_3","unstructured":"Pan, H., Wong, H., and Kapila, V. (2004, January 14\u201317). Output feedback control for spacecraft with coupled translation and attitude dynamics. Proceedings of the American Control Conference, Nassau, Bahamas."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.ast.2010.05.009","article-title":"Nonlinear optimal control of spacecraft approaching a tumbling target","volume":"15","author":"Xin","year":"2011","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"892","DOI":"10.2514\/1.17498","article-title":"Adaptive Output Feedback Control for Spacecraft Rendezvous and Docking Under Measurement Uncertainty","volume":"29","author":"Singla","year":"2006","journal-title":"J. Guid. Control Dyn."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1016\/j.cja.2014.12.030","article-title":"Super twisting controller for on-orbit servicing to non-cooperative target","volume":"28","author":"Chen","year":"2015","journal-title":"Chin. J. Aeronaut."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2598","DOI":"10.1002\/rnc.4038","article-title":"Robust adaptive fixed-time tracking control of 6-DOF spacecraft fly-around mission for noncooperative target","volume":"28","author":"Huang","year":"2018","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1109\/TIE.2016.2609399","article-title":"Adaptive Backstepping Control of Spacecraft Rendezvous and Proximity Operations with Input Saturation and Full-state Constraint","volume":"64","author":"Sun","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2421","DOI":"10.1080\/00207721.2015.1011251","article-title":"Inverse optimal sliding mode control of spacecraft with coupled translation and attitude dynamics","volume":"46","author":"Pukdeboon","year":"2015","journal-title":"Int. J. Syst. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1007\/BF00928706","article-title":"Series Nash solution of two-person, nonzero-sum, linear-quadratic differential games","volume":"7","author":"Cruz","year":"1971","journal-title":"J. Optim. Theory Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1007\/BF00926523","article-title":"Further Properties of Nonzero-Sum Differential Games","volume":"3","author":"Ho","year":"1969","journal-title":"J. Optim. Theory Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1109\/TAES.1972.309520","article-title":"Proportional Navigation with a Maneuvering Target","volume":"AES-8","author":"Guelman","year":"1972","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1137\/0323015","article-title":"The Existence of Value and Saddle Point in Games of Fixed Duration","volume":"23","author":"Berkovitz","year":"1985","journal-title":"SIAM J. Control Optim."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/BF00939877","article-title":"Complex differential games of pursuit-evasion type with state constraints, part 2: Numerical computation of optimal open-loop strategies","volume":"78","author":"Breitner","year":"1993","journal-title":"J. Optim. Theory Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"105","DOI":"10.2514\/1.3960","article-title":"Optimal Fighter Pursuit-Evasion Maneuvers Found Via Two-Sided Optimization","volume":"29","author":"Horie","year":"2006","journal-title":"J. Guid. Control Dyn."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Stupik, J., Pontani, M., and Conway, B. (2013, January 11\u201315). Optimal Pursuit\/Evasion Spacecraft Trajectories in the Hill Reference Frame. Proceedings of the AIAA\/AAS Astrodynamics Specialist Conference, Hilton Head, SC, USA.","DOI":"10.2514\/6.2012-4882"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jagat, A., and Sinclair, A.J. (2014, January 4\u20137). Optimization of Spacecraft Pursuit-Evasion Game Trajectories in the Euler-Hill Reference Frame. Proceedings of the AIAA\/AAS Astrodynamics Specialist Conference, San Diego, CA, USA.","DOI":"10.2514\/6.2014-4131"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3032","DOI":"10.1109\/TAES.2017.2725498","article-title":"Nonlinear Control for Spacecraft Pursuit-Evasion Game Using State-Dependent Riccati Equation Method","volume":"53","author":"Jagat","year":"2017","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1109\/37.126844","article-title":"Reinforcement learning is direct adaptive optimal control","volume":"12","author":"Sutton","year":"1992","journal-title":"IEEE Control Syst. Mag."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Technical Note: Q-Learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1109\/TIE.2005.847576","article-title":"Obstacle avoidance of a mobile robot using hybrid learning approach","volume":"52","author":"Er","year":"2005","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1109\/TITS.2005.853698","article-title":"An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control","volume":"6","author":"Dai","year":"2005","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xiao, H., Li, L., and Zhou, F. (2007, January 18\u201321). Mobile Robot Path Planning Based on Q-ANN. Proceedings of the 2007 IEEE International Conference on Automation and Logistics, Jinan, China.","DOI":"10.1109\/ICAL.2007.4339028"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1109\/TCYB.2015.2509646","article-title":"A Q-Learning Approach to Flocking With UAVs in a Stochastic Environment","volume":"47","author":"Hung","year":"2017","journal-title":"IEEE Trans. Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bilgin, A.T., and Kadioglu-Urtis, E. (2015, January 27\u201331). An approach to multi-agent pursuit evasion games using reinforcement learning. Proceedings of the 2015 International Conference on Advanced Robotics (ICAR), Istanbul, Turkey.","DOI":"10.1109\/ICAR.2015.7251450"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Analikwu, C.V., and Schwartz, H.M. (2016, January 24\u201329). Reinforcement learning in the guarding a territory game. Proceedings of the 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Vancouver, BC, Canada.","DOI":"10.1109\/FUZZ-IEEE.2016.7737798"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1058","DOI":"10.1007\/s40815-016-0284-8","article-title":"A Residual Gradient Fuzzy Reinforcement Learning Algorithm for Differential Games","volume":"19","author":"Awheda","year":"2017","journal-title":"Int. J. Fuzzy Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.robot.2010.09.006","article-title":"Self-learning fuzzy logic controllers for pursuit\u2013evasion differential games","volume":"59","author":"Desouky","year":"2011","journal-title":"Robot. Auton. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"653","DOI":"10.2514\/8.8704","article-title":"Terminal Guidance System for Satellite Rendezvous","volume":"27","author":"Clohessy","year":"1960","journal-title":"J. Aerosp. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1016\/B978-1-4832-1450-4.50045-6","article-title":"Fuzzy Identification of Systems and Its Applications to Modeling and Control","volume":"15","author":"Takagi","year":"1993","journal-title":"Readings Fuzzy Sets Intell. Syst."},{"key":"ref_32","unstructured":"Goldberg, D., and Richardson, J. (1987, January 28\u201331). Genetic Algorithms with Sharing for Multi-modal Function Optimization. Proceedings of the International Conference on Genetic Algorithms, Cambridge, MA, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2253\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:09:11Z","timestamp":1760364551000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,16]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20082253"],"URL":"https:\/\/doi.org\/10.3390\/s20082253","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,16]]}}}