{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T17:57:05Z","timestamp":1772042225594,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T00:00:00Z","timestamp":1676505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Open Fund of National Defense Key Discipline Laboratory of Micro-Spacecraft Technology","award":["HIT.KLOF.MST.201703"],"award-info":[{"award-number":["HIT.KLOF.MST.201703"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The cooperative positioning problem of hypersonic vehicles regarding LEO constellations is the focus of this research study on space-based early warning systems. A hypersonic vehicle is highly maneuverable, and its trajectory is uncertain. New challenges are posed for the cooperative positioning capability of the constellation. In recent years, breakthroughs in artificial intelligence technology have provided new avenues for collaborative multi-satellite intelligent autonomous decision-making technology. This paper addresses the problem of multi-satellite cooperative geometric positioning for hypersonic glide vehicles (HGVs) by the LEO-constellation-tracking system. To exploit the inherent advantages of hierarchical reinforcement learning in intelligent decision making while satisfying the constraints of cooperative observations, an autonomous intelligent decision-making algorithm for satellites that incorporates a hierarchical proximal policy optimization with random hill climbing (MAPPO-RHC) is designed. On the one hand, hierarchical decision making is used to reduce the solution space; on the other hand, it is used to maximize the global reward and to uniformly distribute satellite resources. The single-satellite local search method improves the capability of the decision-making algorithm to search the solution space based on the decision-making results of the hierarchical proximal policy-optimization algorithm, combining both random hill climbing and heuristic methods. Finally, the MAPPO-RHC algorithm\u2019s coverage and positioning accuracy performance is simulated and analyzed in two different scenarios and compared with four intelligent satellite decision-making algorithms that have been studied in recent years. From the simulation results, the decision-making results of the MAPPO-RHC algorithm can obtain more balanced resource allocations and higher geometric positioning accuracy. Thus, it is concluded that the MAPPO-RHC algorithm provides a feasible solution for the real-time decision-making problem of the LEO constellation early warning system.<\/jats:p>","DOI":"10.3390\/s23042225","type":"journal-article","created":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T04:01:52Z","timestamp":1676520112000},"page":"2225","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["An LEO Constellation Early Warning System Decision-Making Method Based on Hierarchical Reinforcement Learning"],"prefix":"10.3390","volume":"23","author":[{"given":"Yu","family":"Cheng","sequence":"first","affiliation":[{"name":"School of Aeronautics, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cheng","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Aeronautics, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shengxin","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Aeronautics, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bindi","family":"You","sequence":"additional","affiliation":[{"name":"School of Aeronautics, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Aeronautics, Harbin Institute of Technology, Harbin 150006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sastry, P.S., Cv, J., Raghavamurthy, D., and Rao, S.S. (2020). Advances in Small Satellite Technologies, Springer. Lecture Notes in Mechanical Engineering.","DOI":"10.1007\/978-981-15-1724-2"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"012088","DOI":"10.1088\/1742-6596\/1060\/1\/012088","article-title":"Research on State Estimation of Hypersonic Glide Vehicle","volume":"1060","author":"Huang","year":"2018","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4505","DOI":"10.1016\/j.asr.2021.09.003","article-title":"Onboard coordination and scheduling of multiple autonomous satellites in an uncertain environment","volume":"68","author":"Yang","year":"2021","journal-title":"Adv. Space Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhang, F., Chen, Y., and Chen, Y. (2018, January 8\u201313). Evolving Constructive Heuristics for Agile Earth Observing Satellite Scheduling Problem with Genetic Programming. Proceedings of the 2018 IEEE Congress on Evolutionary Computation (CEC), Rio de Janeiro, Brazil.","DOI":"10.1109\/CEC.2018.8477939"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4381","DOI":"10.1109\/TMC.2021.3076088","article-title":"Joint Observation and Transmission Scheduling in Agile Satellite Networks","volume":"21","author":"He","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Qi, W., Yang, W., Xing, L., and Yao, F. (2022). Modeling and Solving for Multi-Satellite Cooperative Task Allocation Problem Based on Genetic Programming Method. Mathematics, 10.","DOI":"10.3390\/math10193608"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhao, L., Wang, B., and Shen, C. (2021). A multi-objective scheduling method for operational coordination time using improved triangular fuzzy number representation. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0252293"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.23919\/JSEE.2022.000098","article-title":"Complex task planning method of space-aeronautics cooperative observation based on multi-layer interaction","volume":"99","author":"Jinming","year":"2022","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"28911","DOI":"10.1109\/ACCESS.2022.3157286","article-title":"Dual-Population Artificial Bee Colony Algorithm for Joint Observation Satellite Mission Planning Problem","volume":"10","author":"Jiang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1407","DOI":"10.23919\/JSEE.2021.000120","article-title":"Real-time online rescheduling for multiple agile satellites with emergent tasks","volume":"32","author":"Jun","year":"2021","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7819105","DOI":"10.1155\/2021\/7819105","article-title":"Mission Planning Issues of Imaging Satellites: Summary, Discussion, and Prospects","volume":"2021","author":"Zhang","year":"2021","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TEVC.2019.2934148","article-title":"A Data-Driven Parallel Scheduling Approach for Multiple Agile Earth Observation Satellites","volume":"24","author":"Du","year":"2020","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhou, F., Wang, Y., Zheng, W., Li, Z., and Wen, X. (2022). Fast Distributed Multiple-Model Nonlinearity Estimation for Tracking the Non-Cooperative Highly Maneuvering Target. Remote Sens., 14.","DOI":"10.3390\/rs14174239"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, S., and Yang, J. (2019). A Satellite Task Planning Algorithm Based on a Symmetric Recurrent Neural Network. Symmetry, 11.","DOI":"10.3390\/sym11111373"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1515\/astro-2022-0033","article-title":"High-altitude satellites range scheduling for urgent request utilizing reinforcement learning","volume":"31","author":"Ren","year":"2022","journal-title":"Open Astron."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3103","DOI":"10.1016\/j.asr.2022.01.037","article-title":"HiPGen: An approach for fast generation of multi-satellite observation plans via a hierarchical multi-channel transformer network","volume":"69","author":"Chen","year":"2022","journal-title":"Adv. Space Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1002\/sat.1352","article-title":"An online power allocation algorithm based on deep reinforcement learning in multibeam satellite systems","volume":"38","author":"Zhang","year":"2020","journal-title":"Int. J. Satell. Commun. Netw."},{"key":"ref_18","unstructured":"He, Y., Wu, G., Chen, Y., and Pedrycz, W. (2021). A Two-stage Framework and Reinforcement Learning-based Optimization Algorithms for Complex Scheduling Problems. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xia, K., Feng, J., Yan, C., and Duan, C. (2021). BeiDou Short-Message Satellite Resource Allocation Algorithm Based on Deep Reinforcement Learning. Entropy, 23.","DOI":"10.3390\/e23080932"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1177544","DOI":"10.1155\/2022\/1177544","article-title":"Deep Reinforcement Learning-Based Joint Satellite Scheduling and Resource Allocation in Satellite-Terrestrial Integrated Networks","volume":"2022","author":"Yin","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"146264","DOI":"10.1109\/ACCESS.2019.2943253","article-title":"Joint Optimization of Multi-UAV Target Assignment and Path Planning Based on Multi-Agent Reinforcement Learning","volume":"7","author":"Qie","year":"2019","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bao, X., Zhang, S., and Zhang, X. (2020, January 6\u20138). An Effective Method for Satellite Mission Scheduling Based on Reinforcement Learning. Proceedings of the 2020 Chinese Automation Congress (CAC), Shanghai, China.","DOI":"10.1109\/CAC51589.2020.9327581"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1901","DOI":"10.1109\/LGRS.2020.3009823","article-title":"An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning","volume":"18","author":"Dalin","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_24","first-page":"4268","article-title":"PGA: An Efficient Adaptive Traffic Signal Timing Optimization Scheme Using Actor-Critic Reinforcement Learning Algorithm","volume":"14","author":"Shen","year":"2020","journal-title":"KSII Trans. Internet Inf. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhang, H., Liu, X., Long, K., and Poor, H.V. (2021, January 7\u201311). Primal Dual PPO Learning Resource Allocation in Indoor IRS-Aided Networks. Proceedings of the 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain.","DOI":"10.1109\/GLOBECOM46510.2021.9685203"},{"key":"ref_26","first-page":"100","article-title":"Satellite Online Scheduling Algorithm Based on Proximal Policy","volume":"Volume 854","author":"Liang","year":"2022","journal-title":"Artificial Intelligence in China"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"107607","DOI":"10.1016\/j.asoc.2021.107607","article-title":"Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem","volume":"110","author":"Wei","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Huang, Y., Mu, Z., Wu, S., Cui, B., and Duan, Y. (2021). Revising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning. Remote Sens., 13.","DOI":"10.3390\/rs13122377"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"104316","DOI":"10.1016\/j.engappai.2021.104316","article-title":"Deep replacement: Reinforcement learning based constellation management and autonomous replacement","volume":"104","author":"Kopacz","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"7095","DOI":"10.1109\/JIOT.2021.3071531","article-title":"Enabling Efficient Scheduling in Large-Scale UAV-Assisted Mobile-Edge Computing via Hierarchical Reinforcement Learning","volume":"9","author":"Ren","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"220523","DOI":"10.1109\/ACCESS.2020.3040748","article-title":"Hierarchical Reinforcement-Learning for Real-Time Scheduling of Agile Satellites","volume":"8","author":"Ren","year":"2020","journal-title":"IEEE Access"},{"key":"ref_32","first-page":"346","article-title":"Two-Phase Neural Combinatorial Optimization with Reinforcement Learning for Agile Satellite Scheduling","volume":"17","author":"Zhao","year":"2020","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"92177","DOI":"10.1109\/ACCESS.2022.3202938","article-title":"Unmanned Aerial Vehicle Swarm Cooperative Decision-Making for SEAD Mission: A Hierarchical Multiagent Reinforcement Learning Approach","volume":"10","author":"Yue","year":"2022","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"13124","DOI":"10.1109\/TVT.2020.3020400","article-title":"Joint Optimization of Handover Control and Power Allocation Based on Multi-Agent Deep Reinforcement Learning","volume":"69","author":"Guo","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1614","DOI":"10.1109\/TSMC.2020.3031738","article-title":"Solving the Agile Earth Observation Satellite Scheduling Problem With Time-Dependent Transition Times","volume":"52","author":"Peng","year":"2022","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, J., Luo, Q., Lou, J., and Li, Y. (2020, January 6\u20138). An Adaptive Infrared Tracking Method for Spacebased Surveillance to a Hypersonic Cruise Vehicle. Proceedings of the 2020 Chinese Automation Congress (CAC), Shanghai, China.","DOI":"10.1109\/CAC51589.2020.9327809"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bing, H., Hongru, L., Guoming, L., Zhijun, P., and Tao, J. (2022, January 23\u201325). Simulation modeling and detection performance analysis of space-based infrared early warning system. Proceedings of the 2022 IEEE fifth International Conference on Information Systems and Computer Aided Education (ICISCAE), Dalian, China.","DOI":"10.1109\/ICISCAE55891.2022.9927563"},{"key":"ref_38","first-page":"1","article-title":"Analysis of Infrared Satellite\u2019s Detectability for Common Hypersonic Glide Body","volume":"42","author":"Shi","year":"2021","journal-title":"INFRARED"},{"key":"ref_39","first-page":"1","article-title":"Infrared measurement and composite tracking algorithm for air-breathing hypersonic vehicles","volume":"27","author":"Zhang","year":"2018","journal-title":"J. Electron. Imaging"},{"key":"ref_40","first-page":"41","article-title":"Study on infrared radiation feature of near space hypersonic missile","volume":"45","author":"Zhang","year":"2015","journal-title":"Laser Infrared"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2225\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:37:54Z","timestamp":1760121474000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/4\/2225"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,16]]},"references-count":40,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23042225"],"URL":"https:\/\/doi.org\/10.3390\/s23042225","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,16]]}}}