{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T16:08:44Z","timestamp":1775146124000,"version":"3.50.1"},"reference-count":46,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T00:00:00Z","timestamp":1623974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the China State Key Laboratory of Robotics","award":["19Z1240010018"],"award-info":[{"award-number":["19Z1240010018"]}]},{"name":"the office of military and civilian integration development committee of Shanghai","award":["2019-jmrh1-kj3"],"award-info":[{"award-number":["2019-jmrh1-kj3"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Earth observation satellite task scheduling research plays a key role in space-based remote sensing services. An effective task scheduling strategy can maximize the utilization of satellite resources and obtain larger objective observation profits. In this paper, inspired by the success of deep reinforcement learning in optimization domains, the deep deterministic policy gradient algorithm is adopted to solve a time-continuous satellite task scheduling problem. Moreover, an improved graph-based minimum clique partition algorithm is proposed for preprocessing in the task clustering phase by considering the maximum task priority and the minimum observation slewing angle under constraint conditions. Experimental simulation results demonstrate that the deep reinforcement learning-based task scheduling method is feasible and performs much better than traditional metaheuristic optimization algorithms, especially in large-scale problems.<\/jats:p>","DOI":"10.3390\/rs13122377","type":"journal-article","created":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T11:19:20Z","timestamp":1624015160000},"page":"2377","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["Revising the Observation Satellite Scheduling Problem Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3948-4239","authenticated-orcid":false,"given":"Yixin","family":"Huang","sequence":"first","affiliation":[{"name":"School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8868-0447","authenticated-orcid":false,"given":"Zhongcheng","family":"Mu","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China"}]},{"given":"Shufan","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China"}]},{"given":"Benjie","family":"Cui","sequence":"additional","affiliation":[{"name":"Shanghai Institute of Satellite Engineering, Shanghai 200240, China"}]},{"given":"Yuxiao","family":"Duan","sequence":"additional","affiliation":[{"name":"School of Aeronautics and Astronautics, Shanghai Jiao Tong University, Shanghai 200240, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1016\/j.ejor.2005.12.026","article-title":"A Heuristic for the Multi-Satellite, Multi-Orbit and Multi-User Management of Earth Observation Satellites","volume":"177","author":"Bianchessi","year":"2007","journal-title":"Eur. J. Oper. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1016\/j.ast.2008.01.001","article-title":"Planning and Scheduling Algorithms for the COSMO-SkyMed Constellation","volume":"12","author":"Bianchessi","year":"2008","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-44397-8","article-title":"Estimating Global Ocean Heat Content from Tidal Magnetic Satellite Observations","volume":"9","author":"Irrgang","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3140","DOI":"10.1109\/JSTARS.2015.2406339","article-title":"Generation of Spectral\u2013Temporal Response Surfaces by Combining Multispectral Satellite and Hyperspectral UAV Imagery for Precision Agriculture Applications","volume":"8","author":"Gevaert","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S1270-9638(02)01173-2","article-title":"Selecting and Scheduling Observations of Agile Satellites","volume":"6","author":"Verfaillie","year":"2002","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.ast.2019.03.054","article-title":"Distributed Onboard Mission Planning for Multi-Satellite Systems","volume":"89","author":"Zheng","year":"2019","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Wang, X., Wu, G., Xing, L., and Pedrycz, W. (2020). Agile Earth Observation Satellite Scheduling over 20 Years: Formulations, Methods, and Future Directions. IEEE Syst. J.","DOI":"10.1109\/JSYST.2020.2997050"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106002","DOI":"10.1016\/j.ast.2020.106002","article-title":"Multiobjective Planning for Spacecraft Reorientation under Complex Pointing Constraints","volume":"104","author":"Xu","year":"2020","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1287\/mnsc.46.1.148.15134","article-title":"Three Scheduling Algorithms Applied to the Earth Observing Systems Domain","volume":"46","author":"Wolfe","year":"2000","journal-title":"Manag. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"105994","DOI":"10.1016\/j.ast.2020.105994","article-title":"Orbit Determination for Fuel Station in Multiple SSO Spacecraft Refueling Considering the J2 Perturbation","volume":"105","author":"Zhu","year":"2020","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1016\/j.ejor.2018.11.058","article-title":"A Mixed Integer Linear Programming Model for Multi-Satellite Scheduling","volume":"275","author":"Chen","year":"2019","journal-title":"Eur. J. Oper. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1016\/j.cor.2019.05.030","article-title":"Agile Earth Observation Satellite Scheduling: An Orienteering Problem with Time-Dependent Profits and Travel Times","volume":"111","author":"Peng","year":"2019","journal-title":"Comput. Oper. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.cor.2017.04.006","article-title":"An Adaptive Large Neighborhood Search Metaheuristic for Agile Satellite Scheduling with Time-Dependent Transition Time","volume":"86","author":"Liu","year":"2017","journal-title":"Comput. Oper. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.chaos.2015.12.003","article-title":"Scheduling for Single Agile Satellite, Redundant Targets Problem Using Complex Networks Theory","volume":"83","author":"Wang","year":"2016","journal-title":"Chaos Solitons Fractals"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.ejor.2018.11.043","article-title":"Mixed-Integer Programming Models for Optimal Constellation Scheduling given Cloud Cover Uncertainty","volume":"275","author":"Valicka","year":"2019","journal-title":"Eur. J. Oper. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"110605","DOI":"10.1109\/ACCESS.2019.2925704","article-title":"Scheduling Multiple Agile Earth Observation Satellites for Oversubscribed Targets Using Complex Networks Theory","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Islas, M.A., Rubio, J.d.J., Mu\u00f1iz, S., Ochoa, G., Pacheco, J., Meda-Campa\u00f1a, J.A., Mujica-Vargas, D., Aguilar-Iba\u00f1ez, C., Gutierrez, G.J., and Zacarias, A. (2021). A Fuzzy Logic Model for Hourly Electrical Power Demand Modeling. Electronics, 10.","DOI":"10.3390\/electronics10040448"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1296","DOI":"10.1109\/TFUZZ.2009.2029569","article-title":"SOFMLS: Online Self-Organizing Fuzzy Modified Least-Squares Network","volume":"17","year":"2009","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1023\/A:1018920709696","article-title":"A New Single Model and Derived Algorithms for the Satellite Shot Planning Problem Using Graph Theory Concepts","volume":"69","author":"Gabrel","year":"1997","journal-title":"Ann. Oper. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.ejor.2013.04.009","article-title":"Image Collection Planning for KOrea Multi-Purpose SATellite-2","volume":"230","author":"Jang","year":"2013","journal-title":"Eur. J. Oper. Res."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, S., and Yang, J. (2019). A Satellite Task Planning Algorithm Based on a Symmetric Recurrent Neural Network. Symmetry, 11.","DOI":"10.3390\/sym11111373"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.ast.2014.10.006","article-title":"Mission Scheduling Optimization of SAR Satellite Constellation for Minimizing System Response Time","volume":"40","author":"Kim","year":"2015","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"813","DOI":"10.1016\/j.ijdrr.2018.02.013","article-title":"Satellite Scheduling of Large Areal Tasks for Rapid Response to Natural Disaster Using a Multi-Objective Genetic Algorithm","volume":"28","author":"Niu","year":"2018","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Long, X., Wu, S., Wu, X., Huang, Y., and Mu, Z. (2019). A GA-SA Hybrid Planning Algorithm Combined with Improved Clustering for LEO Observation Satellite Missions. Algorithms, 12.","DOI":"10.3390\/a12110231"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Mao, H., Alizadeh, M., Menache, I., and Kandula, S. (2016). Resource Management with Deep Reinforcement Learning. Proceedings of the 15th ACM Workshop on Hot Topics in Networks, Association for Computing Machinery.","DOI":"10.1145\/3005745.3005750"},{"key":"ref_26","first-page":"1057","article-title":"Policy Gradient Methods for Reinforcement Learning with Function Approximation","volume":"99","author":"Sutton","year":"1999","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","unstructured":"Bello, I., Pham, H., Le, Q.V., Norouzi, M., and Bengio, S. (2016). Neural Combinatorial Optimization with Reinforcement Learning. arXiv."},{"key":"ref_28","first-page":"6348","article-title":"Learning Combinatorial Optimization Algorithms over Graphs","volume":"30","author":"Khalil","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_29","first-page":"9839","article-title":"Reinforcement Learning for Solving the Vehicle Routing Problem","volume":"31","author":"Nazari","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Peng, B., Wang, J., and Zhang, Z. (2019). A Deep Reinforcement Learning Algorithm Using Dynamic Attention Model for Vehicle Routing Problems. International Symposium on Intelligence Computation and Applications, Springer.","DOI":"10.1007\/978-981-15-5577-0_51"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1109\/TITS.2018.2829165","article-title":"A Scalable Reinforcement Learning Algorithm for Scheduling Railway Lines","volume":"20","author":"Khadilkar","year":"2018","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3163","DOI":"10.1109\/TVT.2019.2897134","article-title":"Deep Reinforcement Learning Based Resource Allocation for V2V Communications","volume":"68","author":"Ye","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_33","unstructured":"Hadj-Salah, A., Verdier, R., Caron, C., Picard, M., and Capelle, M. (2019). Schedule Earth Observation Satellites with Deep Reinforcement Learning. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1016\/j.cja.2018.12.018","article-title":"Online Scheduling of Image Satellites Based on Neural Networks and Deep Reinforcement Learning","volume":"32","author":"Haijiao","year":"2019","journal-title":"Chin. J. Aeronaut."},{"key":"ref_35","first-page":"346","article-title":"Two-Phase Neural Combinatorial Optimization with Reinforcement Learning for Agile Satellite Scheduling","volume":"17","author":"Zhao","year":"2020","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lam, J.T., Rivest, F., and Berger, J. (2019). Deep Reinforcement Learning for Multi-Satellite Collection Scheduling. International Conference on Theory and Practice of Natural Computing, Springer.","DOI":"10.1007\/978-3-030-34500-6_13"},{"key":"ref_37","unstructured":"Wu, G., Du, X., Fan, M., Wang, J., Shi, J., and Wang, X. (2020). Ensemble of Heuristic and Exact Algorithm Based on the Divide and Conquer Framework for Multi-Satellite Observation Scheduling. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1884","DOI":"10.1016\/j.cor.2013.02.009","article-title":"A Two-Phase Scheduling Method with the Consideration of Task Clustering for Earth Observing Satellites","volume":"40","author":"Wu","year":"2013","journal-title":"Comput. Oper. Res."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1016\/j.ejor.2015.03.011","article-title":"A Multi-Objective Local Search Heuristic for Scheduling Earth Observations Taken by an Agile Satellite","volume":"245","author":"Tangpattanakul","year":"2015","journal-title":"Eur. J. Oper. Res."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.ast.2019.04.007","article-title":"Task Scheduling and Attitude Planning for Agile Earth Observation Satellite with Intensive Tasks","volume":"90","author":"Wang","year":"2019","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/j.ast.2019.03.028","article-title":"The Optimization Design with Minimum Power for Variable Speed Control Moment Gyroscopes with Integrated Power and Attitude Control","volume":"88","author":"Liu","year":"2019","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1109\/TCAD.1986.1270207","article-title":"Automated Synthesis of Data Paths in Digital Systems","volume":"5","author":"Tseng","year":"1986","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circ. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"576","DOI":"10.1016\/j.cie.2017.09.050","article-title":"Satellite Observation Scheduling with a Novel Adaptive Simulated Annealing Algorithm and a Dynamic Task Clustering Strategy","volume":"113","author":"Wu","year":"2017","journal-title":"Comput. Ind. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level Control through Deep Reinforcement Learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_45","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous Control with Deep Reinforcement Learning. arXiv."},{"key":"ref_46","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2377\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:18:15Z","timestamp":1760163495000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/13\/12\/2377"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,18]]},"references-count":46,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["rs13122377"],"URL":"https:\/\/doi.org\/10.3390\/rs13122377","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,18]]}}}