{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T02:55:31Z","timestamp":1790477731469,"version":"4.1.0"},"reference-count":44,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2024,10,2]],"date-time":"2024-10-02T00:00:00Z","timestamp":1727827200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry\u2019s and National Commission\u2019s Scientific Research Project","award":["TJ2022B020200-003-020"],"award-info":[{"award-number":["TJ2022B020200-003-020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper investigates the single agile optical satellite scheduling problem, which has received increasing attention due to the rapid growth in earth observation requirements. Owing to the complicated constraints and considerable solution space of this problem, the conventional exact methods and heuristic methods, which are sensitive to the problem scale, demand high computational expenses. Thus, an efficient approach is demanded to solve this problem, and this paper proposes a deep reinforcement learning algorithm with a local attention mechanism. A mathematical model is first established to describe this problem, which considers a series of complex constraints and takes the profit ratio of completed tasks as the optimization objective. Then, a neural network framework with an encoder\u2013decoder structure is adopted to generate high-quality solutions, and a local attention mechanism is designed to improve the generation of solutions. In addition, an adaptive learning rate strategy is proposed to guide the actor\u2013critic training algorithm to dynamically adjust the learning rate in the training process to enhance the training effectiveness of the proposed network. Finally, extensive experiments verify that the proposed algorithm outperforms the comparison algorithms in terms of solution quality, generalization performance, and computation efficiency.<\/jats:p>","DOI":"10.3390\/s24196396","type":"journal-article","created":{"date-parts":[[2024,10,2]],"date-time":"2024-10-02T06:27:31Z","timestamp":1727850451000},"page":"6396","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Deep Reinforcement Learning with Local Attention for Single Agile Optical Satellite Scheduling Problem"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2244-7367","authenticated-orcid":false,"given":"Zheng","family":"Liu","sequence":"first","affiliation":[{"name":"National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing 101416, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Xiong","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing 101416, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6043-8045","authenticated-orcid":false,"given":"Chi","family":"Han","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing 101416, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaolan","family":"Yu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Space Target Awareness, Space Engineering University, Beijing 101416, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1109\/TSMC.2015.2431643","article-title":"Coordinated planning of heterogeneous earth observation resources","volume":"46","author":"Wu","year":"2015","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Berger, J., Lo, N., No\u00ebl, M., and Noutegne, L. (2020). DynaQUEST: A New Approach to the Dynamic Multi-satellite Scheduling Problem. Proceedings of the ICORES, SCITEPRESS\u2014Science and Technology Publications, Lda.","DOI":"10.5220\/0008975701940201"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1921","DOI":"10.1109\/JSYST.2021.3072122","article-title":"A Mission Planning Modeling Method of Multipoint Target Imaging Within a Single Pass for Super-Agile Earth Observation Satellite","volume":"16","author":"Lu","year":"2021","journal-title":"IEEE Syst. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.cor.2017.04.006","article-title":"An adaptive large neighborhood search metaheuristic for agile satellite scheduling with time-dependent transition time","volume":"86","author":"Liu","year":"2017","journal-title":"Comput. Oper. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S1270-9638(02)01173-2","article-title":"Selecting and scheduling observations of agile satellites","volume":"6","author":"Verfaillie","year":"2002","journal-title":"Aerosp. Sci. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3520","DOI":"10.1109\/TAES.2021.3088490","article-title":"Agile earth observation satellite scheduling with a quantum annealer","volume":"57","author":"Stollenwerk","year":"2021","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107530","DOI":"10.1016\/j.cie.2021.107530","article-title":"A multi-objective memetic approach for time-dependent agile earth observation satellite scheduling problem","volume":"159","author":"Wei","year":"2021","journal-title":"Comput. Ind. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106010","DOI":"10.1016\/j.cor.2022.106010","article-title":"Unrelated parallel machine scheduling with multiple time windows: An application to earth observation satellite scheduling","volume":"149","author":"Wang","year":"2023","journal-title":"Comput. Oper. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1614","DOI":"10.1109\/TSMC.2020.3031738","article-title":"Solving the agile earth observation satellite scheduling problem with time-dependent transition times","volume":"52","author":"Peng","year":"2020","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2017\/7345941","article-title":"A branch and bound algorithm for agile earth observation satellite scheduling","volume":"2017","author":"Chu","year":"2017","journal-title":"Discret. Dyn. Nat. Soc."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"104946","DOI":"10.1016\/j.cor.2020.104946","article-title":"An exact algorithm for agile earth observation satellite scheduling with time-dependent profits","volume":"120","author":"Peng","year":"2020","journal-title":"Comput. Oper. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1080\/01605682.2019.1609891","article-title":"A new hybrid genetic algorithm for the collection scheduling problem for a satellite constellation","volume":"71","author":"Barkaoui","year":"2020","journal-title":"J. Oper. Res. Soc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.actaastro.2017.04.027","article-title":"Swarm satellite mission scheduling & planning using hybrid dynamic mutation genetic algorithm","volume":"137","author":"Zheng","year":"2017","journal-title":"Acta Astronaut."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3837","DOI":"10.3233\/JIFS-212034","article-title":"An improved method for satellite emergency mission scheduling scheme group decision-making incorporating PSO and MULTIMOORA","volume":"42","author":"Yan","year":"2022","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1016\/j.actaastro.2022.09.020","article-title":"Dynamic regional splitting planning of remote sensing satellite swarm using parallel genetic PSO algorithm","volume":"204","author":"Wu","year":"2023","journal-title":"Acta Astronaut."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1016\/j.asr.2017.10.056","article-title":"Mission planning optimization of video satellite for ground multi-object staring imaging","volume":"61","author":"Cui","year":"2018","journal-title":"Adv. Space Res."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1016\/j.asr.2018.10.007","article-title":"Hierarchical scheduling for real-time agile satellite task scheduling in a dynamic environment","volume":"63","author":"He","year":"2019","journal-title":"Adv. Space Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1080\/0305215X.2019.1657113","article-title":"A hybrid binary artificial bee colony algorithm for the satellite photograph scheduling problem","volume":"52","author":"Luo","year":"2020","journal-title":"Eng. Optim."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3090","DOI":"10.1109\/TAES.2022.3146115","article-title":"Reward factor-based multiple agile satellites scheduling with energy and memory constraints","volume":"58","author":"Chatterjee","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_20","first-page":"047002","article-title":"Area targets observation mission planning of agile satellite considering the drift angle constraint","volume":"4","author":"Du","year":"2018","journal-title":"J. Astron. Telesc. Instrum. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"40006","DOI":"10.1109\/ACCESS.2023.3269066","article-title":"A Hybrid Discrete Artificial Bee Colony Algorithm for Imaging Satellite Mission Planning","volume":"11","author":"Yang","year":"2023","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"eado3937","DOI":"10.1126\/sciadv.ado3937","article-title":"Direct electromagnetic information processing with planar diffractive neural network","volume":"10","author":"Gu","year":"2024","journal-title":"Sci. Adv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1038\/s41928-023-00951-x","article-title":"Programmable surface plasmonic neural networks for microwave detection and processing","volume":"6","author":"Gao","year":"2023","journal-title":"Nat. Electron."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2301938","DOI":"10.1002\/adom.202301938","article-title":"Classification of metal handwritten digits based on microwave diffractive deep neural network","volume":"12","author":"Gu","year":"2024","journal-title":"Adv. Opt. Mater."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ma, Q., Gao, X., Gu, Z., Liu, C., Li, L., You, J.W., and Cui, T.J. (2024). Intelligent neuromorphic computing based on nanophotonics and metamaterials. MRS Commun., 1\u201320.","DOI":"10.1557\/s43579-024-00520-z"},{"key":"ref_26","unstructured":"Nazari, M., Oroojlooy, A., Snyder, L., and Tak\u00e1c, M. (2018). Reinforcement Learning for Solving the Vehicle Routing Problem. Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Neural Information Processing Systems Foundation."},{"key":"ref_27","unstructured":"Kool, W., Van Hoof, H., and Welling, M. (2018). Attention, learn to solve routing problems!. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1109\/TSMC.2020.3020732","article-title":"A generic Markov decision process model and reinforcement learning method for scheduling agile earth observation satellites","volume":"52","author":"He","year":"2020","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"108242","DOI":"10.1016\/j.cie.2022.108242","article-title":"A competitive Markov decision process model and a recursive reinforcement-learning algorithm for fairness scheduling of agile satellites","volume":"169","author":"Ren","year":"2022","journal-title":"Comput. Ind. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chen, M., Chen, Y., Chen, Y., and Qi, W. (2019, January 6\u20139). Deep Reinforcement Learning for Agile Satellite Scheduling Problem. Proceedings of the 2019 IEEE Symposium Series on Computational Intelligence (SSCI), Xiamen, China.","DOI":"10.1109\/SSCI44817.2019.9002957"},{"key":"ref_31","first-page":"346","article-title":"Two-phase neural combinatorial optimization with reinforcement learning for agile satellite scheduling","volume":"17","author":"Zhao","year":"2020","journal-title":"J. Aerosp. Inf. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"107607","DOI":"10.1016\/j.asoc.2021.107607","article-title":"Deep reinforcement learning and parameter transfer based approach for the multi-objective agile earth observation satellite scheduling problem","volume":"110","author":"Wei","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"104822","DOI":"10.1016\/j.cor.2019.104822","article-title":"QUEST\u2014A new quadratic decision model for the multi-satellite scheduling problem","volume":"115","author":"Berger","year":"2020","journal-title":"Comput. Oper. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2863","DOI":"10.1109\/TSMC.2022.3220534","article-title":"Simulated annealing-based heuristic for multiple agile satellites scheduling under cloud coverage uncertainty","volume":"53","author":"Han","year":"2022","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"105626","DOI":"10.1016\/j.cor.2021.105626","article-title":"An improved genetic algorithm for the integrated satellite imaging and data transmission scheduling problem","volume":"139","author":"Zhang","year":"2022","journal-title":"Comput. Oper. Res."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2576","DOI":"10.1109\/TSMC.2023.3345928","article-title":"Generalized model and deep reinforcement learning-based evolutionary method for multitype satellite observation scheduling","volume":"54","author":"Song","year":"2024","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.1109\/TAES.2023.3337066","article-title":"Multi-region Mission Planning by Satellite Swarm Using Simulated Annealing and Neighborhood Search","volume":"60","author":"Wu","year":"2023","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1016\/j.ifacol.2019.11.286","article-title":"Preparation of Papers for IFAC Conferences & Symposia: Autonomous Distribution Algorithm for Formation Satellites Under Emergent Imaging Requests","volume":"52","author":"Yue","year":"2019","journal-title":"IFAC-Pap. Online"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.cor.2018.06.020","article-title":"An improved adaptive large neighborhood search algorithm for multiple agile satellites scheduling","volume":"100","author":"He","year":"2018","journal-title":"Comput. Oper. Res."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3742","DOI":"10.1109\/JSAC.2021.3118402","article-title":"Learning rate optimization for federated learning exploiting over-the-air computation","volume":"39","author":"Xu","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_42","unstructured":"Bresson, X., and Laurent, T. (2021). The transformer network for the traveling salesman problem. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"101236","DOI":"10.1016\/j.swevo.2023.101236","article-title":"RL-GA: A reinforcement learning-based genetic algorithm for electromagnetic detection satellite scheduling problem","volume":"77","author":"Song","year":"2023","journal-title":"Swarm Evol. Comput."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"108890","DOI":"10.1016\/j.cie.2022.108890","article-title":"Multi-satellite scheduling problem with marginal decreasing imaging duration: An improved adaptive ant colony algorithm","volume":"176","author":"Zhou","year":"2023","journal-title":"Comput. Ind. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/19\/6396\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:09:12Z","timestamp":1760112552000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/19\/6396"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,2]]},"references-count":44,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["s24196396"],"URL":"https:\/\/doi.org\/10.3390\/s24196396","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,2]]}}}