{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T06:24:02Z","timestamp":1762410242174,"version":"3.37.3"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T00:00:00Z","timestamp":1705449600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12150007"],"award-info":[{"award-number":["12150007"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Basic Scientific Research Project","award":["JCKY2020903B002"],"award-info":[{"award-number":["JCKY2020903B002"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>During the flyby mission of small celestial bodies in deep space, it is hard for spacecraft to take photos at proper positions only rely on ground-based scheduling, due to the long communication delay and environment uncertainties. Aimed at imaging properly, an autonomous imaging policy generated by the scheduling networks that based on deep reinforcement learning is proposed in this paper. A novel reward function with relative distance variation in consideration is designed to guide the scheduling networks to obtain higher reward. A new part is introduced to the reward function to improve the performance of the networks. The robustness and adaptability of the proposed networks are verified in simulation with different imaging missions. Compared with the results of genetic algorithm (GA), Deep Q-network (DQN) and proximal policy optimization (PPO), the reward obtained by the trained scheduling networks is higher than DQN and PPO in most imaging missions and is equivalent to that of GA but, the decision time of the proposed networks after training is about six orders of magnitude less than that of GA, with less than 1e\u22124\u00a0s. The simulation and analysis results indicate that the proposed scheduling networks have great potential in further onboard application.<\/jats:p>","DOI":"10.1007\/s40747-023-01312-z","type":"journal-article","created":{"date-parts":[[2024,1,17]],"date-time":"2024-01-17T06:01:56Z","timestamp":1705471316000},"page":"3181-3195","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Autonomous imaging scheduling networks of small celestial bodies flyby based on deep reinforcement learning"],"prefix":"10.1007","volume":"10","author":[{"given":"Hang","family":"Hu","sequence":"first","affiliation":[]},{"given":"Weiren","family":"Wu","sequence":"additional","affiliation":[]},{"given":"Yuqi","family":"Song","sequence":"additional","affiliation":[]},{"given":"Wenjian","family":"Tao","sequence":"additional","affiliation":[]},{"given":"Jianing","family":"Song","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1448-1581","authenticated-orcid":false,"given":"Jinxiu","family":"Zhang","sequence":"additional","affiliation":[]},{"given":"Jihe","family":"Wang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,17]]},"reference":[{"issue":"1","key":"1312_CR1","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1006\/icar.1994.1002","volume":"107","author":"J Veverka","year":"1994","unstructured":"Veverka J, Belton M, Klaasen K, Chapman C (1994) Galileo\u2019s encounter with 951 Gaspra: overview. Icarus 107(1):2\u201317","journal-title":"Icarus"},{"issue":"2\u20139","key":"1312_CR2","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1016\/S0094-5765(00)00087-4","volume":"47","author":"MD Rayman","year":"2000","unstructured":"Rayman MD, Varghese P, Lehman DH, Livesay LL (2000) Results from the Deep Space 1 technology validation mission. Acta Astronaut 47(2\u20139):475\u2013487","journal-title":"Acta Astronaut"},{"issue":"9","key":"1312_CR3","doi-asserted-by":"publisher","first-page":"1058","DOI":"10.1016\/j.pss.2010.02.004","volume":"58","author":"A Accomazzo","year":"2010","unstructured":"Accomazzo A, Wirth KR, Lodiot S, K\u00fcppers M, Schwehm G (2010) The flyby of Rosetta at asteroid \u0160teins\u2013mission and science operations. Planet Space Sci 58(9):1058\u20131065","journal-title":"Planet Space Sci"},{"issue":"3\u20134","key":"1312_CR4","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1016\/j.actaastro.2009.06.013","volume":"66","author":"A Accomazzo","year":"2010","unstructured":"Accomazzo A, Ferri P, Lodiot S, Hubault A, Porta R, Pellon-Bailon J-L (2010) The first Rosetta asteroid flyby. Acta Astronaut 66(3\u20134):382\u2013390","journal-title":"Acta Astronaut"},{"issue":"1","key":"1312_CR5","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1016\/j.pss.2011.11.013","volume":"66","author":"R Schulz","year":"2012","unstructured":"Schulz R, Sierks H, K\u00fcppers M, Accomazzo A (2012) Rosetta fly-by at asteroid (21) Lutetia: an overview. Planet Space Sci 66(1):2\u20138","journal-title":"Planet Space Sci"},{"issue":"6245","key":"1312_CR6","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1126\/science.aaa8415","volume":"349","author":"MI Jordan","year":"2015","unstructured":"Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255\u2013260","journal-title":"Science"},{"key":"1312_CR7","first-page":"1","volume":"71","author":"H Xing","year":"2022","unstructured":"Xing H, Xiao Z, Qu R et al (2022) An efficient federated distillation learning system for multitask time series classification. IEEE Trans Instrum Meas 71:1\u201312","journal-title":"IEEE Trans Instrum Meas"},{"key":"1312_CR8","doi-asserted-by":"publisher","DOI":"10.34133\/2021\/9841456","author":"M Huang","year":"2021","unstructured":"Huang M, Xu Y, Qian L et al (2021) A bridge neural network-based optical-SAR image joint intelligent interpretation framework. Space Sci Technol. https:\/\/doi.org\/10.34133\/2021\/9841456","journal-title":"Space Sci Technol"},{"key":"1312_CR9","doi-asserted-by":"publisher","DOI":"10.34133\/2021\/9879246","author":"Q Meng","year":"2021","unstructured":"Meng Q, Huang M, Xu Y et al (2021) Decentralized distributed deep learning with low-bandwidth consumption for smart constellations. Space Sci Technol. https:\/\/doi.org\/10.34133\/2021\/9879246","journal-title":"Space Sci Technol"},{"issue":"4","key":"1312_CR10","doi-asserted-by":"publisher","first-page":"4287","DOI":"10.1007\/s40747-022-00948-7","volume":"9","author":"Y Gu","year":"2022","unstructured":"Gu Y, Zhu Z, Lv J, Shi L, Hou Z, Xu S (2022) DM-DQN: dueling Munchausen deep Q network for robot path planning. Complex Intell Syst 9(4):4287\u20134300","journal-title":"Complex Intell Syst"},{"key":"1312_CR11","doi-asserted-by":"publisher","first-page":"105669","DOI":"10.1109\/ACCESS.2019.2932257","volume":"7","author":"J Xie","year":"2019","unstructured":"Xie J, Shao Z, Li Y, Guan Y, Tan J (2019) Deep reinforcement learning with optimized reward functions for robotic trajectory planning. IEEE Access 7:105669\u2013105679","journal-title":"IEEE Access"},{"key":"1312_CR12","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1007\/s10846-019-01073-3","volume":"98","author":"C Yan","year":"2020","unstructured":"Yan C, Xiang X, Wang C (2020) Towards real-time path planning through deep reinforcement learning for a UAV in dynamic environments. J Intell Robot Syst 98:297\u2013309","journal-title":"J Intell Robot Syst"},{"issue":"5","key":"1312_CR13","doi-asserted-by":"publisher","first-page":"5723","DOI":"10.1109\/TVT.2020.2982508","volume":"69","author":"Q Liu","year":"2020","unstructured":"Liu Q, Shi L, Sun L, Li J, Ding M, Shu F (2020) Path planning for UAV-mounted mobile edge computing with deep reinforcement learning. IEEE Trans Veh Technol 69(5):5723\u20135728","journal-title":"IEEE Trans Veh Technol"},{"key":"1312_CR14","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2022.3208457","author":"F Song","year":"2022","unstructured":"Song F, Xing H, Wang X et al (2022) Evolutionary multi-objective reinforcement learning based trajectory control and task offloading in UAV-assisted mobile edge computing. IEEE Trans Mob Comput. https:\/\/doi.org\/10.1109\/TMC.2022.3208457","journal-title":"IEEE Trans Mob Comput"},{"issue":"7540","key":"1312_CR15","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529\u2013533","journal-title":"Nature"},{"key":"1312_CR16","doi-asserted-by":"crossref","unstructured":"Torrado RR, Bontrager P, Togelius J, Liu J, Perez-Liebana D (2018) Deep reinforcement learning for general video game ai. In: 2018 IEEE conference on computational intelligence and games (CIG), pp 1\u20138","DOI":"10.1109\/CIG.2018.8490422"},{"issue":"6419","key":"1312_CR17","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","volume":"362","author":"D Silver","year":"2018","unstructured":"Silver D, Hubert T, Schrittwieser J et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419):1140\u20131144","journal-title":"Science"},{"issue":"24","key":"1312_CR18","doi-asserted-by":"publisher","first-page":"17508","DOI":"10.1109\/JIOT.2021.3081694","volume":"8","author":"J Chen","year":"2021","unstructured":"Chen J, Xing H, Xiao Z et al (2021) A DRL agent for jointly optimizing computation offloading and resource allocation in MEC. IEEE Internet Things J 8(24):17508\u201317524","journal-title":"IEEE Internet Things J"},{"issue":"4","key":"1312_CR19","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1109\/JAS.2014.7004667","volume":"1","author":"B Gaudet","year":"2014","unstructured":"Gaudet B, Furfaro R (2014) Adaptive pinpoint and fuel efficient mars landing using reinforcement learning. IEEE\/CAA J Autom Sin 1(4):397\u2013411","journal-title":"IEEE\/CAA J Autom Sin"},{"issue":"7","key":"1312_CR20","doi-asserted-by":"publisher","first-page":"1723","DOI":"10.1016\/j.asr.2019.12.030","volume":"65","author":"B Gaudet","year":"2020","unstructured":"Gaudet B, Linares R, Furfaro R (2020) Deep reinforcement learning for six degree-of-freedom planetary landing. Adv Space Res 65(7):1723\u20131741","journal-title":"Adv Space Res"},{"key":"1312_CR21","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.actaastro.2020.02.051","volume":"171","author":"R Furfaro","year":"2020","unstructured":"Furfaro R, Scorsoglio A, Linares R, Massari M (2020) Adaptive generalized ZEM-ZEV feedback guidance for planetary landing via a deep reinforcement learning approach. Acta Astronaut 171:156\u2013171","journal-title":"Acta Astronaut"},{"key":"1312_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.actaastro.2020.02.036","volume":"171","author":"B Gaudet","year":"2020","unstructured":"Gaudet B, Linares R, Furfaro R (2020) Terminal adaptive guidance via reinforcement meta-learning: applications to autonomous asteroid close-proximity operations. Acta Astronaut 171:1\u201313","journal-title":"Acta Astronaut"},{"key":"1312_CR23","unstructured":"Harris A, Teil T, Schaub H (2019) Spacecraft decision-making autonomy using deep reinforcement learning. In: 29th AAS\/AIAA spaceflight mechanics meeting, AAS Paper 19\u2013447"},{"key":"1312_CR24","doi-asserted-by":"crossref","unstructured":"Harris A T, Schaub H (2020) Spacecraft command and control with safety guarantees using shielded deep reinforcement learning. In: AIAA SciTech, AIAA Paper 2020\u20130386","DOI":"10.2514\/6.2020-0386"},{"issue":"4","key":"1312_CR25","doi-asserted-by":"publisher","first-page":"1011","DOI":"10.1016\/j.cja.2018.12.018","volume":"32","author":"H Wang","year":"2019","unstructured":"Wang H, Yang Z, Zhou W, Li D (2019) Online scheduling of image satellites based on neural networks and deep reinforcement learning. Chin J Aeronaut 32(4):1011\u20131019","journal-title":"Chin J Aeronaut"},{"issue":"1","key":"1312_CR26","doi-asserted-by":"publisher","first-page":"70","DOI":"10.2514\/1.I010992","volume":"19","author":"AP Herrmann","year":"2022","unstructured":"Herrmann AP, Schaub H (2022) Monte Carlo tree search methods for the earth-observing satellite scheduling problem. J Aerospace Inf Syst 19(1):70\u201382","journal-title":"J Aerospace Inf Syst"},{"key":"1312_CR27","doi-asserted-by":"crossref","unstructured":"Herrmann A, Schaub H (2022) Autonomous on-board planning for earth-orbiting spacecraft. In: 2022 IEEE aerospace conference (AERO), pp 1\u20139","DOI":"10.1109\/AERO53065.2022.9843331"},{"key":"1312_CR28","doi-asserted-by":"crossref","unstructured":"Herrmann A, Schaub H (2023) Reinforcement learning for the agile earth-observing satellite scheduling problem. IEEE Trans Aerosp Electron Syst","DOI":"10.3389\/frspt.2023.1263489"},{"key":"1312_CR29","doi-asserted-by":"crossref","unstructured":"Chan DM, Agha-mohammadi A (2019) Autonomous imaging and mapping of small bodies using deep reinforcement learning. In: 2019 IEEE aerospace conference, pp 1\u201312","DOI":"10.1109\/AERO.2019.8742147"},{"key":"1312_CR30","doi-asserted-by":"crossref","unstructured":"Pesce V, Agha-mohammadi A, Lavagna M (2018) Autonomous navigation & mapping of small bodies. In: 2018 IEEE aerospace conference, pp 1\u201310","DOI":"10.1109\/AERO.2018.8396797"},{"key":"1312_CR31","doi-asserted-by":"crossref","unstructured":"Piccinin M, Lavagna MR (2020) Deep reinforcement learning approach for small bodies shape reconstruction enhancement. In: AIAA SciTech, AIAA Paper 2020\u20131909","DOI":"10.2514\/6.2020-1909"},{"key":"1312_CR32","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2021.107224","volume":"120","author":"M Piccinin","year":"2022","unstructured":"Piccinin M, Lunghi P, Lavagna M (2022) Deep reinforcement learning-based policy for autonomous imaging planning of small celestial bodies mapping. Aerosp Sci Technol 120:107224","journal-title":"Aerosp Sci Technol"},{"key":"1312_CR33","unstructured":"Konda V, Tsitsiklis J (1999) Actor-critic algorithms. Advances in neural information processing systems, p 12"},{"issue":"8","key":"1312_CR34","doi-asserted-by":"publisher","first-page":"1440","DOI":"10.2514\/1.G005794","volume":"44","author":"A Zavoli","year":"2021","unstructured":"Zavoli A, Federici L (2021) Reinforcement learning for robust trajectory design of interplanetary missions. J Guid Control Dyn 44(8):1440\u20131453","journal-title":"J Guid Control Dyn"},{"key":"1312_CR35","doi-asserted-by":"crossref","unstructured":"Brandonisio A, Capra L, Lavagna M (2023) Deep reinforcement learning spacecraft guidance with state uncertainty for autonomous shape reconstruction of uncooperative target. Advances in Space Research","DOI":"10.1016\/j.asr.2023.07.007"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01312-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01312-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01312-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T18:09:22Z","timestamp":1715882962000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01312-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,17]]},"references-count":35,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["1312"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01312-z","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2024,1,17]]},"assertion":[{"value":"26 June 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}