{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:33:29Z","timestamp":1760146409745,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,10,31]],"date-time":"2024-10-31T00:00:00Z","timestamp":1730332800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62303298"],"award-info":[{"award-number":["62303298"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>In the past decades, multi-agent systems have been a hot topic due to their wide applications, and the formation of multi-agent systems is a branch involving navigation, obstacle avoidance, controller design, and other issues. Due to the increasing requirements for accuracy and efficiency, as well as for a bridge to link the sensing and control stages, the importance of transmission scheduling is gradually emerging, with the scheduling of limited resources under various constraints to better complete tasks becoming a focus of attention. However, most of the literature only considers the formation process as a whole, while overlooking the discrepancies in the formation process at different time stages. In this paper, a multi-stage formation scheduling problem with limited communication resources is studied. A multi-stage model has been proposed based on the different completion levels of formation. Compared to the single-stage model, the proposed multi-stage model reflects the different requirements during the formation process. Furthermore, in order to save communication energy, three transmission modes have been defined to reduce energy consumption in terms of communication frequency and communication radius. Considering the need for the dynamic scheduling of coupled parameters, we propose a reinforcement-learning-based hybrid controller which includes a basic controller and a fuzzy controller. The hybrid controller, which continuously adjusts parameters according to the requirements of each stage, conducts a trade-off between system performance and energy consumption. Additionally, the reinforcement learning ensures that all parameters are optimal in the corresponding situation. The simulation results show that our controller ensures both dynamic and steady-state performance with lower energy consumption. The comparison with other scheduling strategies demonstrates the optimality and effectiveness of our proposed framework and algorithms.<\/jats:p>","DOI":"10.3390\/systems12110465","type":"journal-article","created":{"date-parts":[[2024,11,1]],"date-time":"2024-11-01T09:49:14Z","timestamp":1730454554000},"page":"465","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Learning-Based Multi-Stage Formation Scheduling with a Hybrid Controller"],"prefix":"10.3390","volume":"12","author":[{"given":"Zhichao","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Future Technology, Shanghai University, Shanghai 200444, China"},{"name":"Institute of Artificial Intelligence, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Li","sequence":"additional","affiliation":[{"name":"School of Future Technology, Shanghai University, Shanghai 200444, China"},{"name":"Institute of Artificial Intelligence, Shanghai University, Shanghai 200444, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2882","DOI":"10.1109\/TNSE.2022.3172567","article-title":"Scale-Free Collaborative Protocol Design for Output Synchronization of Heterogeneous Multi-Agent Systems with Nonuniform Communication Delays","volume":"9","author":"Liu","year":"2022","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"5328","DOI":"10.1109\/TCYB.2020.2995154","article-title":"Distributed Stabilization of Multiple Heterogeneous Agents in the Strong\u2013Weak Competition Network: A Switched System Approach","volume":"51","author":"Hu","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_3","first-page":"1129","article-title":"An Improved Artificial Potential Field Method for Path Planning and Formation Control of the Multi-UAV Systems","volume":"69","author":"Pan","year":"2022","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_4","first-page":"184","article-title":"Mean-Square Consensus of Heterogeneous Multi-Agent Systems with Time-Varying Communication Delays and Intermittent Observations","volume":"69","author":"Luo","year":"2022","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_5","first-page":"426","article-title":"Observer-Based Incremental Predictive Control of Networked Multi-Agent Systems with Random Delays and Packet Dropouts","volume":"68","author":"Pang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_6","first-page":"1282","article-title":"Cloud-Based Time-Varying Formation Predictive Control of Multi-Agent Systems with Random Communication Constraints and Quantized Signals","volume":"69","author":"Pang","year":"2022","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_7","first-page":"2473","article-title":"Robust H2 Consensus for Multi-Agent Systems with Parametric Uncertainties","volume":"68","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"13874","DOI":"10.1109\/TCYB.2021.3128051","article-title":"Distributed Time-Varying Optimization of Second-Order Multiagent Systems Under Limited Interaction Ranges","volume":"52","author":"Hong","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1528","DOI":"10.1109\/LWC.2021.3073170","article-title":"Consensus Analysis of Wireless Multi-Agent Systems Over Fading Channels","volume":"10","author":"Oral","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3212","DOI":"10.1109\/TAC.2021.3096899","article-title":"Delay Range for Consensus Achievable by Proportional and PD Feedback Protocols with Time-Varying Delays","volume":"67","author":"Ma","year":"2022","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1109\/JAS.2022.105917","article-title":"Group Hybrid Coordination Control of Multi-Agent Systems with Time-Delays and Additive Noises","volume":"10","author":"Li","year":"2023","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_12","first-page":"2184","article-title":"Sliding Mode Control for Multi-Agent Systems Under Event-Triggering Hybrid Scheduling Strategy","volume":"71","author":"Hu","year":"2024","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"5730","DOI":"10.1109\/TVT.2023.3333291","article-title":"Dynamic Scheduling of IoV Edge Cloud Service Functions Under NFV: A Multi-Agent Reinforcement Learning Approach","volume":"73","author":"Lu","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3027","DOI":"10.1109\/LRA.2020.2974715","article-title":"Spatial Scheduling of Informative Meetings for Multi-Agent Persistent Coverage","volume":"5","author":"Haksar","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1901","DOI":"10.1109\/LGRS.2020.3009823","article-title":"An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning","volume":"18","author":"Dalin","year":"2021","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1109\/TCNS.2022.3153608","article-title":"Decentralized Event-Triggered Scheduling and Control for Multiagent Linear Systems Over Wireless Interference Channels","volume":"9","author":"Tang","year":"2022","journal-title":"IEEE Trans. Control Netw. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"24334","DOI":"10.1109\/JIOT.2024.3391024","article-title":"Multiobjective Harris Hawks Optimization-Based Task Scheduling in Cloud-Fog Computing","volume":"11","author":"Ali","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4462","DOI":"10.1109\/JSEN.2020.3029430","article-title":"Radar Network Time Scheduling for Multi-Target ISAR Task with Game Theory and Multiagent Reinforcement Learning","volume":"21","author":"Liu","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1109\/TCCN.2023.3235719","article-title":"MAGNNETO: A Graph Neural Network-Based Multi-Agent System for Traffic Engineering","volume":"9","author":"Shi","year":"2023","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"642","DOI":"10.23919\/JSEE.2021.000055","article-title":"A single-task and multi-decision evolutionary game model based on multi-agent reinforcement learning","volume":"32","author":"Ye","year":"2021","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4962","DOI":"10.1109\/TNSM.2021.3139607","article-title":"Large-Scale Machine Learning Cluster Scheduling via Multi-Agent Graph Reinforcement Learning","volume":"19","author":"Zhao","year":"2022","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"289","DOI":"10.23919\/JSEE.2023.000045","article-title":"DQN-based decentralized multi-agent JSAP resource allocation for UAV swarm communication","volume":"34","author":"Li","year":"2023","journal-title":"J. Syst. Eng. Electron."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3020","DOI":"10.1109\/TASE.2021.3104716","article-title":"Real-Time Scheduling for Dynamic Partial-No-Wait Multiobjective Flexible Job Shop by Deep Reinforcement Learning","volume":"19","author":"Luo","year":"2022","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"940","DOI":"10.1109\/TMC.2020.3017079","article-title":"Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments Using A3C Learning and Residual Recurrent Neural Networks","volume":"21","author":"Tuli","year":"2022","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3590","DOI":"10.1109\/TNSE.2022.3185019","article-title":"Optimal Leader-Following Consensus Control of Multi-Agent Systems: A Neural Network Based Graphical Game Approach","volume":"9","author":"Ren","year":"2022","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3476","DOI":"10.1109\/TNSM.2021.3049381","article-title":"Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach","volume":"18","author":"Munir","year":"2021","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1109\/TSIPN.2023.3239654","article-title":"Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning","volume":"9","author":"Wang","year":"2023","journal-title":"IEEE Trans. Signal Inf. Process. Over Netw."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Li, Y., Yang, H., and Zheng, J. (2023, January 24\u201326). Energy-Efficient Multi-Stage Formation Scheduling Based on a Hybrid Controller. Proceedings of the 42nd Chinese Control Conference (CCC), Tianjin, China.","DOI":"10.23919\/CCC58697.2023.10239851"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/11\/465\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:26:10Z","timestamp":1760113570000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/11\/465"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,31]]},"references-count":28,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["systems12110465"],"URL":"https:\/\/doi.org\/10.3390\/systems12110465","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2024,10,31]]}}}