{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,20]],"date-time":"2026-06-20T16:37:57Z","timestamp":1781973477266,"version":"3.54.5"},"reference-count":28,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T00:00:00Z","timestamp":1674432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Open Research Fund of The State Key Laboratory for Management and Control of Complex Systems","award":["20220118"],"award-info":[{"award-number":["20220118"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper focuses on the optimal containment control problem for the nonlinear multiagent systems with partially unknown dynamics via an integral reinforcement learning algorithm. By employing integral reinforcement learning, the requirement of the drift dynamics is relaxed. The integral reinforcement learning method is proved to be equivalent to the model-based policy iteration, which guarantees the convergence of the proposed control algorithm. For each follower, the Hamilton\u2013Jacobi\u2013Bellman equation is solved by a single critic neural network with a modified updating law which guarantees the weight error dynamic to be asymptotically stable. Through using input\u2013output data, the approximate optimal containment control protocol of each follower is obtained by applying the critic neural network. The closed-loop containment error system is guaranteed to be stable under the proposed optimal containment control scheme. Simulation results demonstrate the effectiveness of the presented control scheme.<\/jats:p>","DOI":"10.3390\/e25020221","type":"journal-article","created":{"date-parts":[[2023,1,24]],"date-time":"2023-01-24T01:29:45Z","timestamp":1674523785000},"page":"221","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Integral Reinforcement-Learning-Based Optimal Containment Control for Partially Unknown Nonlinear Multiagent Systems"],"prefix":"10.3390","volume":"25","author":[{"given":"Qiuye","family":"Wu","sequence":"first","affiliation":[{"name":"School of Automation, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongheng","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Automation, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0051-7224","authenticated-orcid":false,"given":"Yonghua","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Automation, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"106635","DOI":"10.1016\/j.compag.2021.106635","article-title":"Intelligent IoT-multiagent precision irrigation approach for improving water use efficiency in irrigation systems at farm and district scales","volume":"192","author":"Jimenez","year":"2022","journal-title":"Comp. Electr. Agric."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"103243","DOI":"10.1016\/j.engappai.2019.103243","article-title":"Multi-agent architecture for information retrieval and intelligent monitoring by UAVs in known environments affected by catastrophes","volume":"87","author":"Vallejo","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2197","DOI":"10.1109\/TPWRS.2020.3040310","article-title":"Multi-agent based optimal scheduling and trading for multi-microgrids integrated with urban transportation networks","volume":"36","author":"Liu","year":"2021","journal-title":"IEEE Trans. Power. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.isatra.2022.01.004","article-title":"Neuro-adaptive containment control of unmanned surface vehicles with disturbance observer and collision-free","volume":"129","author":"Deng","year":"2022","journal-title":"ISA Trans."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2174","DOI":"10.1109\/JSYST.2017.2721220","article-title":"A multi-cooperative-based approach to manage communication in wireless instrumentation systems","volume":"12","author":"Hamani","year":"2018","journal-title":"IEEE Syst. J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1109\/TAC.2005.846556","article-title":"Consensus seeking in multiagent systems under dynamically changing interaction topologies","volume":"50","author":"Ren","year":"2005","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"4690","DOI":"10.1016\/j.jfranklin.2019.03.014","article-title":"Coordination of nonholonomic mobile robots for diffusive threat defense","volume":"356","author":"Luo","year":"2019","journal-title":"J. Frankl. Inst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2077","DOI":"10.1109\/TNNLS.2019.2927887","article-title":"Distributed finite-time fault-tolerant containment control for multiple unmanned aerial vehicles","volume":"31","author":"Yu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/TCYB.2020.2970454","article-title":"Observer-based fuzzy adaptive finite-time containment control of nonlinear multiagent systems with input delay","volume":"51","author":"Li","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Li, Z., Xue, H., Pan, Y., and Liang, H. (2022). Distributed adaptive event-triggered containment control for multi-agent systems under a funnel function. Int. J. Robust Nonlinear Control.","DOI":"10.1002\/rnc.6344"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, Y., Liu, M., Lian, J., and Guo, Y. (2022). Collaborative optimal formation control for heterogeneous multi-agent systems. Entropy, 24.","DOI":"10.3390\/e24101440"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1278","DOI":"10.1109\/TSMC.2019.2896614","article-title":"Command filtered backstepping-based attitude containment control for spacecraft formation","volume":"51","author":"Zhao","year":"2021","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, D., Wei, Q., Wang, D., Yang, X., and Li, H. (2017). Adaptive Dynamic Programming with Applications in Optimal Control, Springer.","DOI":"10.1007\/978-3-319-50815-3"},{"key":"ref_14","unstructured":"Bellman, R.E. (1957). Dynamic Programming, Princeton Univ. Press."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1016\/j.automatica.2004.11.034","article-title":"Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach","volume":"41","author":"Lewis","year":"2005","journal-title":"Automatica"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1109\/TSMC.2020.3042876","article-title":"Adaptive dynamic programming for control: A survey and recent advances","volume":"51","author":"Liu","year":"2021","journal-title":"IEEE Trans. Syst. Man. Cybern. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1598","DOI":"10.1016\/j.automatica.2012.05.074","article-title":"Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality","volume":"48","author":"Vamvoudakis","year":"2012","journal-title":"Automatica"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1109\/TFUZZ.2014.2310238","article-title":"Leader-based optimal coordination control for the consensus problem of multiagent differential games via fuzzy adaptive dynamic programming","volume":"23","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Fuzzy. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.isatra.2019.01.021","article-title":"Distributed optimal coordination control for nonlinear multi-agent systems using event-triggered adaptive dynamic programming method","volume":"91","author":"Zhao","year":"2019","journal-title":"ISA Trans."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2799","DOI":"10.1007\/s11071-022-07586-1","article-title":"Simplified optimized finite-time containment control for a class of multi-agent systems with actuator faults","volume":"109","author":"Cui","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1651","DOI":"10.1007\/s11071-022-07513-4","article-title":"Event-triggered optimal containment control for multi-agent systems subject to state constraints via reinforcement learning","volume":"109","author":"Xu","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"896","DOI":"10.1109\/TCSI.2021.3121809","article-title":"Distributed reinforcement learning containment control for multiple nonholonomic mobile robots","volume":"69","author":"Xiao","year":"2022","journal-title":"IEEE Trans. Circuits Syst. I Reg. Papers"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"109081","DOI":"10.1016\/j.automatica.2020.109081","article-title":"Off-policy learning for adaptive optimal output synchronization of heterogeneous multi-agent systems","volume":"119","author":"Chen","year":"2020","journal-title":"Automatica"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1016\/j.neucom.2021.01.063","article-title":"Finite-horizon robust formation-containment control of multi-agent networks with unknown dynamics","volume":"458","author":"Yu","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3197","DOI":"10.1109\/TCYB.2017.2761878","article-title":"Optimal robust output containment of unknown heterogeneous multiagent system using off-policy reinforcement learning","volume":"48","author":"Zuo","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1002\/acs.2950","article-title":"Observer-based adaptive optimal output containment control problem of linear heterogeneous Multiagent systems with relative output measurements","volume":"33","author":"Mazouchi","year":"2019","journal-title":"Int. J. Adapt. Control Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1109\/TCST.2018.2794336","article-title":"Optimal containment control of unknown heterogeneous systems with active leaders","volume":"27","author":"Yang","year":"2019","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3026","DOI":"10.1109\/TIE.2011.2160140","article-title":"Lyapunov, adaptive, and optimal design techniques for cooperative systems on directed communication graphs","volume":"59","author":"Zhang","year":"2012","journal-title":"IEEE Trans. Ind. Electron."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/221\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:14:25Z","timestamp":1760120065000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,23]]},"references-count":28,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["e25020221"],"URL":"https:\/\/doi.org\/10.3390\/e25020221","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,23]]}}}