{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:36:24Z","timestamp":1760232984164,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T00:00:00Z","timestamp":1670976000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and ICT","award":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"],"award-info":[{"award-number":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"]}]},{"name":"Ministry of Education of the Republic of Korea","award":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"],"award-info":[{"award-number":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"]}]},{"name":"National Research Foundation of Korea","award":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"],"award-info":[{"award-number":["NRF-2017R1A2B4007398","NRF-2020S1A5A2A03045921"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Recently, there has been a growing interest in the consensus of a multi-agent system (MAS) with advances in artificial intelligence and distributed computing. Sliding mode control (SMC) is a well-known method that provides robust control in the presence of uncertainties. While our previous study introduced SMC to the reinforcement learning (RL) based on approximate dynamic programming in the context of optimal control, SMC is introduced to a conventional RL framework in this work. As a specific realization, the modified twin delayed deep deterministic policy gradient (DDPG) for consensus was exploited to develop sliding mode RL. Numerical experiments show that the sliding mode RL outperforms existing state-of-the-art RL methods and model-based methods in terms of the mean square error (MSE) performance.<\/jats:p>","DOI":"10.3390\/s22249811","type":"journal-article","created":{"date-parts":[[2022,12,14]],"date-time":"2022-12-14T03:21:52Z","timestamp":1670988112000},"page":"9811","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Reinforcement Learning with Side Information for the Uncertainties"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4073-6747","authenticated-orcid":false,"given":"Janghoon","family":"Yang","sequence":"first","affiliation":[{"name":"Department of A.I. Software Engineering, Seoul Media Institute of Technology, Seoul 07590, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1109\/JPROC.2003.814926","article-title":"Distributed Control Applications within Sensor Networks","volume":"91","author":"Sinopoli","year":"2003","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, X., and Papachristodoulou, A. (2014, January 4\u20136). A distributed PID controller for network congestion control problems. Proceedings of the American Control Conference, Portland, OR, USA.","DOI":"10.1109\/ACC.2014.6858963"},{"key":"ref_3","unstructured":"Jia, D., and Krogh, B. (2002, January 8\u201310). Min-Max Feedback Model Predictive Control for Distributed Control with Communication. Proceedings of the American Control Conference, Anchorage, AK, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.1109\/TII.2019.2905295","article-title":"A Survey on Model-Based Distributed Control and Filtering for Industrial Cyber-Physical Systems","volume":"15","author":"Ding","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"28573","DOI":"10.1109\/ACCESS.2018.2831228","article-title":"Multi-Agent Systems: A Survey","volume":"6","author":"Dorri","year":"2018","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jaimes, A., and Jamshidi, M.M. (2010, January 1\u20136). Consensus-based and network control of UAVs. Proceedings of the 5th International Conference on System of Systems Engineering, Loughborough, UK.","DOI":"10.1109\/SYSOSE.2010.5544106"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zhai, C., Gao, H., and Chen, L. (2017, January 20\u201322). Consensus of autonomous vehicle platoon with time delays. Proceedings of the Chinese Automation Congress (CAC), Jinan, China.","DOI":"10.1109\/CAC.2017.8243591"},{"key":"ref_8","unstructured":"Contzen, M.P., and Raisch, J. (July, January 29). Reactive power consensus in microgrids. Proceedings of the European Control Conference (ECC), Aalborg, Denmark."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"17271","DOI":"10.1109\/ACCESS.2017.2740938","article-title":"Consensus of Teleoperating Cyber-Physical System via Centralized and Decentralized Controllers","volume":"5","author":"Yan","year":"2017","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1520","DOI":"10.1109\/TAC.2004.834113","article-title":"Consensus problems in networks of agents with switching topology and time-delays","volume":"49","author":"Murray","year":"2004","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1109\/TCYB.2017.2651026","article-title":"Asynchronous Periodic Edge-Event Triggered Control for Double-Integrator Networks with Communication Time Delays","volume":"48","author":"Duan","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1259","DOI":"10.1109\/TCYB.2018.2799798","article-title":"Predictor-Based Extended-State-Observer Design for Consensus of MASs with Delays and Disturbances","volume":"49","author":"Wang","year":"2019","journal-title":"IEEE Trans. Cybern."},{"key":"ref_13","first-page":"13","article-title":"Cooperative Containment Control of Multiagent Systems Based on Follower Observers with Time Delay","volume":"47","author":"Wang","year":"2017","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1049\/iet-cta.2012.0855","article-title":"Leaderless and leader-following consensus for heterogeneous multi-agent systems with random link failures","volume":"8","author":"Kim","year":"2014","journal-title":"IET Control Theory Appl."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1209","DOI":"10.1007\/s12555-011-0623-3","article-title":"Average consensus seeking of high-order continuous-time multi-agent systems with multiple time-varying communication delays","volume":"9","author":"Zhang","year":"2011","journal-title":"Int. J. Control Autom. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1016\/j.sysconle.2011.07.004","article-title":"Consensus in multi-agent systems with random delays governed by a Markov chain","volume":"60","author":"Wu","year":"2011","journal-title":"Syst. Control Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sun, Y.-J., Zhang, G.-L., and Zeng, J. (2014). Consensus Analysis for a Class of Heterogeneous Multiagent Systems with Time Delay Based on Frequency Domain Method. Math. Probl. Eng., 2014.","DOI":"10.1155\/2014\/248684"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1016\/j.matcom.2010.10.003","article-title":"Sliding mode control: A survey with applications in math","volume":"81","author":"Pisano","year":"2011","journal-title":"Math. Comput. Simul."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ghayoomi, P., and Ghasemi, R. (2017, January 19\u201320). Observer based sliding mode consensus controller design for nonlinear multi-agent systems. Proceedings of the International Conference on Inventive Systems and Control (ICISC), Coimbatore, India.","DOI":"10.1109\/ICISC.2017.8068735"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhao, N., and Zhu, J.-D. (2021). Robust Consensus Problem of Heterogeneous Uncertain Second-Order Multi-Agent Systems Based on Sliding Mode Control. Front. Control Eng., 2.","DOI":"10.3389\/fcteg.2021.744027"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"47275","DOI":"10.1109\/ACCESS.2022.3171825","article-title":"Consensus of Discrete-Time Nonlinear Multiagent Systems Using Sliding Mode Control Based on Optimal Control","volume":"10","author":"Yuan","year":"2022","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3692","DOI":"10.1016\/j.jfranklin.2019.01.047","article-title":"Integral Sliding-Mode Fixed-Time Consensus Tracking for Second-Order Non-Linear and Time Delay Multi-Agent Systems","volume":"356","author":"Wang","year":"2019","journal-title":"J. Frankl. Inst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"55844","DOI":"10.1109\/ACCESS.2021.3070388","article-title":"A Consensus Control for a Multi-Agent System with Unknown Time-Varying Communication Delays","volume":"9","author":"Yang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4091","DOI":"10.1109\/TIE.2016.2542134","article-title":"Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems with Unknown Dynamics Using Reinforcement Learning Method","volume":"64","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, X., and Su, H. (2020). Completely model-free RL-based consensus of continuous-time multi-agent systems. Appl. Math. Comput., 382.","DOI":"10.1016\/j.amc.2020.125312"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, J., Ji, L., and Li, H. (2021). Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method. Appl. Math. Comput., 410.","DOI":"10.1016\/j.amc.2021.126451"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yang, J. (2022). Deep Learning-Based Consensus Control of a Multi-Agents System with Unknown Time-varying Delay. Electronics, 11.","DOI":"10.3390\/electronics11081176"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.9728\/dcs.2022.23.7.1277","article-title":"Reinforcement Learning for the Consensus of Multi-agents with Unknown Time Varying Delays","volume":"23","author":"Yang","year":"2022","journal-title":"J. Digit. Contents Soc."},{"key":"ref_29","first-page":"226","article-title":"A slide reinforcement learning for the consensus of a multi-agents system","volume":"26","author":"Yang","year":"2022","journal-title":"J. Adv. Navig. Technol."},{"key":"ref_30","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing Function Approximation Error in Actor-Critic Methods. In Proceeding of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.automatica.2018.07.018","article-title":"A new condition for asymptotic consensus over switching graphs","volume":"97","author":"Chowdhury","year":"2018","journal-title":"Automatica"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9811\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:40:59Z","timestamp":1760146859000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9811"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,14]]},"references-count":31,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22249811"],"URL":"https:\/\/doi.org\/10.3390\/s22249811","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,12,14]]}}}