{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:22:36Z","timestamp":1760149356379,"version":"build-2065373602"},"reference-count":49,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,8,2]],"date-time":"2023-08-02T00:00:00Z","timestamp":1690934400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"science and technology research project of the Henan province","award":["222102240014"],"award-info":[{"award-number":["222102240014"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper addresses the problem of decentralized safety control (DSC) of constrained interconnected nonlinear safety-critical systems under reinforcement learning strategies, where asymmetric input constraints and security constraints are considered. To begin with, improved performance functions associated with the actuator estimates for each auxiliary subsystem are constructed. Then, the decentralized control problem with security constraints and asymmetric input constraints is transformed into an equivalent decentralized control problem with asymmetric input constraints using the barrier function. This approach ensures that safety-critical systems operate and learn optimal DSC policies within their safe global domains. Then, the optimal control strategy is shown to ensure that the entire system is uniformly ultimately bounded (UUB). In addition, all signals in the closed-loop auxiliary subsystem, based on Lyapunov theory, are uniformly ultimately bounded, and the effectiveness of the designed method is verified by practical simulation.<\/jats:p>","DOI":"10.3390\/e25081158","type":"journal-article","created":{"date-parts":[[2023,8,2]],"date-time":"2023-08-02T11:04:38Z","timestamp":1690974278000},"page":"1158","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Reinforcement Learning-Based Decentralized Safety Control for Constrained Interconnected Nonlinear Safety-Critical Systems"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8238-5922","authenticated-orcid":false,"given":"Chunbin","family":"Qin","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yinliang","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jishi","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Software, Henan University, Kaifeng 475000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianzeng","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450046, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Son, T.D., and Nguyen, Q. (2019, January 11\u201313). Safety-critical control for non-affine nonlinear systems with application on autonomous vehicle. Proceedings of the 2019 IEEE 58th Conference on Decision and Control (CDC), Nice, France.","DOI":"10.1109\/CDC40024.2019.9029446"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Manjunath, A., and Nguyen, Q. (2021, January 14\u201317). Safe and robust motion planning for dynamic robotics via control barrier functions. Proceedings of the 2021 60th IEEE Conference on Decision and Control (CDC), Austin, TX, USA.","DOI":"10.1109\/CDC45484.2021.9682803"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wang, J., Qin, C., Qiao, X., Zhang, D., Zhang, Z., Shang, Z., and Zhu, H. (2022). Constrained optimal control for nonlinear multi-input safety-critical systems with time-varying safety constraints. Mathematics, 10.","DOI":"10.3390\/math10152744"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1007\/s12239-021-0044-0","article-title":"A multi-objective model predictive control for vehicle adaptive cruise control system based on a new safe distance model","volume":"22","author":"Liu","year":"2021","journal-title":"Int. J. Automot. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3861","DOI":"10.1109\/TAC.2016.2638961","article-title":"Control barrier function based quadratic programs for safety critical systems","volume":"62","author":"Ames","year":"2016","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.neucom.2022.07.072","article-title":"Neural network-based safe optimal robust control for affine nonlinear systems with unmatched disturbances","volume":"506","author":"Qin","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1002\/rnc.6452","article-title":"Safe adaptive learning algorithm with neural network implementation for H\u221e control of nonlinear safety-critical system","volume":"33","author":"Qin","year":"2023","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"104895","DOI":"10.1016\/j.sysconle.2021.104895","article-title":"Extent-compatible control barrier functions","volume":"150","author":"Srinivasan","year":"2021","journal-title":"Syst. Control Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Yang, Y., Yin, Y., He, W., Vamvoudakis, K.G., and Modares, H. (2019, January 10\u201312). Safety-aware reinforcement learning framework with an actor-critic-barrier structure. Proceedings of the 2019 American Control Conference (ACC), Philadelphia, PA, USA.","DOI":"10.23919\/ACC.2019.8815335"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5441","DOI":"10.1109\/TNNLS.2020.2967871","article-title":"Safe intermittent reinforcement learning with static and dynamic event generators","volume":"31","author":"Yang","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3408","DOI":"10.1002\/rnc.5955","article-title":"Adaptive dynamic programming for optimal control of discrete-time nonlinear system with state constraints based on control barrier function","volume":"32","author":"Xu","year":"2022","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1146\/annurev-control-042920-020211","article-title":"Safe learning in robotics: From learning-based control to safe reinforcement learning","volume":"5","author":"Brunke","year":"2022","journal-title":"Annu. Rev. Control Robot. Auton. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"40324","DOI":"10.1109\/ACCESS.2022.3166473","article-title":"Event-triggered safe control for the zero-sum game of nonlinear safety-critical systems with input saturation","volume":"10","author":"Qin","year":"2022","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.arcontrol.2008.03.004","article-title":"Decentralized control: An overview","volume":"32","author":"Bakule","year":"2008","journal-title":"Annu. Rev. Control."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1109\/TFUZZ.2022.3183798","article-title":"Decentralized Event-Triggered Adaptive Control for Interconnected Nonlinear Systems With Actuator Failures","volume":"31","author":"Xu","year":"2022","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3429","DOI":"10.1007\/s11071-022-07771-2","article-title":"Robust NN-based decentralized optimal tracking control for interconnected nonlinear systems via adaptive dynamic programming","volume":"110","author":"Guo","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Feng, Z., Li, R.B., and Wu, L. (2023). Adaptive decentralized control for constrained strong interconnected nonlinear systems and its application to inverted pendulum. IEEE Trans. Neural Netw. Learn. Syst., 1\u201311.","DOI":"10.1109\/TNNLS.2023.3238819"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"19971","DOI":"10.1007\/s11042-020-10449-9","article-title":"Stability analysis of interconnected complex nonlinear systems using the Lyapunov and Finsler property","volume":"80","author":"Zouhri","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.neucom.2022.02.078","article-title":"Adaptive neural network decentralized fault-tolerant control for nonlinear interconnected fractional-order systems","volume":"488","author":"Li","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tan, Y., Yuan, Y., Xie, X., Tian, E., and Liu, J. (2023). Observer-based event-triggered control for interval type-2 fuzzy networked system with network attacks. IEEE Trans. Fuzzy Syst., 1\u201310.","DOI":"10.1109\/TFUZZ.2023.3237846"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"10761","DOI":"10.1109\/TCYB.2021.3069461","article-title":"Adaptive fuzzy decentralized dynamic surface control for switched large-scale nonlinear systems with full-state constraints","volume":"52","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"7478","DOI":"10.1109\/TCYB.2020.3037321","article-title":"Adaptive-critic design for decentralized event-triggered control of constrained nonlinear interconnected systems within an identifier-critic framework","volume":"52","author":"Huo","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bao, C., Wang, P., and Tang, G. (2022). Data-Driven Based Model-Free Adaptive Optimal Control Method for Hypersonic Morphing Vehicle. IEEE Trans. Aerosp. Electron. Syst., 1\u201315.","DOI":"10.1109\/TAES.2022.3230633"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3462","DOI":"10.1080\/00207179.2021.1976420","article-title":"Distributed optimal control for continuous-time nonaffine nonlinear interconnected systems","volume":"95","author":"Farzanegan","year":"2022","journal-title":"Int. J. Control"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"111465","DOI":"10.1016\/j.chaos.2021.111465","article-title":"A numerical approach for a class of nonlinear optimal control problems with piecewise fractional derivative","volume":"152","author":"Heydari","year":"2021","journal-title":"Chaos Solitons Fractals"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1007\/s11071-022-07459-7","article-title":"Data-driven-based event-triggered optimal control of unknown nonlinear systems with input constraints","volume":"109","author":"Liu","year":"2022","journal-title":"Nonlinear Dyn."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1109\/JAS.2021.1004246","article-title":"Adaptive decentralized asymptotic tracking control for large-scale nonlinear systems with unknown strong interconnections","volume":"9","author":"Niu","year":"2021","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.neunet.2020.09.020","article-title":"Particle swarm optimized neural networks based local tracking control scheme of unknown nonlinear interconnected systems","volume":"134","author":"Zhao","year":"2021","journal-title":"Neural Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"126163","DOI":"10.1016\/j.neucom.2023.03.024","article-title":"Event-triggered optimal decentralized control for stochastic interconnected nonlinear systems via adaptive dynamic programming","volume":"539","author":"Zhao","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"119070","DOI":"10.1016\/j.ins.2023.119070","article-title":"Sliding-mode surface-based decentralized event-triggered control of partially unknown interconnected nonlinear systems via reinforcement learning","volume":"641","author":"Wang","year":"2023","journal-title":"Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MCAS.2009.933854","article-title":"Reinforcement learning and adaptive dynamic programming for feedback control","volume":"9","author":"Lewis","year":"2009","journal-title":"IEEE Circuits Syst. Mag."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.neunet.2022.06.039","article-title":"Periodic event-triggered adaptive tracking control design for nonlinear discrete-time systems via reinforcement learning","volume":"154","author":"Tang","year":"2022","journal-title":"Neural Netw."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"8412","DOI":"10.1016\/j.jfranklin.2018.08.024","article-title":"Backstepping-based adaptive dynamic programming for missile-target guidance systems with state and input constraints","volume":"355","author":"Sun","year":"2018","journal-title":"J. Frankl. Inst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1016\/j.neucom.2021.12.057","article-title":"Goal representation adaptive critic design for discrete-time uncertain systems subjected to input constraints: The event-triggered case","volume":"492","author":"Zhao","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neucom.2018.09.011","article-title":"Integral reinforcement learning based decentralized optimal tracking control of unknown nonlinear large-scale interconnected systems with constrained-input","volume":"323","author":"Liu","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4491","DOI":"10.1002\/rnc.5487","article-title":"Adaptive decentralized finite-time tracking control for uncertain interconnected nonlinear systems with input quantization","volume":"31","author":"Sun","year":"2021","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1049\/cth2.12116","article-title":"Finite-horizon optimal tracking control for constrained-input nonlinear interconnected system using aperiodic distributed nonzero-sum games","volume":"15","author":"Duan","year":"2021","journal-title":"IET Control Theory Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3790","DOI":"10.1109\/TAC.2022.3197562","article-title":"Event-based finite-time control for nonlinear multi-agent systems with asymptotic tracking","volume":"68","author":"Li","year":"2023","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"4198","DOI":"10.1109\/TNSE.2022.3196316","article-title":"Fully distributed consensus of switched heterogeneous nonlinear multi-agent systems with bouc-wen hysteresis input","volume":"9","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4187","DOI":"10.1109\/TSMC.2021.3089944","article-title":"Adaptive critics for decentralized stabilization of constrained-input nonlinear interconnected systems","volume":"52","author":"Yang","year":"2021","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"113034","DOI":"10.1016\/j.chaos.2022.113034","article-title":"Reinforcement learning-based decentralized fault tolerant control for constrained interconnected nonlinear systems","volume":"167","author":"Zhao","year":"2023","journal-title":"Chaos Solitons Fractals"},{"key":"ref_42","first-page":"125871","article-title":"Event-triggered distributed self-learning robust tracking control for uncertain nonlinear interconnected systems","volume":"395","author":"Cui","year":"2021","journal-title":"Appl. Math. Comput."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"9902","DOI":"10.1002\/rnc.6365","article-title":"Robust tracking control with reinforcement learning for nonlinear-constrained systems","volume":"32","author":"Tang","year":"2022","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1109\/JAS.2020.1003063","article-title":"Optimal neuro-control strategy for nonlinear systems with asymmetric input constraints","volume":"7","author":"Yang","year":"2020","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"2159","DOI":"10.1016\/S0005-1098(97)00128-3","article-title":"Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation","volume":"33","author":"Beard","year":"1997","journal-title":"Automatica"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1372","DOI":"10.1109\/TCYB.2015.2417170","article-title":"Reinforcement-learning-based robust controller design for continuous-time uncertain nonlinear systems subject to input constraints","volume":"45","author":"Liu","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"111930","DOI":"10.1016\/j.chaos.2022.111930","article-title":"Fault-tolerant adaptive fractional controller design for incommensurate fractional-order nonlinear dynamic systems subject to input and output restrictions","volume":"157","author":"Pishro","year":"2022","journal-title":"Chaos Solitons Fractals"},{"key":"ref_48","first-page":"1367","article-title":"Real-time reachable set control for neutral singular Markov jump systems with mixed delays","volume":"69","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"111257","DOI":"10.1016\/j.chaos.2021.111257","article-title":"Fuzzy logic and gradient descent-based optimal adaptive robust controller with inverted pendulum verification","volume":"151","author":"Lakmesari","year":"2021","journal-title":"Chaos Solitons Fractals"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/8\/1158\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:24:20Z","timestamp":1760127860000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/8\/1158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,2]]},"references-count":49,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["e25081158"],"URL":"https:\/\/doi.org\/10.3390\/e25081158","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,8,2]]}}}