{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T21:38:03Z","timestamp":1775770683309,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,6,11]],"date-time":"2022-06-11T00:00:00Z","timestamp":1654905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["U1504615"],"award-info":[{"award-number":["U1504615"]}]},{"name":"National Natural Science Foundation of China","award":["2018GGJS017"],"award-info":[{"award-number":["2018GGJS017"]}]},{"name":"National Natural Science Foundation of China","award":["222102240014"],"award-info":[{"award-number":["222102240014"]}]},{"name":"Youth Backbone Teachers in Colleges and Universities of Henan Province","award":["U1504615"],"award-info":[{"award-number":["U1504615"]}]},{"name":"Youth Backbone Teachers in Colleges and Universities of Henan Province","award":["2018GGJS017"],"award-info":[{"award-number":["2018GGJS017"]}]},{"name":"Youth Backbone Teachers in Colleges and Universities of Henan Province","award":["222102240014"],"award-info":[{"award-number":["222102240014"]}]},{"name":"Science and Technology Research Project of the Henan Province","award":["U1504615"],"award-info":[{"award-number":["U1504615"]}]},{"name":"Science and Technology Research Project of the Henan Province","award":["2018GGJS017"],"award-info":[{"award-number":["2018GGJS017"]}]},{"name":"Science and Technology Research Project of the Henan Province","award":["222102240014"],"award-info":[{"award-number":["222102240014"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, a robust trajectory tracking control method with state constraints and uncertain disturbances on the ground of adaptive dynamic programming (ADP) is proposed for nonlinear systems. Firstly, the augmented system consists of the tracking error and the reference trajectory, and the tracking control problems with uncertain disturbances is described as the problem of robust control adjustment. In addition, considering the nominal system of the augmented system, the guaranteed cost tracking control problem is transformed into the optimal control problem by using the discount coefficient in the nominal system. A new safe Hamilton\u2013Jacobi\u2013Bellman (HJB) equation is proposed by combining the cost function with the control barrier function (CBF), so that the behavior of violating the safety regulations for the system states will be punished. In order to solve the new safe HJB equation, a critic neural network (NN) is used to approximate the solution of the safe HJB equation. According to the Lyapunov stability theory, in the case of state constraints and uncertain disturbances, the system states and the parameters of the critic neural network are guaranteed to be uniformly ultimately bounded (UUB). At the end of this paper, the feasibility of the proposed method is verified by a simulation example.<\/jats:p>","DOI":"10.3390\/e24060816","type":"journal-article","created":{"date-parts":[[2022,6,11]],"date-time":"2022-06-11T09:33:26Z","timestamp":1654940006000},"page":"816","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Robust Trajectory Tracking Control for Continuous-Time Nonlinear Systems with State Constraints and Uncertain Disturbances"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8238-5922","authenticated-orcid":false,"given":"Chunbin","family":"Qin","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaopeng","family":"Qiao","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3172-1498","authenticated-orcid":false,"given":"Jinguang","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dehua","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence, Henan University, Zhengzhou 450000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Xie, R., Tian, X., Zhang, Q., and Pan, X. (2021, January 19\u201321). Research on longitudinal control algorithm for intelligent automatic driving. Proceedings of the 2021 IEEE 4th International Conference on Automation, Electronics and Electrical Engineering (AUTEEE), Shenyang, China.","DOI":"10.1109\/AUTEEE52864.2021.9668762"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1109\/TCST.2017.2654063","article-title":"Obstacle Avoidance for Low-Speed Autonomous Vehicles with Barrier Function","volume":"26","author":"Chen","year":"2018","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1109\/LRA.2021.3135569","article-title":"Model-Free Safety-Critical Control for Robotic Systems","volume":"7","author":"Molnar","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"103388","DOI":"10.1016\/j.robot.2019.103388","article-title":"Safety barrier functions and multi-camera tracking for human\u2013robot shared environment","volume":"124","author":"Ferraguti","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1109\/TSMC.2020.3042876","article-title":"Adaptive Dynamic Programming for Control: A Survey and Recent Advances","volume":"51","author":"Liu","year":"2021","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"303","DOI":"10.3724\/SP.J.1004.2013.00303","article-title":"An overview of research on adaptive dynamic programming","volume":"39","author":"Zhang","year":"2013","journal-title":"Acta Autom. Sin."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MCI.2009.932261","article-title":"Adaptive dynamic programming: An introduction","volume":"4","author":"Wang","year":"2009","journal-title":"IEEE Comput. Intell. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1016\/j.automatica.2010.02.018","article-title":"Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem","volume":"46","author":"Vamvoudakis","year":"2010","journal-title":"Automatica"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1641","DOI":"10.1109\/TSMCB.2012.2227253","article-title":"Zero-Sum Two-Player Game Theoretic Formulation of Affine Nonlinear Discrete-Time Systems Using Neural Networks","volume":"43","author":"Mehraeen","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_10","unstructured":"Munos, R., Baird, L., and Moore, A. (1999, January 10\u201316). Gradient descent approaches to neural-net-based solutions of the Hamilton Jacobi Bellman equation. Proceedings of the International Joint Conference on Neural Networks, Washington, DC, USA."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1109\/TCYB.2015.2492242","article-title":"Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems","volume":"46","author":"Wei","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4958","DOI":"10.1109\/TCYB.2019.2926631","article-title":"Continuous-time time-varying policy iteration","volume":"50","author":"Wei","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1167","DOI":"10.1016\/j.automatica.2014.02.015","article-title":"Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics","volume":"50","author":"Kiumarsi","year":"2014","journal-title":"Automatica"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1109\/TNNLS.2015.2464080","article-title":"Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP","volume":"27","author":"Wei","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3051","DOI":"10.1109\/TAC.2014.2317301","article-title":"Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning","volume":"59","author":"Modares","year":"2014","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1080\/00207179.2013.863432","article-title":"Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming","volume":"87","author":"Qin","year":"2014","journal-title":"Int. J. Control"},{"key":"ref_17","first-page":"220","article-title":"Event-triggered single-network ADP method for constrained optimal tracking control of continuous-time non-linear systems","volume":"352","author":"Cui","year":"2019","journal-title":"Appl. Math. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.neunet.2017.09.005","article-title":"Neural network robust tracking control with adaptive critic framework for uncertain nonlinear systems","volume":"97","author":"Wang","year":"2018","journal-title":"Neural Netw."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.neucom.2021.10.081","article-title":"Robust tracking control of uncertain nonlinear systems with adaptive dynamic programming","volume":"471","author":"Zhao","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, D., Liu, D., Li, H., and Ma, H. (2015, January 1\u20133). Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems. Proceedings of the American Control Conference, Chicago, IL, USA.","DOI":"10.1109\/ACC.2015.7171175"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2550","DOI":"10.1109\/TNNLS.2015.2441749","article-title":"H\u221e Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning","volume":"26","author":"Modares","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"9506","DOI":"10.1109\/ACCESS.2021.3140076","article-title":"Adaptive Event-Triggered Near-Optimal Tracking Control for Unknown Continuous-Time Nonlinear Systems","volume":"10","author":"Wang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1016\/j.automatica.2014.05.011","article-title":"Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning","volume":"50","author":"Modares","year":"2014","journal-title":"Automatica"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"654","DOI":"10.1109\/TIE.2017.2722424","article-title":"Adaptive-Critic-Based Robust Trajectory Tracking of Uncertain Dynamics and Its Application to A Spring-Mass-Damper System","volume":"65","author":"Wang","year":"2018","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3024","DOI":"10.1177\/0142331220940161","article-title":"Compensator-based approximate optimal control for affine nonlinear systems with input constraints and unmatched disturbances","volume":"42","author":"Lu","year":"2020","journal-title":"Trans. Inst. Meas. Control"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2834","DOI":"10.1109\/TCYB.2014.2357896","article-title":"Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems","volume":"44","author":"Liu","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1676","DOI":"10.1049\/iet-cta.2014.0186","article-title":"Online approximate optimal control for affine nonlinear systems with unknown internal dynamics using adaptive dynamic programming","volume":"8","author":"Yang","year":"2014","journal-title":"IET Control Theory Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1049\/iet-cta.2015.0769","article-title":"Using reinforcement learning techniques to solve continuous-time nonlinear optimal tracking problem without system dynamics","volume":"10","author":"Zhu","year":"2016","journal-title":"IET Control Theory Appl."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Marvi, Z., and Kiumarsi, B. (2020, January 1\u20133). Safe Off-policy Reinforcement Learning Using Barrier Functions. Proceedings of the 2020 American Control Conference (ACC), Denver, USA.","DOI":"10.23919\/ACC45564.2020.9147584"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3316","DOI":"10.1016\/j.jfranklin.2019.12.017","article-title":"Online barrier-actor-critic learning for H\u221e control with full-state constraints and input saturation","volume":"357","author":"Yang","year":"2020","journal-title":"J. Frankl. Inst."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ames, A.D., Grizzle, J.W., and Tabuada, P. (2014, January 15\u201317). Control barrier function based quadratic programs with application to adaptive cruise control. Proceedings of the 53rd IEEE Conference on Decision and Control, Los Angeles, CA, USA.","DOI":"10.1109\/CDC.2014.7040372"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3706","DOI":"10.1002\/rnc.4962","article-title":"Safe reinforcement learning for dynamical games","volume":"30","author":"Yang","year":"2020","journal-title":"Int. J. Robust Nonlinear"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1109\/TRO.2020.3031254","article-title":"Control of Mobile Robots Using Barrier Functions Under Temporal Logic Specifications","volume":"37","author":"Srinivasan","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Cohen, M., and Belta, C. (2020, January 14\u201318). Approximate Optimal Control for Safety-Critical Systems with Control Barrier Functions. Proceedings of the 59th IEEE Conference on Decision and Control (CDC), Jeju, Korea.","DOI":"10.1109\/CDC42340.2020.9303896"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ames, A.D., Coogan, S., Egerstedt, M., Notomista, G., Sreenath, K., and Tabuada, P. (2019, January 25\u201328). Control Barrier Functions: Theory and Applications. Proceedings of the 18th European Control Conference (ECC), Naples, Italy.","DOI":"10.23919\/ECC.2019.8796030"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3408","DOI":"10.1002\/rnc.5955","article-title":"Adaptive dynamic programming for optimal control of discrete-time nonlinear system with state constraints based on control barrier function","volume":"32","author":"Xu","year":"2022","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.neucom.2015.08.119","article-title":"Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming","volume":"198","author":"Yang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lewis, F.L., Vrabie, D.L., and Syrmos, V.L. (2012). Optimal Control, John Wiley & Sons. [3rd ed.].","DOI":"10.1002\/9781118122631"},{"key":"ref_39","unstructured":"Lewis, F.L., Jagannathan, S., and Yesildirek, A. (1999). Neural Network Control of Robot Manipulators and Nonlinear Systems, Taylor& Francis."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/6\/816\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:28:04Z","timestamp":1760138884000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/6\/816"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,11]]},"references-count":39,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["e24060816"],"URL":"https:\/\/doi.org\/10.3390\/e24060816","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,11]]}}}