{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T06:55:41Z","timestamp":1782370541175,"version":"3.54.5"},"reference-count":27,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,20]],"date-time":"2021-08-20T00:00:00Z","timestamp":1629417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005073","name":"Agency for Defense Development","doi-asserted-by":"publisher","award":["UD190031RD"],"award-info":[{"award-number":["UD190031RD"]}],"id":[{"id":"10.13039\/501100005073","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003626","name":"Defense Acquisition Program Administration","doi-asserted-by":"publisher","award":["UD190031RD"],"award-info":[{"award-number":["UD190031RD"]}],"id":[{"id":"10.13039\/501100003626","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The paper develops the adaptive dynamic programming toolbox (ADPT), which is a MATLAB-based software package and computationally solves optimal control problems for continuous-time control-affine systems. The ADPT produces approximate optimal feedback controls by employing the adaptive dynamic programming technique and solving the Hamilton\u2013Jacobi\u2013Bellman equation approximately. A novel implementation method is derived to optimize the memory consumption by the ADPT throughout its execution. The ADPT supports two working modes: model-based mode and model-free mode. In the former mode, the ADPT computes optimal feedback controls provided the system dynamics. In the latter mode, optimal feedback controls are generated from the measurements of system trajectories, without the requirement of knowledge of the system model. Multiple setting options are provided in the ADPT, such that various customized circumstances can be accommodated. Compared to other popular software toolboxes for optimal control, the ADPT features computational precision and time efficiency, which is illustrated with its applications to a highly non-linear satellite attitude control problem.<\/jats:p>","DOI":"10.3390\/s21165609","type":"journal-article","created":{"date-parts":[[2021,8,22]],"date-time":"2021-08-22T22:59:27Z","timestamp":1629673167000},"page":"5609","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["The Adaptive Dynamic Programming Toolbox"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0016-9079","authenticated-orcid":false,"given":"Xiaowei","family":"Xing","sequence":"first","affiliation":[{"name":"School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6496-4189","authenticated-orcid":false,"given":"Dong Eui","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon 34141, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,20]]},"reference":[{"key":"ref_1","unstructured":"Kirk, D.E. (1970). Optimal Control Theory: An Introduction, Prentice-Hall."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Lewis, F.L., Vrabie, D.L., and Syrmos, V.L. (2012). Optimal Control, John Wiley & Sons, Inc.","DOI":"10.1002\/9781118122631"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1254","DOI":"10.1016\/0021-8928(61)90005-3","article-title":"On the optimal stabilization of nonlinear systems","volume":"25","year":"1961","journal-title":"J. Appl. Math. Mech."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/0005-1098(77)90070-X","article-title":"Design of nonlinear automatic flight control systems","volume":"13","author":"Garrard","year":"1977","journal-title":"Automatica"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1016\/0005-1098(71)90008-2","article-title":"A method for suboptimal design of nonlinear feedback systems","volume":"7","author":"Nishikawa","year":"1971","journal-title":"Automatica"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1109\/TSMC.1979.4310171","article-title":"An approximation theory of optimal control for trainable manipulators","volume":"SMC-9","author":"Saridis","year":"1979","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2159","DOI":"10.1016\/S0005-1098(97)00128-3","article-title":"Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation","volume":"33","author":"Beard","year":"1997","journal-title":"Automatica"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1023\/A:1022664528457","article-title":"Approximate solutions to the time-invariant Hamilton-Jacobi-Bellman equation","volume":"96","author":"Beard","year":"1998","journal-title":"J. Optim. Theory Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1016\/j.automatica.2004.11.034","article-title":"Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach","volume":"41","author":"Lewis","year":"2005","journal-title":"Automatica"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2699","DOI":"10.1016\/j.automatica.2012.06.096","article-title":"Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics","volume":"48","author":"Jiang","year":"2012","journal-title":"Automatica"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1016\/j.neunet.2009.03.008","article-title":"Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems","volume":"22","author":"Vrabie","year":"2009","journal-title":"Neural Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1109\/TNNLS.2013.2294968","article-title":"Robust adaptive dynamic programming and feedback stabilization of nonlinear systems","volume":"25","author":"Jiang","year":"2014","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Jiang, Y., and Jiang, Z.-P. (2014). Robust Adaptive Dynamic Programming, John Wiley & Sons, Inc.","DOI":"10.1109\/ASCC.2013.6606031"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1109\/TNNLS.2014.2328590","article-title":"Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations","volume":"26","author":"Lee","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_16","unstructured":"Krener, A.J. Nonlinear Systems Toolbox. MATLAB Toolbox Available upon Request from ajkrener@ucdavis.edu."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Giftthaler, M., Neunert, M., St\u00e4uble, M., and Buchli, J. (2018, January 16\u201319). The Control Toolbox\u2014An open-source C++ library for robotics, optimal and model predictive control. Proceedings of the IEEE 2018 IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), Brisbane, Australia.","DOI":"10.1109\/SIMPAR.2018.8376281"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1002\/oca.939","article-title":"ACADO Toolkit\u2014An open source framework for automatic control and dynamic optimization","volume":"32","author":"Houska","year":"2011","journal-title":"Optim. Control Appl. Meth."},{"key":"ref_19","unstructured":"Verschueren, R., Frison, G., Kouzoupis, D., Frey, J., van Duijkeren, N., Zanelli, A., Novoselnik, B., Albin, T., Quirynen, R., and Diehl, M. (2019). ACADOS: A modular open-source framework for fast embedded optimal control. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2558904","article-title":"GPOPS-II: A MATLAB software for solving multiple-phase optimal control problems using hp-adaptive Gaussian quadrature collocation methods and sparse nonlinear programming","volume":"41","author":"Patterson","year":"2014","journal-title":"ACM Trans. Math. Softw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Cox, D.A., Little, J., and O\u2019Shea, D. (2015). Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, Springer.","DOI":"10.1007\/978-3-319-16721-3"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4981","DOI":"10.1002\/rnc.4294","article-title":"On controller design for systems on manifolds in Euclidean space","volume":"28","author":"Chang","year":"2018","journal-title":"Int. J. Robust Nonlinear Control"},{"key":"ref_23","unstructured":"Ko, W. (2020). A Stable Embedding Technique for Control of Satellite Attitude Represented in Unit Quaternions. [Master\u2019s Thesis, Korea Advanced Institute of Science & Technology]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1007\/s42835-020-00622-3","article-title":"Tracking controller design for satellite attitude under unknown constant disturbance using stable embedding","volume":"16","author":"Ko","year":"2021","journal-title":"J. Electr. Eng. Technol."},{"key":"ref_25","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Gurney, K. (1997). An Introduction to Neural Networks, UCL Press.","DOI":"10.4324\/9780203451519"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Caterini, A.L., and Chang, D.E. (2018). Deep Neural Networks in a Mathematical Framework, Springer.","DOI":"10.1007\/978-3-319-75304-1"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5609\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:47:50Z","timestamp":1760165270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5609"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,20]]},"references-count":27,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21165609"],"URL":"https:\/\/doi.org\/10.3390\/s21165609","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,20]]}}}