{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T18:48:15Z","timestamp":1775501295599,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2021,4,24]],"date-time":"2021-04-24T00:00:00Z","timestamp":1619222400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,24]],"date-time":"2021-04-24T00:00:00Z","timestamp":1619222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"ERC advanced Grant","award":["668998 (OCLOC)"],"award-info":[{"award-number":["668998 (OCLOC)"]}]},{"DOI":"10.13039\/501100009057","name":"University of Graz","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009057","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Optim Appl"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Policy iteration is a widely used technique to solve the Hamilton Jacobi Bellman (HJB) equation, which arises from nonlinear optimal feedback control theory. Its convergence analysis has attracted much attention in the unconstrained case. Here we analyze the case with control constraints both for the HJB equations which arise in deterministic and in stochastic control cases. The linear equations in each iteration step are solved by an implicit upwind scheme. Numerical examples are conducted to solve the HJB equation with control constraints and comparisons are shown with the unconstrained cases.<\/jats:p>","DOI":"10.1007\/s10589-021-00278-3","type":"journal-article","created":{"date-parts":[[2021,4,25]],"date-time":"2021-04-25T07:21:56Z","timestamp":1619335316000},"page":"785-809","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Policy iteration for Hamilton\u2013Jacobi\u2013Bellman equations with control constraints"],"prefix":"10.1007","volume":"87","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3764-1245","authenticated-orcid":false,"given":"Sudeep","family":"Kundu","sequence":"first","affiliation":[]},{"given":"Karl","family":"Kunisch","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,4,24]]},"reference":[{"issue":"5","key":"278_CR1","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1016\/j.automatica.2004.11.034","volume":"41","author":"M Abu-Khalaf","year":"2005","unstructured":"Abu-Khalaf, M., Lewis, F.L.: Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Autom. J. IFAC 41(5), 779\u2013791 (2005)","journal-title":"Autom. J. IFAC"},{"key":"278_CR2","doi-asserted-by":"crossref","unstructured":"Achdou, Y., Han, J., Lasry, J.-M., Lions, P.-L., Moll, B.: Income and wealth distribution in macroeconomics: a continuous-time approach, National Bureau of Economic Research (2017)","DOI":"10.3386\/w23732"},{"key":"278_CR3","volume-title":"Sobolev Spaces","author":"RA Adams","year":"2003","unstructured":"Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Elsevier\/Academic Press, Amsterdam (2003)"},{"issue":"1","key":"278_CR4","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1137\/130932284","volume":"37","author":"A Alla","year":"2015","unstructured":"Alla, A., Falcone, M., Kalise, D.: An efficient policy iteration algorithm for dynamic programming equations. SIAM J. Sci. Comput. 37(1), 181\u2013200 (2015)","journal-title":"SIAM J. Sci. Comput."},{"key":"278_CR5","doi-asserted-by":"publisher","first-page":"2384","DOI":"10.1137\/18M1203900","volume":"41","author":"A Alla","year":"2019","unstructured":"Alla, A., Falcone, M., Saluzzi, L.: An efficient DP algorithm on a tree-structure for finite horizon optimal control problems. SIAM J. Sci. Comput. 41, 2384\u20132406 (2019)","journal-title":"SIAM J. Sci. Comput."},{"key":"278_CR6","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-8176-4755-1","volume-title":"Optimal Control and Viscosity Solutions of Hamilton\u2013Jacobi\u2013Bellman Equations","author":"M Bardi","year":"1997","unstructured":"Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton\u2013Jacobi\u2013Bellman Equations. Birkh\u00e4user, Boston (1997)"},{"issue":"12","key":"278_CR7","doi-asserted-by":"publisher","first-page":"2159","DOI":"10.1016\/S0005-1098(97)00128-3","volume":"33","author":"RW Beard","year":"1997","unstructured":"Beard, R.W., Saridis, G.N., Wen, J.T.: Galerkin approximation of the generalized Hamilton\u2013Jacobi\u2013Bellman equation. Automatica 33(12), 2159\u20132177 (1997)","journal-title":"Automatica"},{"issue":"3","key":"278_CR8","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1023\/A:1022664528457","volume":"96","author":"RW Beard","year":"1998","unstructured":"Beard, R.W., Saridis, G.N., Wen, J.T.: Approximate solutions to the time-invariant Hamilton\u2013Jacobi\u2013Bellman equation. J. Optim. Theory Appl. 96(3), 589\u2013626 (1998)","journal-title":"J. Optim. Theory Appl."},{"issue":"5","key":"278_CR9","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1080\/002071798221542","volume":"71","author":"RW Beard","year":"1998","unstructured":"Beard, R.W., Mclain, T.W.: Successive Galerkin approximation algorithms for nonlinear optimal and robust control. Int. J. Control 71(5), 717\u2013743 (1998)","journal-title":"Int. J. Control"},{"issue":"4","key":"278_CR10","doi-asserted-by":"publisher","first-page":"3001","DOI":"10.1137\/08073041X","volume":"47","author":"O Bokanowski","year":"2009","unstructured":"Bokanowski, O., Maroso, S., Zidani, H.: Some convergence results for Howard\u2019s algorithm. SIAM J. Numer. Anal. 47(4), 3001\u20133026 (2009)","journal-title":"SIAM J. Numer. Anal."},{"issue":"1","key":"278_CR11","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1137\/140998482","volume":"38","author":"O Bokanowski","year":"2016","unstructured":"Bokanowski, O., Falcone, M., Sahu, S.: An efficient filtered scheme for some first order time-dependent Hamilton\u2013Jacobi equations. SIAM J. Sci. Comput. 38(1), 171\u2013195 (2016)","journal-title":"SIAM J. Sci. Comput."},{"issue":"3","key":"278_CR12","doi-asserted-by":"publisher","first-page":"1008","DOI":"10.1137\/S0036142901387336","volume":"41","author":"JF Bonnans","year":"2003","unstructured":"Bonnans, J.F., Zidani, H.: Consistency of generalized finite difference schemes for the stochastic HJB equation. SIAM J. Numer. Anal. 41(3), 1008\u20131021 (2003)","journal-title":"SIAM J. Numer. Anal."},{"key":"278_CR13","doi-asserted-by":"crossref","unstructured":"Breiten, T., Kunisch, K., Pfeiffer, L.: Taylor expansions of the value function associated with a bilinear optimal control problem. In Press, Ann. I. H. Poincare-AN (2019)","DOI":"10.1016\/j.anihpc.2019.01.001"},{"issue":"5","key":"278_CR14","doi-asserted-by":"publisher","first-page":"2625","DOI":"10.1137\/110841576","volume":"34","author":"S Cacace","year":"2012","unstructured":"Cacace, S., Cristiani, E., Falcone, M., Picarelli, A.: A patchy dynamic programming scheme for a class of Hamilton\u2013Jacobi\u2013Bellman equations. SIAM J. Sci. Comput. 34(5), 2625\u20132649 (2012)","journal-title":"SIAM J. Sci. Comput."},{"issue":"9","key":"278_CR15","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1016\/j.sysconle.2013.06.001","volume":"62","author":"P Cannarsa","year":"2013","unstructured":"Cannarsa, P., Frankowska, H.: Local regularity of the value function in optimal control. Syst. Control Lett. 62(9), 791\u2013794 (2013)","journal-title":"Syst. Control Lett."},{"issue":"2","key":"278_CR16","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1007\/BF01442176","volume":"11","author":"I Capuzzo-Dolcetta","year":"1984","unstructured":"Capuzzo-Dolcetta, I., Ishii, H.: Approximate solutions of the Bellman equation of deterministic control theory. Appl. Math. Optim. 11(2), 161\u2013181 (1984)","journal-title":"Appl. Math. Optim."},{"issue":"167","key":"278_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1090\/S0025-5718-1984-0744921-8","volume":"43","author":"MG Crandall","year":"1984","unstructured":"Crandall, M.G., Lions, P.L.: Two approximations of solutions of Hamilton\u2013Jacobi equations. Math. Comput. 43(167), 1\u201319 (1984)","journal-title":"Math. Comput."},{"key":"278_CR18","doi-asserted-by":"crossref","unstructured":"Dolgov, S., Kalise, D., Kunisch, K.: Tensor decompositions for high-dimensional Hamilton\u2013Jacobi\u2013Bellman equations. arXiv:1908.01533 (2019)","DOI":"10.1515\/9783110543599"},{"key":"278_CR19","volume-title":"Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations","author":"M Falcone","year":"2014","unstructured":"Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2014)"},{"key":"278_CR20","doi-asserted-by":"crossref","unstructured":"Falcone, M., Ferretti, R.: Numerical methods for Hamilton-Jacobi type equations. Handb. Numer. Anal., Elsevier\/North-Holland, Amsterdam,17, 603\u2013626 (2016)","DOI":"10.1016\/bs.hna.2016.09.018"},{"key":"278_CR21","volume-title":"Deterministic and Stochastic Optimal Control","author":"WH Fleming","year":"2012","unstructured":"Fleming, W.H., Rishel, R.W.: Deterministic and Stochastic Optimal Control. Springer, Berlin (2012)"},{"key":"278_CR22","volume-title":"Controlled Markov Processes and Viscosity Solutions","author":"WH Fleming","year":"2006","unstructured":"Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer, Philadelphia (2006)"},{"key":"278_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10915-016-0240-7","volume":"43","author":"J Garcke","year":"2017","unstructured":"Garcke, J., Kr\u00f6ner, A.: Suboptimal feedback control of PDEs by solving HJB equations on adaptive sparse grids. J. Sci. Comput. 43, 1\u201328 (2017)","journal-title":"J. Sci. Comput."},{"issue":"5","key":"278_CR24","doi-asserted-by":"publisher","first-page":"1787","DOI":"10.1137\/S0363012902411581","volume":"43","author":"R Goebel","year":"2005","unstructured":"Goebel, R.: Convex optimal control problems with smooth Hamiltonians. SIAM J. Control Optim. 43(5), 1787\u20131811 (2005)","journal-title":"SIAM J. Control Optim."},{"issue":"5","key":"278_CR25","doi-asserted-by":"publisher","first-page":"886","DOI":"10.1109\/TAC.2007.895915","volume":"52","author":"R Goebel","year":"2007","unstructured":"Goebel, R., Subbotin, M.: Continuous time linear quadratic regulator with control constraints via convex duality. IEEE Trans. Autom. Control 52(5), 886\u2013892 (2007)","journal-title":"IEEE Trans. Autom. Control"},{"issue":"2","key":"278_CR26","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1137\/0323018","volume":"23","author":"R Gonzalez","year":"1985","unstructured":"Gonzalez, R., Rofman, E.: On deterministic control problems: an approximation procedure for the optimal cost. I. The stationary problem. SIAM J. Control Optim. 23(2), 242\u2013266 (1985)","journal-title":"SIAM J. Control Optim."},{"key":"278_CR27","volume-title":"Dynamic Programming and Markov Processes","author":"R Howard","year":"1960","unstructured":"Howard, R.: Dynamic Programming and Markov Processes. MIT Press, Cambridge (1960)"},{"issue":"2","key":"278_CR28","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1137\/17M1116635","volume":"40","author":"D Kalise","year":"2018","unstructured":"Kalise, D., Kunisch, K.: Polynomial approximation of high-dimensional Hamilton\u2013Jacobi\u2013Bellman equations and applications to feedback control of semilinear parabolic PDEs. SIAM J. Sci. Comput. 40(2), 629\u2013652 (2018)","journal-title":"SIAM J. Sci. Comput."},{"key":"278_CR29","unstructured":"Kravaris, C., Mousavere, D.: Convergence properties of the Newton\u2013Kantorovich iteration for the Hamilton\u2013Jacobi\u2013Bellman equation. In: Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005 Seville, Spain, December 12\u201315 (2005)"},{"issue":"2","key":"278_CR30","first-page":"442","volume":"21","author":"A Kr\u00f6ner","year":"2014","unstructured":"Kr\u00f6ner, A., Kunisch, K., Zidani, H.: Optimal feedback control of undamped wave equations by solving a HJB equation, ESAIM: COCV. EDP Sci. 21(2), 442\u2013464 (2014)","journal-title":"EDP Sci."},{"key":"278_CR31","doi-asserted-by":"crossref","unstructured":"Kushner, H.J., Dupuis, P.: Numerical Methods for Stochastic Control Problems in Continuous Time. Springer, New York, xii+475 (2001)","DOI":"10.1007\/978-1-4613-0007-6"},{"key":"278_CR32","unstructured":"Kunisch, K., Walter, D.: Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation. arXiv:2002.08625v1"},{"issue":"1","key":"278_CR33","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1137\/0305004","volume":"5","author":"RJ Leake","year":"1967","unstructured":"Leake, R.J., Liu, R.-W.: Construction of suboptimal control sequences. SIAM J. Control 5(1), 54\u201363 (1967)","journal-title":"SIAM J. Control"},{"issue":"1","key":"278_CR34","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1137\/0307007","volume":"7","author":"DL Lukes","year":"1969","unstructured":"Lukes, D.L.: Optimal regulation of nonlinear dynamical systems. SIAM J. Control 7(1), 75\u2013100 (1969)","journal-title":"SIAM J. Control"},{"key":"278_CR35","doi-asserted-by":"crossref","unstructured":"Lyshevski, S.E.: Optimal control of nonlinear continuous-time systems: design of bounded controllers via generalized nonquadratic functionals. In: Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No. 98CH36207) 1, 205\u2013209 (1998)","DOI":"10.1109\/ACC.1998.694659"},{"key":"278_CR36","volume-title":"Stochastic Differential Equations and Applications","author":"X Mao","year":"2007","unstructured":"Mao, X.: Stochastic Differential Equations and Applications. Elsevier, Amsterdam (2007)"},{"key":"278_CR37","doi-asserted-by":"crossref","unstructured":"Osher, S., Fedkiw, R.: Level set methods and dynamic implicit surfaces. Springer-Verlag, New York, Applied Mathematical Sciences (2003)","DOI":"10.1007\/b98879"},{"issue":"3","key":"278_CR38","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1109\/TSMC.1979.4310171","volume":"9","author":"GN Saridis","year":"1979","unstructured":"Saridis, G.N., Lee, C.S.G.: An approximation theory of optimal control for trainable manipulators. IEEE Trans. Syst. Man Cybern. 9(3), 152\u2013159 (1979)","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"278_CR39","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5767-7","volume-title":"The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors","author":"C Sparrow","year":"1982","unstructured":"Sparrow, C.: The Lorenz Equations: Bifurcations, Chaos, and Strange Attractors. Springer, New York (1982)"},{"key":"278_CR40","doi-asserted-by":"publisher","unstructured":"Stefansson, E., Leong, Y.P.: Sequential alternating least squares for solving high dimensional linear Hamilton\u2013Jacobi\u2013Bellman Equations. In: Proceedings of IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). https:\/\/doi.org\/10.1109\/IROS.2016.7759553 (2016)","DOI":"10.1109\/IROS.2016.7759553"},{"key":"278_CR41","doi-asserted-by":"crossref","unstructured":"Wang, F.Y., Saridis, G.N.: On successive approximation of optimal control of stochastic dynamic systems, Modeling uncertainty, Internat. Ser. Oper. Res. Management Sci., Kluwer Acad. Publ., Boston, MA 46, 333\u2013358 (2002)","DOI":"10.1007\/0-306-48102-2_16"}],"container-title":["Computational Optimization and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-021-00278-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10589-021-00278-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-021-00278-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,13]],"date-time":"2024-04-13T12:05:28Z","timestamp":1713009928000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10589-021-00278-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,24]]},"references-count":41,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["278"],"URL":"https:\/\/doi.org\/10.1007\/s10589-021-00278-3","relation":{},"ISSN":["0926-6003","1573-2894"],"issn-type":[{"value":"0926-6003","type":"print"},{"value":"1573-2894","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,24]]},"assertion":[{"value":"10 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}