{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:34:18Z","timestamp":1787322858836,"version":"build-2736575974"},"reference-count":30,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"name":"Republic of Cyprus","award":["POST-DOC\/0916\/0139"],"award-info":[{"award-number":["POST-DOC\/0916\/0139"]}]},{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","award":["317726"],"award-info":[{"award-number":["317726"]}],"id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["POST-DOC\/0916\/0139"],"award-info":[{"award-number":["POST-DOC\/0916\/0139"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Control Optim."],"published-print":{"date-parts":[[2019,1]]},"abstract":"<jats:p>We analyze the per unit-time infinite horizon average cost Markov control model, subject to a total variation distance ambiguity on the controlled process conditional distribution. This stochastic optimal control problem is formulated as a minimax optimization problem in which the minimization is over the admissible set of control strategies, while the maximization is over the set of conditional distributions which are in a ball, with respect to the total variation distance, centered at a nominal distribution. We derive two new equivalent dynamic programming equations, and a new policy iteration algorithm. The main feature of the new dynamic programming equations is that the optimal control strategies are insensitive to inaccuracies or ambiguities in the controlled process conditional distribution. The main feature of the new policy iteration algorithm is that the policy evaluation and policy improvement steps are performed using the maximizing conditional distribution, which is obtained via a water filling solution of aggregating states together to form new states. Throughout the paper, we illustrate the new dynamic programming equations and the corresponding policy iteration algorithm to various examples.<\/jats:p>","DOI":"10.1137\/18m1210514","type":"journal-article","created":{"date-parts":[[2019,8,21]],"date-time":"2019-08-21T11:29:17Z","timestamp":1566386957000},"page":"2843-2872","source":"Crossref","is-referenced-by-count":11,"title":["Infinite Horizon Average Cost Dynamic Programming Subject to Total Variation Distance Ambiguity"],"prefix":"10.1137","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7301-6252","authenticated-orcid":true,"given":"Ioannis","family":"Tzortzis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charalambos D.","family":"Charalambous","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Themistoklis","family":"Charalambous","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2019,8,21]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1137\/0331018"},{"key":"atypb2","doi-asserted-by":"crossref","unstructured":"J. Baras and M. Rabi,\n                      Maximum entropy models, dynamic games, and robust output feedback control for automata\n                      , in Proceedings of the 44th IEEE Conference on Decision and Control, 2005, pp. 1043-1049.","DOI":"10.1109\/CDC.2005.1582295"},{"key":"atypb3","unstructured":"T. Basar and P. Bernhard,\n                      H-$\\infty$ Optimal Control and Related Minimax Design Problems: A Dynamic Game Approach\n                      , Collection Syst\u00e8mes complexes, Birkh\u00e4user, Basel, 1995."},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1137\/S0363012993255879"},{"key":"atypb5","unstructured":"D. Bertsekas,\n                      Dynamic Programming and Stochastic Control\n                      , Academic Press, New York, 1976."},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1137\/0322062"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1137\/0327034"},{"key":"atypb8","unstructured":"P. E. Caines,\n                      Linear Stochastic Systems\n                      , John Wiley & Sons, New York, 1988."},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1080\/17442509608834063"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2007.894517"},{"key":"atypb11","first-page":"1909","author":"Charalambous C. D.","year":"2012","journal-title":"IEEE"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2014.2321951"},{"key":"atypb13","unstructured":"T. Cover and J. Thomas,\n                      Elements of Information Theory\n                      , John Wiley & Sons, New York, 1991."},{"key":"atypb14","unstructured":"N. Dunford and J. Schwartz,\n                      Linear Operators: General Theory\n                      , Interscience Publishers, New York, 1958."},{"key":"atypb15","doi-asserted-by":"crossref","unstructured":"P. Dupuis and R. Ellis,\n                      A Weak Convergence Approach to the Theory of Large Deviations\n                      , John Wiley & Sons, New York, 1997.","DOI":"10.1002\/9781118165904"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1111\/j.1751-5823.2002.tb00178.x"},{"key":"atypb17","doi-asserted-by":"crossref","unstructured":"O. Hernandez-Lerma and J. B. Lasserre,\n                      Discrete-time Markov Control Processes: Basic Optimality Criteria\n                      , Appl. Math. (N.Y.) 30, Springer-Verlag, New York, 1996.","DOI":"10.1007\/978-1-4612-0729-0"},{"key":"atypb18","doi-asserted-by":"publisher","DOI":"10.1109\/9.286253"},{"key":"atypb19","unstructured":"P. R. Kumar and P. Varaiya,\n                      Stochastic Systems: Estimation, Identification, and Adaptive Control\n                      , Prentice Hall, Upper Saddle River, NJ, 1986."},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2016.0786"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1109\/9.847720"},{"key":"atypb22","doi-asserted-by":"crossref","unstructured":"M. L. Puterman,\n                      Markov Decision Processes\n                      , John Wiley & Sons, New York, 1994.","DOI":"10.1002\/9780470316887"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1016\/0167-6911(93)E0158-D"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1137\/140955707"},{"key":"atypb25","first-page":"1515","author":"Tzortzis I.","year":"2016","journal-title":"IEEE"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1007\/PL00009843"},{"key":"atypb27","unstructured":"J. H. Van Schuppen,\n                      Mathematical Control and System Theory of Discrete-Time Stochastic Systems\n                      , preprint, 2017."},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1287\/moor.1120.0540"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1109\/LCSYS.2017.2711553"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2015.2495174"}],"container-title":["SIAM Journal on Control and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/18M1210514","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:29:44Z","timestamp":1787318984000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/18M1210514"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1]]},"references-count":30,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,1]]}},"alternative-id":["10.1137\/18M1210514"],"URL":"https:\/\/doi.org\/10.1137\/18m1210514","relation":{},"ISSN":["0363-0129","1095-7138"],"issn-type":[{"value":"0363-0129","type":"print"},{"value":"1095-7138","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1]]}}}