{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T14:53:34Z","timestamp":1784213614134,"version":"3.55.0"},"reference-count":21,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2015,5,15]],"date-time":"2015-05-15T00:00:00Z","timestamp":1431648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we present connections between recent developments on the linearly-solvable stochastic optimal control framework with early work in control theory based on the fundamental dualities between free energy and relative entropy. We extend these connections to nonlinear stochastic systems with non-affine controls by using the generalized version of the Feynman\u2013Kac lemma. We present alternative formulations of the linearly-solvable stochastic optimal control framework and discuss information theoretic and thermodynamic interpretations. On the algorithmic side, we present iterative stochastic optimal control algorithms and applications to nonlinear stochastic systems. We conclude with an overview of the frameworks presented and discuss limitations, differences and future directions.<\/jats:p>","DOI":"10.3390\/e17053352","type":"journal-article","created":{"date-parts":[[2015,5,15]],"date-time":"2015-05-15T07:57:02Z","timestamp":1431676622000},"page":"3352-3375","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations"],"prefix":"10.3390","volume":"17","author":[{"given":"Evangelos","family":"Theodorou","sequence":"first","affiliation":[{"name":"The Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology, Atlanta,GA 30332-0150, USA"},{"name":"Institute of Robotics and intelligent Machines, Georgia Institute of Technology, Atlanta, GA30332-0150, USA\u00a0"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2015,5,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1063\/1.2709596","article-title":"An introduction to stochastic control theory, path integrals and reinforcement learning","volume":"887","author":"Marro","year":"2007","journal-title":"Cooperative Behavior in Neural Systems"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"P11011","DOI":"10.1088\/1742-5468\/2005\/11\/P11011","article-title":"Path integrals and symmetry breaking for optimal control theory","volume":"11","author":"Kappen","year":"2005","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_3","unstructured":"Theodorou, E., Buchli, J., and Schaal, S. (2010). A Generalized Path Integral Approach to Reinforcement Learning. J. Mach. Learn. Res., 3137\u20133181."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"11478","DOI":"10.1073\/pnas.0710743106","article-title":"Efficient computation of optimal actions","volume":"106","author":"Todorov","year":"2009","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Scholkopf, B., Platt, J., and Hoffman, T. (2007). Advances in Neural Information Processing Systems 19, MIT Press.","DOI":"10.7551\/mitpress\/7503.001.0001"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pan, Y., and Theodorou, E. (2014, January 9\u201312). Nonparametric infinite horizon Kullback-Leibler stochastic control, Orlando, USA.","DOI":"10.1109\/ADPRL.2014.7010616"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Friedman, A. (1975). Stochastic Differential Equations And Applications, Academic Press.","DOI":"10.1016\/B978-0-12-268201-8.50010-4"},{"key":"ref_8","unstructured":"Karatzas, I., and Shreve, S.E. (1991). Brownian Motion and Stochastic Calculus (Graduate Texts in Mathematics), Springer. [2]."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"\u00d8ksendal, B.K. (2003). Stochastic differential equations: An Introduction with Applications, Springer. [6].","DOI":"10.1007\/978-3-642-14394-6"},{"key":"ref_10","unstructured":"Horowitz, M.B., Damle, A., and Burdick, J.W. Linear Hamilton Jacobi Bellman Equations in High Dimensions, arXiv:1404.1089."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Theodorou, E., and Todorov, E. (2012, January 10\u201313). Relative entropy and free energy dualities: Connections to Path Integral and KL control, Maui, HI, USA.","DOI":"10.1109\/CDC.2012.6426381"},{"key":"ref_12","first-page":"329","article-title":"Exit probabilities and optimal stochastic control","volume":"9","author":"Fleming","year":"1971","journal-title":"Appl. Math. Optim."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/BF01211853","article-title":"Connections between stochastic control and dynamic games","volume":"9","author":"Meneghini","year":"1996","journal-title":"Math. Control Signals Syst. (MCSS)"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1813","DOI":"10.1137\/S0363012901393894","article-title":"A Variational Approach to Nonlinear Estimation","volume":"42","author":"Mitter","year":"2003","journal-title":"SIAM J. Control Optim."},{"key":"ref_15","unstructured":"Fleming, W.H., and Soner, H.M. (2006). Controlled Markov Processes and Viscosity Solutions, Springer. [2]."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/0034-4877(91)90045-O","article-title":"The many facets of entropy","volume":"30","author":"Wehrl","year":"1991","journal-title":"Rep. Math. Phys."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1137\/0329064","article-title":"A monte carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems","volume":"29","author":"Yang","year":"1991","journal-title":"SIAM J. Control Optim."},{"key":"ref_18","unstructured":"Fleming, W.H., and Soner, H.M. (1993). Controlled Markov Processes and Viscosity Solutions, Springer. [1]."},{"key":"ref_19","unstructured":"Stengel, R.F. (1994). Optimal Control and Estimation, Dover Publications."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Leadbetter, R., Cambanis, S., and Pipiras, P. (2014). Basic Course in Measure and Probabilty, Cambridge University Press.","DOI":"10.1017\/CBO9781139103947"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"200201","DOI":"10.1103\/PhysRevLett.95.200201","article-title":"Linear theory for control of nonlinear stochastic systems","volume":"95","author":"Kappen","year":"2005","journal-title":"Phys. Rev. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3352\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:46:21Z","timestamp":1760215581000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3352"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,15]]},"references-count":21,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,5]]}},"alternative-id":["e17053352"],"URL":"https:\/\/doi.org\/10.3390\/e17053352","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,5,15]]}}}