{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:10:46Z","timestamp":1760235046163,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T00:00:00Z","timestamp":1626998400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF2010151"],"award-info":[{"award-number":["W911NF2010151"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"name":"U.S. D.O.D. SMART Scholarship","award":["2018-66068"],"award-info":[{"award-number":["2018-66068"]}]},{"DOI":"10.13039\/501100005302","name":"Onassis Foundation","doi-asserted-by":"publisher","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}],"id":[{"id":"10.13039\/501100005302","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Stochastic spatio-temporal processes are prevalent across domains ranging from the modeling of plasma, turbulence in fluids to the wave function of quantum systems. This letter studies a measure-theoretic description of such systems by describing them as evolutionary processes on Hilbert spaces, and in doing so, derives a framework for spatio-temporal manipulation from fundamental thermodynamic principles. This approach yields a variational optimization framework for controlling stochastic fields. The resulting scheme is applicable to a wide class of spatio-temporal processes and can be used for optimizing parameterized control policies. Our simulated experiments explore the application of two forms of this approach on four stochastic spatio-temporal processes, with results that suggest new perspectives and directions for studying stochastic control problems for spatio-temporal systems.<\/jats:p>","DOI":"10.3390\/e23080941","type":"journal-article","created":{"date-parts":[[2021,7,23]],"date-time":"2021-07-23T10:31:44Z","timestamp":1627036304000},"page":"941","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Leveraging Stochasticity for Open Loop and Model Predictive Control of Spatio-Temporal Systems"],"prefix":"10.3390","volume":"23","author":[{"given":"George I.","family":"Boutselis","sequence":"first","affiliation":[{"name":"Department of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30313, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9371-9633","authenticated-orcid":false,"given":"Ethan N.","family":"Evans","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30313, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcus A.","family":"Pereira","sequence":"additional","affiliation":[{"name":"Institute of Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA 30313, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Evangelos A.","family":"Theodorou","sequence":"additional","affiliation":[{"name":"Department of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA 30313, USA"},{"name":"Institute of Robotics and Intelligent Machines, Georgia Institute of Technology, Atlanta, GA 30313, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chow, P. (2007). Stochastic Partial Differential Equations, Taylor & Francis.","DOI":"10.1201\/9781420010305"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Da Prato, G., and Zabczyk, J. (2014). Stochastic Equations in Infinite Dimensions, Cambridge University Press. Encyclopedia of Mathematics and its Applications.","DOI":"10.1017\/CBO9781107295513"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1250","DOI":"10.1137\/S0036141002409167","article-title":"Stochastic Navier\u2013Stokes Equations for Turbulent Flows","volume":"35","author":"Mikulevicius","year":"2004","journal-title":"SIAM J. Math. Anal."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Dumont, G., Payeur, A., and Longtin, A. (2017). A stochastic-field description of finite-size spiking neural networks. PLoS Comput. Biol., 13.","DOI":"10.1371\/journal.pcbi.1005691"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1080\/17442507908833142","article-title":"Stochastic partial differential equations and filtering of diffusion processes","volume":"3","author":"Pardoux","year":"1980","journal-title":"Stochastics"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4627","DOI":"10.1103\/PhysRevE.49.4627","article-title":"Temperature effects in a nonlinear model of monolayer Scheibe aggregates","volume":"49","author":"Bang","year":"1994","journal-title":"Phys. Rev. E"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1142\/S0219024905003049","article-title":"Modeling term structure dynamics: An infinite dimensional approach","volume":"8","author":"Cont","year":"2005","journal-title":"Int. J. Theor. Appl. Financ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"S237","DOI":"10.1088\/1464-4266\/7\/10\/006","article-title":"Hamilton-Jacobi-Bellman equations for quantum optimal feedback control","volume":"7","author":"Gough","year":"2005","journal-title":"J. Opt. B Quantum Semiclassical Opt."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1088\/0953-4075\/38\/3\/002","article-title":"Bellman equations for optimal feedback control of qubit states","volume":"38","author":"Bouten","year":"2005","journal-title":"J. Phys. B At. Mol. Opt. Phys."},{"key":"ref_10","unstructured":"Pontryagin, L., Boltyanskii, V., Gamkrelidze, R., and Mishchenko, E. (1962). The Mathematical Theory of Optimal Processes, Pergamon Press."},{"key":"ref_11","unstructured":"Bellman, R., and Kalaba, R. (1964). Selected Papers On Mathematical Trends in Control Theory, Dover Publications."},{"key":"ref_12","unstructured":"Yong, J., and Zhou, X. (1999). Stochastic Controls: Hamiltonian Systems and HJB Equations, Springer. Stochastic Modelling and Applied Probability."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lou, Y., Hu, G., and Christofides, P.D. (2009, January 10\u201312). Model predictive control of nonlinear stochastic PDEs: Application to a sputtering process. Proceedings of the 2009 American Control Conference, St. Louis, MO, USA.","DOI":"10.1109\/ACC.2009.5160139"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.physd.2017.02.011","article-title":"Controlling roughening processes in the stochastic Kuramoto-Sivashinsky equation","volume":"348","author":"Gomes","year":"2017","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Pardoux, E., and Rascanu, A. (2014). Stochastic Differential Equations, Backward SDEs, Partial Differential Equations, Springer.","DOI":"10.1007\/978-3-319-05714-9"},{"key":"ref_16","unstructured":"Fleming, W.H., and Soner, H.M. (2006). Controlled Markov Processes and Viscosity Solutions, Springer. [2nd ed.]. Applications of Mathematics."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/j.automatica.2017.09.004","article-title":"Stochastic optimal control via forward and backward stochastic differential equations and importance sampling","volume":"87","author":"Exarchos","year":"2018","journal-title":"Automatica"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"344","DOI":"10.2514\/1.G001921","article-title":"Model Predictive Path Integral Control: From Theory to Parallel Computation","volume":"40","author":"Williams","year":"2017","journal-title":"J. Guid. Control. Dyn."},{"key":"ref_19","unstructured":"Evans, E.N., Pereira, M.A., Boutselis, G.I., and Theodorou, E.A. (November, January 30). Variational Optimization Based Reinforcement Learning for Infinite Dimensional Stochastic Systems. Proceedings of the Conference on Robot Learning, Osaka, Japan."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Evans, E.N., Kendall, A.P., Boutselis, G.I., and Theodorou, E.A. (2020, January 12\u201316). Spatio-Temporal Stochastic Optimization: Theory and Applications to Optimal Control and Co-Design. Proceedings of the 2020 Robotics: Sciences and Systems (RSS) Conference.","DOI":"10.15607\/RSS.2020.XVI.049"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Evans, E.N., Kendall, A.P., and Theodorou, E.A. (2021). Stochastic Spatio-Temporal Optimization for Control and Co-Design of Systems in Robotics and Applied Physics. arXiv.","DOI":"10.1007\/s10514-021-10003-y"},{"key":"ref_22","unstructured":"Bieker, K., Peitz, S., Brunton, S.L., Kutz, J.N., and Dellnitz, M. (2019). Deep Model Predictive Control with Online Learning for Complex Physical Systems. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1017\/jfm.2019.469","article-title":"Cluster-based feedback control of turbulent post-stall separated flows","volume":"875","author":"Nair","year":"2019","journal-title":"J. Fluid Mech."},{"key":"ref_24","unstructured":"Mohan, A.T., and Gaitonde, D.V. (2018). A deep learning based approach to reduced order modeling for turbulent flow control using LSTM neural networks. arXiv."},{"key":"ref_25","unstructured":"Morton, J., Jameson, A., Kochenderfer, M.J., and Witherden, F. (2018). Deep dynamical modeling and control of unsteady fluid flows. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1017\/jfm.2019.62","article-title":"Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control","volume":"865","author":"Rabault","year":"2019","journal-title":"J. Fluid Mech."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/0167-6911(86)90100-3","article-title":"Robust stabilization of infinite dimensional systems by finite dimensional controllers","volume":"7","author":"Curtain","year":"1986","journal-title":"Syst. Control Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"673","DOI":"10.1109\/TAC.1978.1101798","article-title":"Feedback control of flexible systems","volume":"23","author":"Balas","year":"1978","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Fabbri, G., Gozzi, F., and Swiech, A. (2017). Stochastic Optimal Control in Infinite Dimensions\u2014Dynamic Programming and HJB Equations, Springer. Number 82 in Probability Theory and Stochastic Modelling.","DOI":"10.1007\/978-3-319-53067-3"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1137\/S0363012996311307","article-title":"Control of the Stochastic Burgers Model of Turbulence","volume":"37","author":"Debussche","year":"1999","journal-title":"SIAM J. Control Optim."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1214\/009117905000000567","article-title":"Large deviation for diffusions and Hamilton-Jacobi equation in Hilbert spaces","volume":"34","author":"Feng","year":"2006","journal-title":"Ann. Probab."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Theodorou, E.A., Boutselis, G.I., and Bakshi, K. (2018, January 17\u201319). Linearly Solvable Stochastic Optimal Control for Infinite-Dimensional Systems. Proceedings of the 2018 IEEE Conference on Decision and Control (CDC), Miami, FL, USA.","DOI":"10.1109\/CDC.2018.8618948"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"11478","DOI":"10.1073\/pnas.0710743106","article-title":"Efficient computation of optimal actions","volume":"106","author":"Todorov","year":"2009","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Theodorou, E., and Todorov, E. (2012, January 10\u201313). Relative entropy and free energy dualities: Connections to Path Integral and KL control. Proceedings of the IEEE Conference on Decision and Control, Maui, HI, USA.","DOI":"10.1109\/CDC.2012.6426381"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Theodorou, E.A. (2015). Nonlinear Stochastic Control and Information Theoretic Dualities: Connections, Interdependencies and Thermodynamic Interpretations. Entropy, 17.","DOI":"10.3390\/e17053352"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"P11011","DOI":"10.1088\/1742-5468\/2005\/11\/P11011","article-title":"Path integrals and symmetry breaking for optimal control theory","volume":"11","author":"Kappen","year":"2005","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_37","first-page":"55","article-title":"Stability of semilinear equations with boundary and pointwise noise","volume":"22","author":"Maslowski","year":"1995","journal-title":"Ann. Della Sc. Norm. Super. Pisa Cl. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1051\/cocv:2007001","article-title":"Optimal control of a stochastic heat equation with boundary-noise and boundary-control","volume":"13","author":"Debussche","year":"2007","journal-title":"ESAIM Control. Optim. Calc. Var."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1007\/s10955-016-1446-7","article-title":"Adaptive Importance Sampling for Control and Inference","volume":"162","author":"Kappen","year":"2016","journal-title":"J. Stat. Phys."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/BF01194987","article-title":"Stochastic Burgers\u2019 equation","volume":"1","author":"Debussche","year":"1994","journal-title":"Nonlinear Differ. Equ. Appl. NoDEA"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2006","DOI":"10.1063\/1.1692305","article-title":"Forced model equation for turbulence","volume":"12","author":"Jeng","year":"1969","journal-title":"Phys. Fluids"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lord, G.J., Powell, C.E., and Shardlow, T. (2014). An Introduction to Computational Stochastic PDEs, Cambridge Texts in Applied Mathematics, Cambridge University Press.","DOI":"10.1017\/CBO9781139017329"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1020","DOI":"10.1137\/S0363012996303190","article-title":"Ergodic boundary\/point control of stochastic semilinear systems","volume":"36","author":"Duncan","year":"1998","journal-title":"SIAM J. Control Optim."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/941\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:33:53Z","timestamp":1760164433000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/941"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,23]]},"references-count":43,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["e23080941"],"URL":"https:\/\/doi.org\/10.3390\/e23080941","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,7,23]]}}}