{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T06:17:07Z","timestamp":1785305827912,"version":"3.55.0"},"reference-count":58,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,21]],"date-time":"2023-01-21T00:00:00Z","timestamp":1674259200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"JSPS Research Fellowship","doi-asserted-by":"publisher","award":["21J20436"],"award-info":[{"award-number":["21J20436"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"JSPS Research Fellowship","doi-asserted-by":"publisher","award":["19H05799"],"award-info":[{"award-number":["19H05799"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"JSPS Research Fellowship","doi-asserted-by":"publisher","award":["JPMJCR2011"],"award-info":[{"award-number":["JPMJCR2011"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["21J20436"],"award-info":[{"award-number":["21J20436"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["19H05799"],"award-info":[{"award-number":["19H05799"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["JPMJCR2011"],"award-info":[{"award-number":["JPMJCR2011"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"JST CREST","doi-asserted-by":"publisher","award":["21J20436"],"award-info":[{"award-number":["21J20436"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"JST CREST","doi-asserted-by":"publisher","award":["19H05799"],"award-info":[{"award-number":["19H05799"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"JST CREST","doi-asserted-by":"publisher","award":["JPMJCR2011"],"award-info":[{"award-number":["JPMJCR2011"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Memory-limited partially observable stochastic control (ML-POSC) is the stochastic optimal control problem under incomplete information and memory limitation. To obtain the optimal control function of ML-POSC, a system of the forward Fokker\u2013Planck (FP) equation and the backward Hamilton\u2013Jacobi\u2013Bellman (HJB) equation needs to be solved. In this work, we first show that the system of HJB-FP equations can be interpreted via Pontryagin\u2019s minimum principle on the probability density function space. Based on this interpretation, we then propose the forward-backward sweep method (FBSM) for ML-POSC. FBSM is one of the most basic algorithms for Pontryagin\u2019s minimum principle, which alternately computes the forward FP equation and the backward HJB equation in ML-POSC. Although the convergence of FBSM is generally not guaranteed in deterministic control and mean-field stochastic control, it is guaranteed in ML-POSC because the coupling of the HJB-FP equations is limited to the optimal control function in ML-POSC.<\/jats:p>","DOI":"10.3390\/e25020208","type":"journal-article","created":{"date-parts":[[2023,1,23]],"date-time":"2023-01-23T02:27:45Z","timestamp":1674440865000},"page":"208","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5141-261X","authenticated-orcid":false,"given":"Takehiro","family":"Tottori","sequence":"first","affiliation":[{"name":"Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-8654, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8474-7942","authenticated-orcid":false,"given":"Tetsuya J.","family":"Kobayashi","sequence":"additional","affiliation":[{"name":"Department of Mathematical Informatics, Graduate School of Information Science and Technology, The University of Tokyo, Tokyo 113-8654, Japan"},{"name":"Institute of Industrial Science, The University of Tokyo, Tokyo 153-8505, Japan"},{"name":"Department of Electrical Engineering and Information Systems, Graduate School of Engineering, The University of Tokyo, Tokyo 113-8654, Japan"},{"name":"Universal Biology Institute, The University of Tokyo, Tokyo 113-8654, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fox, R., and Tishby, N. (2016, January 12\u201314). Minimum-information LQG control Part II: Retentive controllers. Proceedings of the 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, NV, USA.","DOI":"10.1109\/CDC.2016.7799130"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fox, R., and Tishby, N. (2016, January 12\u201314). Minimum-information LQG control part I: Memoryless controllers. Proceedings of the 2016 IEEE 55th Conference on Decision and Control (CDC), Las Vegas, NV, USA.","DOI":"10.1109\/CDC.2016.7799131"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, W., and Todorov, E. (2006, January 13\u201315). An Iterative Optimal Control and Estimation Design for Nonlinear Stochastic System. Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, CA, USA.","DOI":"10.1109\/CDC.2006.377485"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1439","DOI":"10.1080\/00207170701364913","article-title":"Iterative linearization methods for approximately optimal control and estimation of non-linear stochastic system","volume":"80","author":"Li","year":"2007","journal-title":"Int. J. Control."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"128102","DOI":"10.1103\/PhysRevLett.126.128102","article-title":"Connection between the Bacterial Chemotactic Network and Optimal Filtering","volume":"126","author":"Nakamura","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"013120","DOI":"10.1103\/PhysRevResearch.4.013120","article-title":"Optimal sensing and control of run-and-tumble chemotaxis","volume":"4","author":"Nakamura","year":"2022","journal-title":"Phys. Rev. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"042401","DOI":"10.1103\/PhysRevE.98.042401","article-title":"Chemotaxis emerges as the optimal solution to cooperative search games","volume":"98","author":"Pezzotta","year":"2018","journal-title":"Phys. Rev. E"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"083401","DOI":"10.1088\/1742-5468\/ac12c6","article-title":"Optimal collision avoidance in swarms of active Brownian particles","volume":"2021","author":"Borra","year":"2021","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1137\/0311020","article-title":"Dynamic Programming Conditions for Partially Observable Stochastic Systems","volume":"11","author":"Davis","year":"1973","journal-title":"SIAM J. Control."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bensoussan, A. (1992). Stochastic Control of Partially Observable Systems, Cambridge University Press.","DOI":"10.1017\/CBO9780511526503"},{"key":"ref_11","unstructured":"Fabbri, G., Gozzi, F., and \u015awi\u0119ch, A. (2017). Probability Theory and Stochastic Modelling, Springer International Publishing."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, G., Wu, Z., and Xiong, J. (2018). An Introduction to Optimal Control of FBSDE with Incomplete Information, Springer International Publishing. Springer Briefs in Mathematics.","DOI":"10.1007\/978-3-319-79039-8"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1051\/cocv\/2021085","article-title":"Mean field approach to stochastic control with partial information","volume":"27","author":"Bensoussan","year":"2021","journal-title":"ESAIM Control. Optim. Calc. Var."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tottori, T., and Kobayashi, T.J. (2022). Memory-Limited Partially Observable Stochastic Control and Its Mean-Field Control Approach. Entropy, 24.","DOI":"10.3390\/e24111599"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/TAC.1962.1105490","article-title":"Optimal stochastic control","volume":"7","author":"Kushner","year":"1962","journal-title":"IRE Trans. Autom. Control."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yong, J., and Zhou, X.Y. (1999). Stochastic Controls, Springer.","DOI":"10.1007\/978-1-4612-1466-3"},{"key":"ref_17","unstructured":"Nisio, M. (2015). Probability Theory and Stochastic Modelling, Springer."},{"key":"ref_18","unstructured":"Bensoussan, A. (2018). Interdisciplinary Applied Mathematics, Springer International Publishing."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kushner, H.J., and Dupuis, P.G. (1992). Numerical Methods for Stochastic Control Problems in Continuous Time, Springer.","DOI":"10.1007\/978-1-4684-0441-8"},{"key":"ref_20","unstructured":"Fleming, W.H., and Soner, H.M. (2006). Controlled Markov Processes and Viscosity Solutions, Springer. [2nd ed.]. Number 25 in Applications of Mathematics."},{"key":"ref_21","unstructured":"Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley-Interscience."},{"key":"ref_22","unstructured":"Pontryagin, L.S. (1987). Mathematical Theory of Optimal Processes, CRC Press."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Vinter, R. (2010). Optimal Control, Birkh\u00e4user Boston.","DOI":"10.1007\/978-0-8176-8086-2"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lewis, F.L., Vrabie, D., and Syrmos, V.L. (2012). Optimal Control, John Wiley & Sons.","DOI":"10.1002\/9781118122631"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Aschepkov, L.T., Dolgy, D.V., Kim, T., and Agarwal, R.P. (2016). Optimal Control, Springer International Publishing.","DOI":"10.1007\/978-3-319-49781-5"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bensoussan, A., Frehse, J., and Yam, P. (2013). Mean Field Games and Mean Field Type Control Theory, Springer. Springer Briefs in Mathematics.","DOI":"10.1007\/978-1-4614-8508-7"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Carmona, R., and Delarue, F. (2018). Probabilistic Theory of Mean Field Games with Applications I, Springer Nature. Number Volume 83 in Probability Theory and Stochastic Modelling.","DOI":"10.1007\/978-3-319-56436-4"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Carmona, R., and Delarue, F. (2018). Probabilistic Theory of Mean Field Games with Applications II, Springer International Publishing. Volume 84, Probability Theory and Stochastic Modelling.","DOI":"10.1007\/978-3-319-56436-4"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/978-3-319-11292-3_4","article-title":"The Master Equation for Large Population Equilibriums","volume":"Volume 100","author":"Crisan","year":"2014","journal-title":"Stochastic Analysis and Applications 2014"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1441","DOI":"10.1016\/j.matpur.2014.11.005","article-title":"The Master equation in mean field theory","volume":"103","author":"Bensoussan","year":"2015","journal-title":"J. Math. Pures Appl."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2093","DOI":"10.1016\/j.spa.2016.10.004","article-title":"On the interpretation of the Master Equation","volume":"127","author":"Bensoussan","year":"2017","journal-title":"Stoch. Process. Their Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1371","DOI":"10.1016\/0041-5553(63)90353-7","article-title":"On a method of successive approximations for the solution of problems of optimal control","volume":"2","author":"Krylov","year":"1963","journal-title":"USSR Comput. Math. Math. Phys."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/0005-1098(66)90009-4","article-title":"Successive approximation methods for the solution of optimal control problems","volume":"3","author":"Mitter","year":"1966","journal-title":"Automatica"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1002\/oca.4660030201","article-title":"Method of successive approximations for solution of optimal control problems","volume":"3","author":"Chernousko","year":"1982","journal-title":"Optim. Control. Appl. Methods"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lenhart, S., and Workman, J.T. (2007). Optimal Control Applied to Biological Models, Chapman and Hall\/CRC.","DOI":"10.1201\/9781420011418"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"20210241","DOI":"10.1098\/rsif.2021.0241","article-title":"Implementation and acceleration of optimal control for systems biology","volume":"18","author":"Sharp","year":"2021","journal-title":"J. R. Soc. Interface"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF02251947","article-title":"A numerical method for solving parabolic equations with opposite orientations","volume":"20","author":"Hackbusch","year":"1978","journal-title":"Computing"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1007\/s10589-011-9454-7","article-title":"Convergence of the forward-backward sweep method in optimal control","volume":"53","author":"McAsey","year":"2012","journal-title":"Comput. Optim. Appl."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Carlini, E., and Silva, F.J. (2013, January 10\u201313). Semi-Lagrangian schemes for mean field game models. Proceedings of the 52nd IEEE Conference on Decision and Control, Firenze, Italy.","DOI":"10.1109\/CDC.2013.6760358"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1137\/120902987","article-title":"A Fully Discrete Semi-Lagrangian Scheme for a First Order Mean Field Game Problem","volume":"52","author":"Carlini","year":"2014","journal-title":"SIAM J. Numer. Anal."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4269","DOI":"10.3934\/dcds.2015.35.4269","article-title":"A semi-Lagrangian scheme for a degenerate second order mean field game system","volume":"35","author":"Carlini","year":"2015","journal-title":"Discret. Contin. Dyn. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lauriere, M. (2021). Numerical Methods for Mean Field Games and Mean Field Type Control. arXiv.","DOI":"10.1090\/psapm\/078\/06"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1137\/0306023","article-title":"On the Separation Theorem of Stochastic Control","volume":"6","author":"Wonham","year":"1968","journal-title":"SIAM J. Control."},{"key":"ref_44","first-page":"1","article-title":"Maximum Principle Based Algorithms for Deep Learning","volume":"18","author":"Li","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"113133","DOI":"10.1016\/j.cam.2020.113133","article-title":"Symplectic Runge\u2013Kutta discretization of a regularized forward\u2013backward sweep iteration for optimal control problems","volume":"383","author":"Liu","year":"2021","journal-title":"J. Comput. Appl. Math."},{"key":"ref_46","unstructured":"Bellman, R. (1957). Dynamic Programming, Princeton University Press."},{"key":"ref_47","unstructured":"Howard, R.A. (1960). Dynamic Programming and Markov Processes, John Wiley."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"200201","DOI":"10.1103\/PhysRevLett.95.200201","article-title":"Linear Theory for Control of Nonlinear Stochastic Systems","volume":"95","author":"Kappen","year":"2005","journal-title":"Phys. Rev. Lett."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"P11011","DOI":"10.1088\/1742-5468\/2005\/11\/P11011","article-title":"Path integrals and symmetry breaking for optimal control theory","volume":"2005","author":"Kappen","year":"2005","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1109\/TAC.2016.2547979","article-title":"An Iterative Method for Nonlinear Stochastic Optimal Control Based on Path Integrals","volume":"62","author":"Satoh","year":"2017","journal-title":"IEEE Trans. Autom. Control."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Cacace, S., Camilli, F., and Goffi, A. (2021). A policy iteration method for Mean Field Games. arXiv.","DOI":"10.1051\/cocv\/2021081"},{"key":"ref_52","unstructured":"Lauri\u00e8re, M., Song, J., and Tang, Q. (2021). Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians. arXiv."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Camilli, F., and Tang, Q. (2022). Rates of convergence for the policy iteration method for Mean Field Games systems. arXiv.","DOI":"10.1016\/j.jmaa.2022.126138"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"9183","DOI":"10.1073\/pnas.1922204117","article-title":"A machine learning framework for solving high-dimensional mean field game and mean field control problems","volume":"117","author":"Ruthotto","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"e2024713118","DOI":"10.1073\/pnas.2024713118","article-title":"Alternating the population and control neural networks to solve high-dimensional stochastic mean-field games","volume":"118","author":"Lin","year":"2021","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1016\/j.crma.2014.07.008","article-title":"Dynamic programming for mean-field type control","volume":"352","author":"Pironneau","year":"2014","journal-title":"C. R. Math."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"902","DOI":"10.1007\/s10957-015-0785-x","article-title":"Dynamic programming for mean-field type control","volume":"169","author":"Pironneau","year":"2016","journal-title":"J. Optim. Theory Appl."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1051\/cocv\/2017019","article-title":"Bellman equation and viscosity solutions for mean-field stochastic control problem","volume":"24","author":"Pham","year":"2018","journal-title":"ESAIM Control. Optim. Calc. Var."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/208\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:12:38Z","timestamp":1760119958000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/208"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,21]]},"references-count":58,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["e25020208"],"URL":"https:\/\/doi.org\/10.3390\/e25020208","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,21]]}}}