{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T15:45:28Z","timestamp":1787240728109,"version":"build-2736575974"},"reference-count":36,"publisher":"Institute for Operations Research and the Management Sciences (INFORMS)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Operations Research"],"published-print":{"date-parts":[[2026,5]]},"abstract":"<jats:p>Bayesian Learning for Data-Driven Stochastic Optimal Control<\/jats:p>\n                  <jats:p>Stochastic optimal control (SOC) provides a principled approach to dynamic decision making under uncertainty. It models the transition of the system state with a dynamic equation driven by randomness, with the assumption that the distribution of randomness is known. However, in practical problems, modeling randomness frequently depends on data or observations, which introduces uncertainty regarding the distribution of randomness. To address the data-driven SOC problem, in \u201cEpisodic Bayesian Optimal Control with Unknown Randomness Distributions,\u201d Shapiro, Zhou, Lin, and Wang propose a new approach that incorporates Bayesian learning with optimal control in an episodic manner. They show the convergence and convergence rate results for their approach. They also develop an efficient computational method for a class of problems that have convex cost functions and linear state dynamics.<\/jats:p>","DOI":"10.1287\/opre.2023.0446","type":"journal-article","created":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T13:43:17Z","timestamp":1753278197000},"page":"1437-1455","source":"Crossref","is-referenced-by-count":1,"title":["Episodic Bayesian Optimal Control with Unknown Randomness Distributions"],"prefix":"10.1287","volume":"74","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4776-0053","authenticated-orcid":false,"given":"Alexander","family":"Shapiro","sequence":"first","affiliation":[{"name":"H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5399-6508","authenticated-orcid":false,"given":"Enlu","family":"Zhou","sequence":"additional","affiliation":[{"name":"H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6967-8237","authenticated-orcid":false,"given":"Yifan","family":"Lin","sequence":"additional","affiliation":[{"name":"H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2943-434X","authenticated-orcid":false,"given":"Yuhao","family":"Wang","sequence":"additional","affiliation":[{"name":"H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"109","reference":[{"key":"B1","unstructured":"Abbasi-Yadkori Y, Szepesv\u00e1ri C (2015) Bayesian optimal control of smoothly parameterized systems.\n                      UAI\u201915: Proc. 31st Conf. Uncertainty Artificial Intelligence\n                      (AUAI Press, Arlington, VA), 2\u201311."},{"key":"B2","unstructured":"Abeille M, Lazaric A (2018) Improved regret bounds for Thompson sampling in linear quadratic control problems.\n                      Internat. Conf. Machine Learn.\n                      (PMLR, New York), 1\u20139."},{"key":"B3","volume-title":"-Optimal Control and Related Minimax Design Problems \u2013 A Dynamic Game Approach","author":"Basar T","year":"2008"},{"key":"B4","volume-title":"Stochastic Optimal Control, the Discrete Time Case","author":"Bertsekas D","year":"1978"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1137\/17M1137917"},{"key":"B6","doi-asserted-by":"crossref","unstructured":"Blanchet J, Shapiro A (2023) Statistical limit theorems in distributionally robust optimization.\n                      WSC\u201923: Proc. Winter Simulation Conf.\n                      (IEEE Press, Piscataway, NJ), 31\u201345.","DOI":"10.1109\/WSC60868.2023.10408421"},{"key":"B7","first-page":"23","volume-title":"Le Calcul Des Probabilites et Ses Applications","author":"Doob JL","year":"1948"},{"key":"B8","volume-title":"Optimal Learning: Computational Procedures for Bayes-Adaptive Markov Decision Processes","author":"Duff MO","year":"2002"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5604-5"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1016\/0304-4068(89)90018-9"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1137\/S0363012901383837"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jet.2004.12.006"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974263.ch11"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1561\/2400000044"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1287\/educ.1063.0021"},{"key":"B16","doi-asserted-by":"crossref","unstructured":"Lin Y, Ren Y, Zhou E (2022) Bayesian risk Markov decision processes. Koyejo S, Mohamed S, Agarwal A, Belgrave D, Cho K, Oh A, eds.\n                      NIPS\u201922: Proc. 36th Internat. Conf. Neural Inform. Processing Systems\n                      (Curran Associates, Red Hook, NY), 17430\u201317442.","DOI":"10.52202\/068431-1267"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1137\/22M1478951"},{"key":"B18","unstructured":"Osband I, Van Roy B (2014) Near-optimal reinforcement learning in factored MDPs.\n                      NIPS\u201914: Proc. 28th Internat. Conf. Neural Inform. Processing Systems\n                      , vol. 1 (MIT Press, Cambridge, MA), 604\u2013612."},{"key":"B19","unstructured":"Osband I, Van Roy B (2017) Why is posterior sampling better than optimism for reinforcement learning?\n                      Internat. Conf. Machine Learning\n                      (PMLR), 2701\u20132710."},{"key":"B20","unstructured":"Osband I, Russo D, Van Roy B (2013) (More) efficient reinforcement learning via posterior sampling.\n                      NIPS\u201913: Proc. 27th Internat. Conf. Neural Inform. Processing Systems\n                      , vol. 2 (Curran Associates, Red Hook, NY), 3003\u20133011."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1007\/BF01582895"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.2307\/1426080"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/BF00535479"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2011.11.005"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.orl.2021.06.019"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611976595"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1137\/21M1465548"},{"issue":"81","key":"B28","first-page":"1","volume":"19","author":"S\u00eerbu M","year":"2014","journal-title":"Electronic Comm. Probab."},{"key":"B29","unstructured":"Strens MJA (2000) A Bayesian framework for reinforcement learning.\n                      ICML\u201900: Proc. 17th Internat. Conf. Machine Learn.\n                      (Morgan Kaufmann Publishers Inc. San Francisco), 943\u2013950."},{"key":"B30","unstructured":"Theocharous G, Wen Z, Abbasi-Yadkori Y, Vlassis N (2017) Posterior sampling for large scale reinforcement learning. Preprint, submitted November 21, https:\/\/arxiv.org\/abs\/1711.07979."},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1137\/18M1210514"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511802256"},{"issue":"2","key":"B33","first-page":"430","volume":"61","author":"Van Parys BPG","year":"2016","journal-title":"IEEE Trans. Automatic Control"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1137\/16M1101933"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2020.3030884"},{"key":"B36","volume-title":"Foundations of Inventory Management","author":"Zipkin P","year":"2000"}],"container-title":["Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/pubsonline.informs.org\/doi\/pdf\/10.1287\/opre.2023.0446","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T08:15:22Z","timestamp":1779696922000},"score":1,"resource":{"primary":{"URL":"https:\/\/pubsonline.informs.org\/doi\/10.1287\/opre.2023.0446"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5]]},"references-count":36,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5]]}},"alternative-id":["10.1287\/opre.2023.0446"],"URL":"https:\/\/doi.org\/10.1287\/opre.2023.0446","relation":{},"ISSN":["0030-364X","1526-5463"],"issn-type":[{"value":"0030-364X","type":"print"},{"value":"1526-5463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5]]}}}