{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T09:48:14Z","timestamp":1770544094043,"version":"3.49.0"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2023,3,28]],"date-time":"2023-03-28T00:00:00Z","timestamp":1679961600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,3,28]],"date-time":"2023-03-28T00:00:00Z","timestamp":1679961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007210","name":"RWTH Aachen University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007210","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We explore efficient estimation of statistical quantities, particularly rare event probabilities, for stochastic reaction networks. Consequently, we propose an importance sampling (IS) approach to improve the Monte Carlo (MC) estimator efficiency based on an approximate tau-leap scheme. The crucial step in the IS framework is choosing an appropriate change of probability measure to achieve substantial variance reduction. This task is typically challenging and often requires insights into the underlying problem. Therefore, we propose an automated approach to obtain a highly efficient path-dependent measure change based on an original connection in the stochastic reaction network context between finding optimal IS parameters within a class of probability measures and a stochastic optimal control formulation. Optimal IS parameters are obtained by solving a variance minimization problem. First, we derive an associated dynamic programming equation. Analytically solving this backward equation is challenging, hence we propose an approximate dynamic programming formulation to find near-optimal control parameters. To mitigate the curse of dimensionality, we propose a learning-based method to approximate the value function using a neural network, where the parameters are determined via a stochastic optimization algorithm. Our analysis and numerical experiments verify that the proposed learning-based IS approach substantially reduces MC estimator variance, resulting in a lower computational complexity in the rare event regime, compared with standard tau-leap MC estimators.<\/jats:p>","DOI":"10.1007\/s11222-023-10222-6","type":"journal-article","created":{"date-parts":[[2023,3,28]],"date-time":"2023-03-28T05:03:04Z","timestamp":1679979784000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Learning-based importance sampling via stochastic optimal control for stochastic reaction networks"],"prefix":"10.1007","volume":"33","author":[{"given":"Chiheb","family":"Ben Hammouda","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nadhir","family":"Ben Rached","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ra\u00fal","family":"Tempone","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5982-9073","authenticated-orcid":false,"given":"Sophia","family":"Wiechert","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,3,28]]},"reference":[{"key":"10222_CR1","doi-asserted-by":"publisher","first-page":"195","DOI":"10.4208\/jcm.2009.10-m1004","volume":"28","author":"A Abdulle","year":"2010","unstructured":"Abdulle, A., Yucheng, H., Li, T.: Chebyshev methods with discrete noise: the $$\\tau $$-ROCK methods. J. Comput. Math. 28, 195\u2013217 (2010)","journal-title":"J. Comput. Math."},{"key":"10222_CR2","unstructured":"Ahn, T.-H., Sandu, A., Han, X. Implicit simulation methods for stochastic chemical kinetics (2013). arXiv:1303.3614"},{"issue":"1","key":"10222_CR3","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1137\/110840546","volume":"10","author":"D Anderson","year":"2012","unstructured":"Anderson, D., Higham, D.: Multilevel Monte Carlo for continuous Markov chains, with applications in biochemical kinetics. SIAM Multiscale Model. Simul. 10(1), 146\u2013179 (2012)","journal-title":"SIAM Multiscale Model. Simul."},{"issue":"21","key":"10222_CR4","doi-asserted-by":"publisher","first-page":"214107","DOI":"10.1063\/1.2799998","volume":"127","author":"DF Anderson","year":"2007","unstructured":"Anderson, D.F.: A modified next reaction method for simulating chemical systems with time dependent propensities and delays. J. Chem. Phys. 127(21), 214107 (2007)","journal-title":"J. Chem. Phys."},{"key":"10222_CR5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-16895-1","volume-title":"Stochastic Analysis of Biochemical Systems","author":"DF Anderson","year":"2015","unstructured":"Anderson, D.F., Kurtz, T.G.: Stochastic Analysis of Biochemical Systems, vol. 1. Springer, Berlin (2015)"},{"issue":"18","key":"10222_CR6","doi-asserted-by":"publisher","first-page":"4183","DOI":"10.1103\/PhysRevLett.86.4183","volume":"86","author":"JP Aparicio","year":"2001","unstructured":"Aparicio, J.P., Solari, H.G.: Population dynamics: Poisson approximation and its relation to the Langevin process. Phys. Rev. Lett. 86(18), 4183 (2001)","journal-title":"Phys. Rev. Lett."},{"key":"10222_CR7","unstructured":"Banisch, R., Hartmann, C. Meshless discretization of LQ-type stochastic control problems (2013). arXiv:1309.7497"},{"issue":"2","key":"10222_CR8","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1080\/07362994.2015.1116396","volume":"34","author":"C Bayer","year":"2016","unstructured":"Bayer, C., Moraes, A., Tempone, R., Vilanova, P.: An efficient forward-reverse expectation\u2013maximization algorithm for statistical inference in stochastic reaction networks. Stoch. Anal. Appl. 34(2), 193\u2013231 (2016)","journal-title":"Stoch. Anal. Appl."},{"key":"10222_CR9","unstructured":"Ben Hammouda, C.: Hierarchical approximation methods for option pricing and stochastic reaction networks. Ph.D. Thesis (2020)"},{"issue":"6","key":"10222_CR10","doi-asserted-by":"publisher","first-page":"1665","DOI":"10.1007\/s11222-020-09965-3","volume":"30","author":"C Ben\u00a0Hammouda","year":"2020","unstructured":"Ben\u00a0Hammouda, C., Rached, N.B., Tempone, R.: Importance sampling for a robust and efficient multilevel Monte Carlo estimator for stochastic reaction networks. Stat. Comput. 30(6), 1665\u20131689 (2020)","journal-title":"Stat. Comput."},{"issue":"2","key":"10222_CR11","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1007\/s11075-016-0158-z","volume":"74","author":"C Ben Hammouda","year":"2017","unstructured":"Ben Hammouda, C., Moraes, A., Tempone, R.: Multilevel hybrid split-step implicit tau-leap. Numer. Algorithms 74(2), 527\u2013560 (2017)","journal-title":"Numer. Algorithms"},{"issue":"6","key":"10222_CR12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11222-021-10055-1","volume":"31","author":"N Ben Rached","year":"2021","unstructured":"Ben Rached, N., Haji-Ali, A.-L., Rubino, G., Tempone, R.: Efficient importance sampling for large sums of independent and identically distributed random variables. Stat. Comput. 31(6), 1\u201313 (2021)","journal-title":"Stat. Comput."},{"key":"10222_CR13","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-1686-9","volume-title":"Mathematical Models in Population Biology and Epidemiology","author":"F Brauer","year":"2012","unstructured":"Brauer, F., Castillo-Chavez, C.: Mathematical Models in Population Biology and Epidemiology, vol. 40. Springer, Berlin (2012)"},{"key":"10222_CR14","unstructured":"Cao, Y., Petzold, L.: Trapezoidal tau-leaping formula for the stochastic simulation of biochemical systems. In: Proceedings of Foundations of Systems Biology in Engineering (FOSBE 2005), pp. 149\u2013152 (2005)"},{"issue":"2","key":"10222_CR15","doi-asserted-by":"publisher","first-page":"07B605\\_1","DOI":"10.1063\/1.4811286","volume":"139","author":"Y Cao","year":"2013","unstructured":"Cao, Y., Liang, J.: Adaptively biased sequential importance sampling for rare events in reaction networks with comparison to exact solutions from finite buffer dCME method. J. Chem. Phys. 139(2), 07B605_1 (2013)","journal-title":"J. Chem. Phys."},{"issue":"4","key":"10222_CR16","first-page":"01B628","volume":"134","author":"BJ Daigle\u00a0Jr","year":"2011","unstructured":"Daigle\u00a0Jr, B.J., Roh, M.K., Gillespie, D.T., Petzold, L.R.: Automated estimation of rare event probabilities in biochemical systems. J. Chem. Phys. 134(4), 01B628 (2011)","journal-title":"J. Chem. Phys."},{"issue":"1","key":"10222_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1137\/110842545","volume":"10","author":"P Dupuis","year":"2012","unstructured":"Dupuis, P., Spiliopoulos, K., Wang, H.: Importance sampling for multiscale diffusions. Multiscale Model. Simul. 10(1), 1\u201327 (2012)","journal-title":"Multiscale Model. Simul."},{"key":"10222_CR18","unstructured":"Engblom, S.: On the stability of stochastic jump kinetics (2012). arXiv:1202.3892"},{"key":"10222_CR19","series-title":"Wiley Series in Probability and Mathematical Statistics","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316658","volume-title":"Markov Processes: Characterization and Convergence","author":"SN Ethier","year":"1986","unstructured":"Ethier, S.N., Kurtz, T.G.: Markov Processes: Characterization and Convergence. Wiley Series in Probability and Mathematical Statistics, Wiley, New York (1986)"},{"key":"10222_CR20","volume-title":"Controlled Markov Processes and Viscosity Solutions","author":"WH Fleming","year":"2006","unstructured":"Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions, vol. 25. Springer, Berlin (2006)"},{"issue":"3","key":"10222_CR21","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1287\/opre.1070.0496","volume":"56","author":"MB Giles","year":"2008","unstructured":"Giles, M.B.: Multilevel Monte Carlo path simulation. Oper. Res. 56(3), 607\u2013617 (2008)","journal-title":"Oper. Res."},{"key":"10222_CR22","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1017\/S096249291500001X","volume":"24","author":"MB Giles","year":"2015","unstructured":"Giles, M.B.: Multilevel Monte Carlo methods. Acta Numer. 24, 259\u2013328 (2015)","journal-title":"Acta Numer."},{"issue":"22","key":"10222_CR23","doi-asserted-by":"publisher","first-page":"224103","DOI":"10.1063\/1.5090979","volume":"150","author":"CS Gillespie","year":"2019","unstructured":"Gillespie, C.S., Golightly, A.: Guided proposals for efficient weighted stochastic simulation. J. Chem. Phys. 150(22), 224103 (2019)","journal-title":"J. Chem. Phys."},{"issue":"17","key":"10222_CR24","doi-asserted-by":"publisher","first-page":"174103","DOI":"10.1063\/1.3116791","volume":"130","author":"DT Gillespie","year":"2009","unstructured":"Gillespie, D.T., Roh, M., Petzold, L.R.: Refining the weighted stochastic simulation algorithm. J. Chem. Phys. 130(17), 174103 (2009)","journal-title":"J. Chem. Phys."},{"issue":"4","key":"10222_CR25","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1016\/0021-9991(76)90041-3","volume":"22","author":"DT Gillespie","year":"1976","unstructured":"Gillespie, D.T.: A general method for numerically simulating the stochastic time evolution of coupled chemical reactions. J. Comput. Phys. 22(4), 403\u2013434 (1976)","journal-title":"J. Comput. Phys."},{"issue":"4","key":"10222_CR26","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1063\/1.1378322","volume":"115","author":"DT Gillespie","year":"2001","unstructured":"Gillespie, D.T.: Approximate accelerated stochastic simulation of chemically reacting systems. J. Chem. Phys. 115(4), 1716\u20131733 (2001)","journal-title":"J. Chem. Phys."},{"issue":"6","key":"10222_CR27","doi-asserted-by":"publisher","first-page":"e1003669","DOI":"10.1371\/journal.pcbi.1003669","volume":"10","author":"A Gupta","year":"2014","unstructured":"Gupta, A., Briat, C., Khammash, M.: A scalable computational framework for establishing long-term behavior of stochastic reaction networks. PLoS Comput. Biol. 10(6), e1003669 (2014)","journal-title":"PLoS Comput. Biol."},{"issue":"1","key":"10222_CR28","doi-asserted-by":"publisher","first-page":"350","DOI":"10.3390\/e16010350","volume":"16","author":"C Hartmann","year":"2014","unstructured":"Hartmann, C., Banisch, R., Sarich, M., Badowski, T., Sch\u00fctte, C.: Characterization of rare events in molecular dynamics. Entropy 16(1), 350\u2013376 (2014)","journal-title":"Entropy"},{"issue":"6","key":"10222_CR29","doi-asserted-by":"publisher","first-page":"063107","DOI":"10.1063\/1.5090271","volume":"29","author":"C Hartmann","year":"2019","unstructured":"Hartmann, C., Kebiri, O., Neureither, L., Richter, L.: Variational approach to rare event simulation using least-squares regression. Chaos Interdisc. J. Nonlinear Sci. 29(6), 063107 (2019)","journal-title":"Chaos Interdisc. J. Nonlinear Sci."},{"issue":"11","key":"10222_CR30","doi-asserted-by":"publisher","first-page":"626","DOI":"10.3390\/e19110626","volume":"19","author":"C Hartmann","year":"2017","unstructured":"Hartmann, C., Richter, L., Sch\u00fctte, C., Zhang, W.: Variational characterization of free energy: theory and algorithms. Entropy 19(11), 626 (2017)","journal-title":"Entropy"},{"issue":"1","key":"10222_CR31","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1007\/s00440-017-0755-3","volume":"170","author":"C Hartmann","year":"2018","unstructured":"Hartmann, C., Sch\u00fctte, C., Weber, M., Zhang, W.: Importance sampling in path space for diffusion processes with slow-fast variables. Probab. Theory Relat. Fields 170(1), 177\u2013228 (2018)","journal-title":"Probab. Theory Relat. Fields"},{"issue":"7","key":"10222_CR32","doi-asserted-by":"publisher","first-page":"1671","DOI":"10.1007\/s11538-009-9419-5","volume":"71","author":"SC Hensel","year":"2009","unstructured":"Hensel, S.C., Rawlings, J.B., Yin, J.: Stochastic kinetic modeling of vesicular stomatitis virus intracellular growth. Bull. Math. Biol. 71(7), 1671\u20131692 (2009)","journal-title":"Bull. Math. Biol."},{"key":"10222_CR33","doi-asserted-by":"crossref","unstructured":"Kebiri, O., Neureither, L., Hartmann, C. Adaptive importance sampling with forward-backward stochastic differential equations. In: International workshop on Stochastic Dynamics out of Equilibrium, pp. 265\u2013281. Springer, Berlin (2017)","DOI":"10.1007\/978-3-030-15096-9_7"},{"key":"10222_CR34","unstructured":"Kingma, D.P., Ba, J. Adam: A Method for Stochastic Optimization (2014). arXiv:1412.6980"},{"key":"10222_CR35","volume-title":"Handbook of Monte Carlo Methods","author":"DP Kroese","year":"2013","unstructured":"Kroese, D.P., Taimre, T., Botev, Z.I.: Handbook of Monte Carlo Methods. Wiley, New York (2013)"},{"issue":"16","key":"10222_CR36","doi-asserted-by":"publisher","first-page":"10B619","DOI":"10.1063\/1.2987701","volume":"129","author":"H Kuwahara","year":"2008","unstructured":"Kuwahara, H., Mura, I.: An efficient and exact stochastic simulation method to analyze rare events in biochemical systems. J. Chem. Phys. 129(16), 10B619 (2008)","journal-title":"J. Chem. Phys."},{"issue":"4","key":"10222_CR37","doi-asserted-by":"publisher","first-page":"738","DOI":"10.1287\/mnsc.41.4.738","volume":"41","author":"P L\u2019Ecuyer","year":"1995","unstructured":"L\u2019Ecuyer, P.: Note: On the interchange of derivative and expectation for likelihood ratio derivative estimators. Manag. Sci. 41(4), 738\u2013747 (1995)","journal-title":"Manag. Sci."},{"issue":"2","key":"10222_CR38","doi-asserted-by":"publisher","first-page":"01B612\\_1","DOI":"10.1063\/1.4904980","volume":"142","author":"C Lester","year":"2015","unstructured":"Lester, C., Yates, C.A., Giles, M.B., Baker, R.E.: An adaptive multi-level simulation algorithm for stochastic biological systems. J. Chem. Phys. 142(2), 01B612_1 (2015)","journal-title":"J. Chem. Phys."},{"key":"10222_CR39","doi-asserted-by":"crossref","unstructured":"Li, T.: Analysis of explicit tau-leaping schemes for simulating chemically reacting systems. Multiscale Model. Simul. 6(2), 417\u2013436 (2007)","DOI":"10.1137\/06066792X"},{"issue":"4","key":"10222_CR40","doi-asserted-by":"publisher","first-page":"A2091","DOI":"10.1137\/140972081","volume":"38","author":"A Moraes","year":"2016","unstructured":"Moraes, A., Tempone, R., Vilanova, P.: A multilevel adaptive reaction-splitting simulation method for stochastic reaction networks. SIAM J. Sci. Comput. 38(4), A2091\u2013A2117 (2016)","journal-title":"SIAM J. Sci. Comput."},{"issue":"4","key":"10222_CR41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42985-021-00102-x","volume":"2","author":"N N\u00fcsken","year":"2021","unstructured":"N\u00fcsken, N., Richter, L.: Solving high-dimensional Hamilton\u2013Jacobi\u2013Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space. Partial Differ. Equ. Appl. 2(4), 1\u201348 (2021)","journal-title":"Partial Differ. Equ. Appl."},{"key":"10222_CR42","unstructured":"Rached, N.B., Haji-Ali, A.L., Mohan, S., Tempone, R.: Double Loop Monte Carlo Estimator with Importance Sampling for Mckean\u2013Vlasov Stochastic Differential Equation (2022). arXiv:2207.06926"},{"issue":"11","key":"10222_CR43","doi-asserted-by":"publisher","first-page":"4999","DOI":"10.1063\/1.1545446","volume":"118","author":"CV Rao","year":"2003","unstructured":"Rao, C.V., Arkin, A.P.: Stochastic chemical kinetics and the quasi-steady-state assumption: application to the Gillespie algorithm. J. Chem. Phys. 118(11), 4999\u20135010 (2003)","journal-title":"J. Chem. Phys."},{"key":"10222_CR44","unstructured":"Rathinam, M.: Moment Growth Bounds on Continuous Time Markov Processes on Non-negative Integer Lattices (2013). arXiv:1304.5169"},{"issue":"2","key":"10222_CR45","doi-asserted-by":"publisher","first-page":"897","DOI":"10.1016\/j.jcp.2006.10.034","volume":"224","author":"M Rathinam","year":"2007","unstructured":"Rathinam, M., El Samad, H.: Reversible-equivalent-monomolecular tau: a leaping method for small number and stiff stochastic chemical systems. J. Comput. Phys. 224(2), 897\u2013923 (2007)","journal-title":"J. Comput. Phys."},{"issue":"8","key":"10222_CR46","doi-asserted-by":"publisher","first-page":"3097","DOI":"10.1007\/s11538-018-0509-0","volume":"81","author":"MK Roh","year":"2019","unstructured":"Roh, M.K.: Data-driven method for efficient characterization of rare event probabilities in biochemical systems. Bull. Math. Biol. 81(8), 3097\u20133120 (2019)","journal-title":"Bull. Math. Biol."},{"issue":"17","key":"10222_CR47","doi-asserted-by":"publisher","first-page":"174106","DOI":"10.1063\/1.3493460","volume":"133","author":"MK Roh","year":"2010","unstructured":"Roh, M.K., Gillespie, D.T., Petzold, L.R.: State-dependent biasing method for importance sampling in the weighted stochastic simulation algorithm. J. Chem. Phys. 133(17), 174106 (2010)","journal-title":"J. Chem. Phys."},{"issue":"3","key":"10222_CR48","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1006\/jtbi.2002.3078","volume":"218","author":"S Ranjan","year":"2002","unstructured":"Ranjan, S., You, L., Summers, J., Stochastic vs, J.Y.: deterministic modeling of intracellular viral kinetics. J. Theor. Biol. 218(3), 309\u2013321 (2002)","journal-title":"J. Theor. Biol."},{"issue":"6","key":"10222_CR49","doi-asserted-by":"publisher","first-page":"A2654","DOI":"10.1137\/14096493X","volume":"36","author":"W Zhang","year":"2014","unstructured":"Zhang, W., Wang, H., Hartmann, C., Weber, M., Sch\u00fctte, C.: Applications of the cross-entropy method to importance sampling and optimal control of diffusions. SIAM J. Sci. Comput. 36(6), A2654\u2013A2672 (2014)","journal-title":"SIAM J. Sci. Comput."}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-023-10222-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11222-023-10222-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-023-10222-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,7]],"date-time":"2023-05-07T04:02:38Z","timestamp":1683432158000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11222-023-10222-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,28]]},"references-count":49,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["10222"],"URL":"https:\/\/doi.org\/10.1007\/s11222-023-10222-6","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"value":"0960-3174","type":"print"},{"value":"1573-1375","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,28]]},"assertion":[{"value":"21 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 March 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"58"}}