{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T19:43:50Z","timestamp":1787341430109,"version":"build-2736575974"},"reference-count":49,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"3","funder":[{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"publisher","award":["FA9550-12-1-0200"],"award-info":[{"award-number":["FA9550-12-1-0200"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Optim."],"published-print":{"date-parts":[[2018,1]]},"abstract":"<jats:p>We consider the problem of estimating the expected value of information (the knowledge gradient) for Bayesian learning problems where the belief model is nonlinear in the parameters. Our goal is to maximize an objective function represented by a nonlinear parametric belief model, while simultaneously learning the unknown parameters, by guiding a sequential experimentation process which is expensive. We overcome the problem of computing the expected value of an experiment, which is computationally intractable, by using a sampled approximation, which helps to guide experiments but does not provide an accurate estimate of the unknown parameters. We then introduce a resampling process which allows the sampled model to adapt to new information, exploiting past experiments. We show theoretically that the method generates sequences that converge asymptotically to the true parameters, while simultaneously maximizing the objective function. We show empirically that the process exhibits rapid convergence, yielding good results with a very small number of experiments.<\/jats:p>","DOI":"10.1137\/16m1073042","type":"journal-article","created":{"date-parts":[[2018,8,21]],"date-time":"2018-08-21T12:24:10Z","timestamp":1534854250000},"page":"2327-2359","source":"Crossref","is-referenced-by-count":5,"title":["Optimal Learning for Stochastic Optimization with Nonlinear Parametric Belief Models"],"prefix":"10.1137","volume":"28","author":[{"given":"Xinyu","family":"He","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Warren B.","family":"Powell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2018,8,21]]},"reference":[{"key":"atypb1","first-page":"39","volume":"23","author":"Agrawal S.","year":"2012","journal-title":"Proc. Mach. Learn. Res."},{"key":"atypb2","first-page":"4228","author":"Ahmadi H.","year":"2014","journal-title":"NJ"},{"key":"atypb3","unstructured":"J.Y. Audibert and S. Bubeck,\n                      Best arm identification in multi-armed bandits\n                      , in Proceedings of COLT: 23th Conference on Learning Theory, Association for Computational Learning, 2010; available athttps:\/\/hal.inria.fr\/hal-00654404."},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013689704352"},{"key":"atypb5","first-page":"535","author":"Barton R. R.","year":"2006","journal-title":"Amsterdam"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1007\/s11047-008-9098-4"},{"key":"atypb7","doi-asserted-by":"crossref","unstructured":"G. I. Bischi, M. Kopel, C. Chiarella, and F. Szidarovszky,\n                      Nonlinear Oligopolies: Stability and Bifurcations\n                      , Springer, Berlin, 2010,https:\/\/doi.org\/10.1007\/978-3-642-02106-0.","DOI":"10.1007\/978-3-642-02106-0"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1016\/j.jebo.2005.08.006"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1080\/00207720701792131"},{"key":"atypb10","first-page":"2249","author":"Chapelle O.","year":"2011","journal-title":"NY"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008349927281"},{"key":"atypb12","doi-asserted-by":"crossref","unstructured":"C.H. Chen,\n                      An effective approach to smartly allocate computing budget for discrete event simulation\n                      , in Proceedings of 1995 34th IEEE Conference on Decision and Control 3, IEEE Press, Piscataway, NJ, 1995, pp. 2598-2603,https:\/\/doi.org\/10.1109\/CDC.1995.478499.","DOI":"10.1109\/CDC.1995.478499"},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.1137\/140971129"},{"key":"atypb14","doi-asserted-by":"crossref","unstructured":"S. E. Chick,\n                      Chapter 9: Subjective probability and Bayesian methodology\n                      , in Simulation, Handbooks Oper. Res. Management Sci. 13, S. G. Henderson and B. L. Nelson, eds., North-Holland, Amsterdam, 2006, pp. 225-257,https:\/\/doi.org\/10.1016\/S0927-0507(06)13009-1.","DOI":"10.1016\/S0927-0507(06)13009-1"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1287\/opre.49.5.732.10615"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1137\/070693424"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.1080.0314"},{"key":"atypb18","doi-asserted-by":"crossref","unstructured":"M. C. Fu,\n                      Chapter 19: Gradient estimation\n                      , in Simulation, Handbooks Oper. Res. Management Sci. 13, S. G. Henderson and B. L. Nelson, eds., North-Holland, Amsterdam, 2006, pp. 575-616,https:\/\/doi.org\/10.1016\/S0927-0507(06)13019-4.","DOI":"10.1016\/S0927-0507(06)13019-4"},{"key":"atypb19","doi-asserted-by":"crossref","unstructured":"J. Gittins, K. Glazebrook, and R. Weber,\n                      Multi-Armed Bandit Allocation Indices\n                      , 2nd ed., John Wiley, New York, 2011,https:\/\/doi.org\/10.1002\/9780470980033.","DOI":"10.1002\/9780470980033"},{"key":"atypb20","first-page":"229","author":"Gramacy R. B.","year":"2011","journal-title":"Oxford"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1016\/0378-3758(95)00169-7"},{"key":"atypb22","doi-asserted-by":"crossref","unstructured":"T. Hastie, R. Tibshirani, and J. Friedman,\n                      The Elements of Statistical Learning: Data Mining, Inference, and Prediction\n                      , 2nd ed., Springer, New York, 2009,https:\/\/doi.org\/10.1007\/b94608.","DOI":"10.1007\/b94608"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2007.900656"},{"key":"atypb24","first-page":"75","author":"Hong L.","year":"2009","journal-title":"NJ"},{"key":"atypb25","first-page":"150","volume":"4","author":"Jamil M.","year":"2013","journal-title":"Int. J. Math. Model. Numer. Optim."},{"key":"atypb26","doi-asserted-by":"crossref","unstructured":"H. Jiang and U. V. Shanbhag,\n                      On the solution of stochastic optimization problems in imperfect information regimes\n                      , in Proceedings of the 2013 Winter Simulations Conference, IEEE Press, Piscataway, NJ, pp. 821-832,https:\/\/doi.org\/10.1109\/WSC.2013.6721474.","DOI":"10.1109\/WSC.2013.6721474"},{"key":"atypb27","first-page":"3801","author":"Jiang H.","year":"2015","journal-title":"NJ"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1137\/140955495"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008306431147"},{"key":"atypb30","doi-asserted-by":"crossref","unstructured":"L. P. Kaelbling,\n                      Learning in Embedded Systems\n                      , MIT Press, Cambridge, MA, 1993.","DOI":"10.7551\/mitpress\/4168.001.0001"},{"key":"atypb31","first-page":"501","author":"Kim S.-H.","year":"2006","journal-title":"Amsterdam"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1016\/0005-1098(90)90154-A"},{"key":"atypb33","unstructured":"A. J. Miller,\n                      Subset Selection in Regression\n                      , Monogr. Statist. Appl. Probab., Chapman & Hall\/CRC Press, Boca Raton, FL, 2002."},{"key":"atypb34","doi-asserted-by":"crossref","unstructured":"K. L. Moore,\n                      Iterative Learning Control for Deterministic Systems\n                      , Springer, New York, 1993.","DOI":"10.1007\/978-1-4471-1912-8"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.1100.0417"},{"key":"atypb36","doi-asserted-by":"crossref","unstructured":"S. O\u0301lafsson,\n                      Chapter 21: Metaheuristics\n                      , in Simulation, Handbooks Oper. Res. Management Sci. 13, S. G. Henderson and B. L. Nelson, eds., North-Holland, Amsterdam, 2006, pp. 633-654,https:\/\/doi.org\/10.1016\/S0927-0507(06)13021-2.","DOI":"10.1016\/S0927-0507(06)13021-2"},{"key":"atypb37","unstructured":"F. Olsson,\n                      A Literature Survey of Active Machine Learning in the Context of Natural Language Processing\n                      , Technical report 2009:06, Swedish Institute of Computer Science, Stockholm, 2009; available athttps:\/\/soda.swedish-ict.se\/3600."},{"key":"atypb38","doi-asserted-by":"crossref","unstructured":"W. B. Powell,\n                      Approximate Dynamic Programming: Solving the Curses of Dimensionality\n                      , 2nd ed., John Wiley, New York, 2011.","DOI":"10.1002\/9781118029176"},{"key":"atypb39","doi-asserted-by":"crossref","unstructured":"I. Ryzhov and W. B. Powell,\n                      The knowledge gradient algorithm for online subset selection\n                      , in Proceedings of the 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, IEEE Press, Piscataway, NJ, 2009,https:\/\/doi.org\/10.1109\/ADPRL.2009.4927537.","DOI":"10.1109\/ADPRL.2009.4927537"},{"key":"atypb40","unstructured":"B. Settles,\n                      Active Learning Literature Survey\n                      , Computer Sciences Technical Report 1648, University of Wisconsin-Madison, Madison, WI, 2009."},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007678930559"},{"key":"atypb42","doi-asserted-by":"crossref","unstructured":"J. Spall,\n                      Introduction to Stochastic Search and Optimization\n                      , 1st ed., John Wiley, New York, 2003.","DOI":"10.1002\/0471722138"},{"key":"atypb43","first-page":"560","author":"Streeter M. J.","year":"2006","journal-title":"Berlin"},{"key":"atypb44","doi-asserted-by":"crossref","unstructured":"R. Sutton and A. Barto,\n                      Reinforcement Learning: An Introduction\n                      , MIT Press, Cambridge, MA, 1998.","DOI":"10.1109\/TNN.1998.712192"},{"key":"atypb45","first-page":"175","volume":"9","author":"Szidarovszky F.","year":"2004","journal-title":"J. Econ. Res."},{"key":"atypb46","first-page":"453","volume":"15","author":"Szidarovszky F.","year":"2004","journal-title":"Pure Math. Appl."},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/25.3-4.285"},{"key":"atypb48","first-page":"107","author":"Tong S.","year":"2001","journal-title":"New York"},{"key":"atypb49","doi-asserted-by":"publisher","DOI":"10.9746\/sicetr1965.14.706"}],"container-title":["SIAM Journal on Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/16M1073042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T19:19:15Z","timestamp":1787339955000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/16M1073042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1]]},"references-count":49,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2018,1]]}},"alternative-id":["10.1137\/16M1073042"],"URL":"https:\/\/doi.org\/10.1137\/16m1073042","relation":{},"ISSN":["1052-6234","1095-7189"],"issn-type":[{"value":"1052-6234","type":"print"},{"value":"1095-7189","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1]]}}}