{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T19:24:09Z","timestamp":1787340249456,"version":"3.56.0"},"reference-count":67,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM\/ASA J. Uncertainty Quantification"],"published-print":{"date-parts":[[2017,1]]},"abstract":"<jats:p>Polynomial chaos and Gaussian process emulation are methods for surrogate-based uncertainty quantification and have been developed independently in their respective communities over the last 25 years. Despite tackling similar problems in the field, to our knowledge there has yet to be a critical comparison of the two approaches in the literature. We begin by providing a detailed description of polynomial chaos and Gaussian process approaches for building a surrogate model of a black-box function. The accuracy of each surrogate method is then tested and compared for two simulators used in industry: a land-surface model (adJULES) and a launch vehicle controller (VEGACONTROL). We analyze surrogates built on experimental designs of various size and type to investigate their performance in a range of modeling scenarios. Specifically, polynomial chaos and Gaussian process surrogates are built on Sobol sequence and tensor grid designs. Their accuracy is measured by their ability to estimate the mean, standard deviation, exceedance probabilities, and probability density function of the simulator output, as well as a root mean square error metric, based on an independent validation design. We find that one method does not unanimously outperform the other, but advantages can be gained in some cases, such that the preferred method depends on the modeling goals of the practitioner. Our conclusions are likely to depend somewhat on the modeling choices for the surrogates as well as the design strategy. We hope that this work will spark future comparisons of the two methods in their more advanced formulations and for different sampling strategies.<\/jats:p>","DOI":"10.1137\/15m1046812","type":"journal-article","created":{"date-parts":[[2017,4,25]],"date-time":"2017-04-25T16:45:37Z","timestamp":1493138737000},"page":"403-435","source":"Crossref","is-referenced-by-count":56,"title":["Comparison of Surrogate-Based Uncertainty Quantification Methods for Computationally Expensive Simulators"],"prefix":"10.1137","volume":"5","author":[{"given":"N. E.","family":"Owen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"P.","family":"Challenor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"P. P.","family":"Menon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S.","family":"Bennani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2017,4,25]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2012.04.020"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1198\/TECH.2009.08019"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.5194\/gmd-4-723-2011"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.3166\/remn.15.81-92"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.5194\/gmd-4-677-2011"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1016\/j.probengmech.2009.10.003"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2010.12.021"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2008.07.007"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1137\/0916069"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.5194\/gmd-4-701-2011"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1016\/j.jspi.2009.08.006"},{"key":"atypb12","unstructured":"N. Cressie (2015),\n                      Statistics for Spatial Data\n                      , Wiley, New York."},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1991.10475138"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1137\/S1064827503427741"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1002\/nme.3116"},{"key":"atypb16","doi-asserted-by":"crossref","unstructured":"R. P. Dwight and Z.H. Han (2009),\n                      Efficient uncertainty quantification using gradient-enhanced kriging\n                      , in Proceedings of the 50th AIAA\/ASME\/ASCE\/AHS\/ASC Structures, Structural Dynamics and Materials Conference.","DOI":"10.2514\/6.2009-2276"},{"key":"atypb17","doi-asserted-by":"crossref","unstructured":"B. Efron and R. J. Tibshirani (1994),\n                      An Introduction to the Bootstrap\n                      , CRC Press, Boca Raton, FL.","DOI":"10.1201\/9780429246593"},{"key":"atypb18","doi-asserted-by":"crossref","unstructured":"M. S. Eldred and J. Burkardt (2009),\n                      Comparison of non-intrusive polynomial chaos and stochastic collocation methods for uncertainty quantification\n                      , in Proceedings of the 47th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition.","DOI":"10.2514\/6.2009-976"},{"key":"atypb19","doi-asserted-by":"crossref","unstructured":"M. S. Eldred, C. G. Webster, and P. Constantine (2008),\n                      Evaluation of non-intrusive approaches for Wiener-Askey generalized polynomial chaos\n                      , in Proceedings of the 49th AIAA\/ASME\/ASCE\/AHS\/ASC Structures, Structural Dynamics and Materials Conference.","DOI":"10.2514\/6.2008-1892"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2007.1900"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1080\/00401706.2012.715835"},{"key":"atypb22","doi-asserted-by":"crossref","unstructured":"R. G. Ghanem and P. D. Spanos (1991),\n                      Stochastic Finite Elements: A Spectral Approach\n                      , Springer, New York.","DOI":"10.1007\/978-1-4612-3094-6"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1061\/(ASCE)0733-9399(2002)128:1(66)"},{"key":"atypb24","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1017\/S0001924000066045","volume":"101","author":"Giunta A.","year":"1997","journal-title":"Aeronaut. J."},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2007.06.016"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1007\/s00211-015-0734-5"},{"key":"atypb27","doi-asserted-by":"crossref","unstructured":"R. Haylock and A. O'Hagan (1996),\n                      On inference for outputs of computationally expensive algorithms with uncertainty on the inputs\n                      , in Bayesian Statistics, Vol. 5, J. Bernardo, J. Berger, A. Dawid, and A. Smith, eds., Oxford University Press, Oxford, UK, pp. 629-637.","DOI":"10.1093\/oso\/9780198523567.003.0041"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1137\/S1064827503426693"},{"key":"atypb29","doi-asserted-by":"crossref","unstructured":"S. Hosder, R. W. Walters, and M. Balch (2007),\n                      Efficient sampling for non-intrusive polynomial chaos applications with multiple uncertain input variables\n                      , in Proceedings of the 48th AIAA\/ASME\/ASCE\/AHS\/ASC Structures, Structural Dynamics and Materials Conference.","DOI":"10.2514\/6.2007-1939"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2013.02.035"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/87.1.1"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9868.00294"},{"key":"atypb33","doi-asserted-by":"crossref","unstructured":"S. W. Kirkpatrick (2000),\n                      Development and Validation of High Fidelity Vehicle Crash Simulation Models\n                      , Tech. report, SAE, Warrendale, PA.","DOI":"10.4271\/2000-01-0627"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1016\/j.fluiddyn.2005.12.003"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1137\/120884122"},{"key":"atypb36","doi-asserted-by":"publisher","DOI":"10.1006\/jcph.2001.6889"},{"key":"atypb37","doi-asserted-by":"publisher","DOI":"10.1006\/jcph.2002.7104"},{"key":"atypb38","unstructured":"D. Liu, A. Litvinenko, C. Schillings, and V. Schulz (2015),\n                      Quantification of Airfoil Geometry-Induced Aerodynamic Uncertainties-Comparison of Approaches\n                      , preprint,arXiv:1505.05731."},{"key":"atypb39","doi-asserted-by":"publisher","DOI":"10.1016\/j.jspi.2009.12.004"},{"key":"atypb40","doi-asserted-by":"publisher","DOI":"10.1198\/TECH.2009.08040"},{"key":"atypb41","first-page":"239","volume":"21","author":"McKay M. D.","year":"1979","journal-title":"Technometrics"},{"key":"atypb42","doi-asserted-by":"crossref","unstructured":"A. Mujumdhar, P. P. Menon, C. Roux, and S. Bennani (2015),\n                      Cross-entropy based probabilistic analysis of VEGA launcher performance\n                      , in Advances in Aerospace Guidance, Navigation and Control, J. Bordeneuve-Guibe\u0301, A. Drouin, and C. Roos, eds., Springer, Berlin.","DOI":"10.1007\/978-3-319-17518-8_41"},{"key":"atypb43","doi-asserted-by":"publisher","DOI":"10.1137\/130929461"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1016\/0022-314X(88)90025-X"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1137\/060663660"},{"key":"atypb46","doi-asserted-by":"publisher","DOI":"10.1007\/s40072-015-0055-9"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/89.4.769"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2004.05304.x"},{"key":"atypb49","unstructured":"A. O'Hagan (2013),\n                      Polynomial Chaos: A Tutorial and Critique from a Statistician's Perspective\n                      , Manuscript."},{"key":"atypb50","doi-asserted-by":"publisher","DOI":"10.1137\/15M1007914"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1016\/S0098-3004(00)00016-9"},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1137\/120882834"},{"key":"atypb53","doi-asserted-by":"crossref","unstructured":"C. E. Rasmussen and C. K. Williams (2006),\n                      Gaussian Processes for Machine Learning\n                      , MIT Press, Cambridge, MA.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"atypb54","doi-asserted-by":"publisher","DOI":"10.1016\/S0010-2180(02)00503-5"},{"key":"atypb55","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v051.i01"},{"key":"atypb56","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1177012413"},{"key":"atypb57","doi-asserted-by":"crossref","unstructured":"T. J. Santner, B. J. Williams, and W. I. Notz (2003),\n                      The Design and Analysis of Computer Experiments\n                      , Springer-Verlag, New York.","DOI":"10.1007\/978-1-4757-3799-8"},{"key":"atypb58","unstructured":"B. W. Silverman (1986),\n                      Density Estimation for Statistics and Data Analysis\n                      , Monogr. Statist. Appl. Probab. 26, CRC Press, Bora Raton, FL."},{"key":"atypb59","doi-asserted-by":"publisher","DOI":"10.1016\/j.cageo.2009.11.004"},{"key":"atypb60","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2005.03.023"},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-005-9038-8"},{"key":"atypb62","doi-asserted-by":"publisher","DOI":"10.2307\/2371268"},{"key":"atypb63","doi-asserted-by":"publisher","DOI":"10.1007\/s00382-013-1896-4"},{"key":"atypb64","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-015-0153-x"},{"key":"atypb65","doi-asserted-by":"publisher","DOI":"10.1016\/j.jsv.2007.09.017"},{"key":"atypb66","doi-asserted-by":"publisher","DOI":"10.1137\/040615201"},{"key":"atypb67","doi-asserted-by":"publisher","DOI":"10.1016\/S0021-9991(03)00092-5"}],"container-title":["SIAM\/ASA Journal on Uncertainty Quantification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/15M1046812","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T18:26:39Z","timestamp":1787336799000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/15M1046812"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,1]]},"references-count":67,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,1]]}},"alternative-id":["10.1137\/15M1046812"],"URL":"https:\/\/doi.org\/10.1137\/15m1046812","relation":{},"ISSN":["2166-2525"],"issn-type":[{"value":"2166-2525","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,1]]}}}