{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T04:26:51Z","timestamp":1776400011681,"version":"3.51.2"},"reference-count":62,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,9,13]],"date-time":"2022-09-13T00:00:00Z","timestamp":1663027200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Bayesian techniques for engineering problems, which rely on Gaussian process (GP) regression, are known for their ability to quantify epistemic and aleatory uncertainties and for being data efficient. The mathematical elegance of applying these methods usually comes at a high computational cost when compared to deterministic and empirical Bayesian methods. Furthermore, using these methods becomes practically infeasible in scenarios characterized by a large number of inputs and thousands of training data. The focus of this work is on enhancing Gaussian process based metamodeling and model calibration tasks, when the size of the training datasets is significantly large. To achieve this goal, we employ a stochastic variational inference algorithm that enables rapid statistical learning of the calibration parameters and hyperparameter tuning, while retaining the rigor of Bayesian inference. The numerical performance of the algorithm is demonstrated on multiple metamodeling and model calibration problems with thousands of training data.<\/jats:p>","DOI":"10.3390\/e24091291","type":"journal-article","created":{"date-parts":[[2022,9,13]],"date-time":"2022-09-13T21:06:52Z","timestamp":1663103212000},"page":"1291","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Multifidelity Model Calibration in Structural Dynamics Using Stochastic Variational Inference on Manifolds"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5830-8508","authenticated-orcid":false,"given":"Panagiotis","family":"Tsilifis","sequence":"first","affiliation":[{"name":"Probabilistic Design Group, General Electric Research, Niskayuna, NY 12309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Piyush","family":"Pandita","sequence":"additional","affiliation":[{"name":"Probabilistic Design Group, General Electric Research, Niskayuna, NY 12309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sayan","family":"Ghosh","sequence":"additional","affiliation":[{"name":"Probabilistic Design Group, General Electric Research, Niskayuna, NY 12309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liping","family":"Wang","sequence":"additional","affiliation":[{"name":"Probabilistic Design Group, General Electric Research, Niskayuna, NY 12309, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"571","DOI":"10.2307\/1266632","article-title":"A review of response surface methodology: A literature survey","volume":"8","author":"Hill","year":"1966","journal-title":"Technometrics"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/j.promfg.2018.02.034","article-title":"Industry 4.0\u2014A glimpse","volume":"20","author":"Vaidya","year":"2018","journal-title":"Procedia Manuf."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Rasmussen, C.E. (2003). Gaussian Processes in Machine Learning, Springer.","DOI":"10.1007\/978-3-540-28650-9_4"},{"key":"ref_4","unstructured":"Hensman, J., Fusi, N., and Lawrence, N.D. (2013, January 11\u201315). Gaussian processes for Big data. Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, Bellevue, WA, USA."},{"key":"ref_5","unstructured":"Damianou, A., and Lawrence, N.D. (2013, January 29). Deep gaussian processes. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics, Scottsdale, AZ, USA."},{"key":"ref_6","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_7","unstructured":"Gal, Y., and Ghahramani, Z. (2016, January 19\u201324). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_8","first-page":"1","article-title":"Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles","volume":"30","author":"Lakshminarayanan","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1115\/1.2888303","article-title":"Polynomial chaos in stochastic finite elements","volume":"57","author":"Ghanem","year":"1990","journal-title":"J. Appl. Mech."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.jcp.2017.04.009","article-title":"Reduced Wiener chaos representation of random fields via basis adaptation and projection","volume":"341","author":"Tsilifis","year":"2017","journal-title":"J. Comput. Phys."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"022001","DOI":"10.1115\/1.4047683","article-title":"Novel Performance-Oriented Tolerance Design Method Based on Locally Inferred Sensitivity Analysis and Improved Polynomial Chaos Expansion","volume":"143","author":"Sa","year":"2021","journal-title":"J. Mech. Des."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1615\/Int.J.UncertaintyQuantification.2018021315","article-title":"Stochastic multiobjective optimization on a budget: Application to multipass wire drawing with quantified uncertainties","volume":"8","author":"Pandita","year":"2018","journal-title":"Int. J. Uncertain. Quantif."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"101404","DOI":"10.1115\/1.4043930","article-title":"Bayesian optimal design of experiments for inferring the statistical expectation of expensive black-box functions","volume":"141","author":"Pandita","year":"2019","journal-title":"J. Mech. Des."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"111412","DOI":"10.1115\/1.4034104","article-title":"Extending expected improvement for high-dimensional stochastic optimization of expensive black-box functions","volume":"138","author":"Pandita","year":"2016","journal-title":"J. Mech. Des."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.jcp.2018.12.010","article-title":"Compressive sensing adaptation for polynomial chaos expansions","volume":"380","author":"Tsilifis","year":"2019","journal-title":"J. Comput. Phys."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"011401","DOI":"10.1115\/1.4041483","article-title":"Model discrepancy quantification in simulation-based design of dynamical systems","volume":"141","author":"Hu","year":"2019","journal-title":"J. Mech. Des."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1137\/16M1082469","article-title":"Survey of multifidelity methods in uncertainty propagation, inference, and optimization","volume":"60","author":"Peherstorfer","year":"2018","journal-title":"Siam Rev."},{"key":"ref_18","first-page":"398","article-title":"A Multi Level Monte Carlo method with control variate for elliptic PDEs with log-normal coefficients","volume":"3","author":"Nobile","year":"2015","journal-title":"Stoch. Partial Differ. Equ. Anal. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1137\/130932715","article-title":"A survey of projection-based model reduction methods for parametric dynamical systems","volume":"57","author":"Benner","year":"2015","journal-title":"SIAM Rev."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.paerosci.2008.11.001","article-title":"Recent advances in surrogate-based optimization","volume":"45","author":"Forrester","year":"2009","journal-title":"Prog. Aerosp. Sci."},{"key":"ref_21","first-page":"3251","article-title":"Multi-fidelity optimization via surrogate modelling","volume":"463","author":"Forrester","year":"2007","journal-title":"Proc. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1137\/17M1141096","article-title":"Compressive sensing with cross-validation and stop-sampling for sparse polynomial chaos expansions","volume":"6","author":"Huan","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"114147","DOI":"10.1016\/j.cma.2021.114147","article-title":"Bayesian learning of orthogonal embeddings for multi-fidelity Gaussian Processes","volume":"386","author":"Tsilifis","year":"2021","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4405","DOI":"10.1109\/TNNLS.2019.2957109","article-title":"When Gaussian process meets big data: A review of scalable GPs","volume":"31","author":"Liu","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_25","first-page":"14648","article-title":"Exact Gaussian processes on a million data points","volume":"32","author":"Wang","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Berns, F., and Beecks, C. (2020, January 7\u20139). Towards Large-scale Gaussian Process Models for Efficient Bayesian Machine Learning. Proceedings of the 9th International Conference on Data Science, Technology and Applications\u2014DATA, Paris, France.","DOI":"10.5220\/0009874702750282"},{"key":"ref_27","first-page":"031705","article-title":"srMO-BO-3GP: A sequential regularized multi-objective Bayesian optimization for constrained design applications using an uncertain Pareto classifier","volume":"144","author":"Tran","year":"2022","journal-title":"J. Mech. Des."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"030904","DOI":"10.1115\/1.4046747","article-title":"Advances in bayesian probabilistic modeling for industrial applications","volume":"6","author":"Ghosh","year":"2020","journal-title":"ASCE-ASME J. Risk Uncertain. Eng. Syst. Part B Mech. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"074502","DOI":"10.1115\/1.4050246","article-title":"Scalable Fully Bayesian Gaussian Process Modeling and Calibration with Adaptive Sequential Monte Carlo for Industrial Applications","volume":"143","author":"Pandita","year":"2021","journal-title":"J. Mech. Des."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.jcp.2015.10.008","article-title":"Dimension-independent likelihood-informed MCMC","volume":"304","author":"Cui","year":"2016","journal-title":"J. Comput. Phys."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1137\/17M1134640","article-title":"Transport map accelerated markov chain monte carlo","volume":"6","author":"Parno","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2321","DOI":"10.1007\/s10444-019-09711-y","article-title":"A transport-based multifidelity preconditioner for Markov chain Monte Carlo","volume":"45","author":"Peherstorfer","year":"2019","journal-title":"Adv. Comput. Math."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"7815","DOI":"10.1016\/j.jcp.2012.07.022","article-title":"Bayesian inference with optimal maps","volume":"231","author":"Marzouk","year":"2012","journal-title":"J. Comput. Phys."},{"key":"ref_34","unstructured":"Ranganath, R., Gerrish, S., and Blei, D. (2014, January 22\u201325). Black box variational inference. Proceedings of the Artificial Intelligence and Statistics, Reykjavic, Iceland."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_36","unstructured":"Titsias, M., and L\u00e1zaro-Gredilla, M. (2014, January 21\u201326). Doubly stochastic variational Bayes for non-conjugate inference. Proceedings of the International Conference on Machine Learning, Beijing, China."},{"key":"ref_37","first-page":"20180285","article-title":"Bayesian adaptation of chaos representations using variational inference and sampling on geodesics","volume":"474","author":"Tsilifis","year":"2018","journal-title":"Proc. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"109498","DOI":"10.1016\/j.jcp.2020.109498","article-title":"Sparse Polynomial Chaos expansions using variational relevance vector machines","volume":"416","author":"Tsilifis","year":"2020","journal-title":"J. Comput. Phys."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"031004","DOI":"10.1115\/1.4034102","article-title":"Computationally efficient variational approximations for Bayesian inverse problems","volume":"1","author":"Tsilifis","year":"2016","journal-title":"J. Verif. Valid. Uncertain. Quantif."},{"key":"ref_40","first-page":"2348","article-title":"Practical variational inference for neural networks","volume":"24","author":"Graves","year":"2011","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_41","unstructured":"Paisley, J., Blei, D.M., and Jordan, M.I. (July, January 26). Variational Bayesian inference with stochastic search. Proceedings of the 29th International Coference on International Conference on Machine Learning, Edinburgh, UK."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"121402","DOI":"10.1115\/1.4044396","article-title":"Computational creativity via assisted variational synthesis of mechanisms using deep generative models","volume":"141","author":"Deshpande","year":"2019","journal-title":"J. Mech. Des."},{"key":"ref_43","first-page":"111401","article-title":"Stochastic variational inference","volume":"14","author":"Hoffman","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"ref_44","unstructured":"Salimbeni, H., and Deisenroth, M. (2017). Doubly stochastic variational inference for deep Gaussian processes. arXiv."},{"key":"ref_45","unstructured":"Hoang, T., Hoang, Q., and Low, B. (2015, January 6\u201311). A unifying framework of anytime sparse Gaussian process regression models with stochastic variational inference for big data. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1111\/1467-9868.00294","article-title":"Bayesian calibration of computer models","volume":"63","author":"Kennedy","year":"2001","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/biomet\/87.1.1","article-title":"Predicting the output from a complex computer code when fast approximations are available","volume":"87","author":"Kennedy","year":"2000","journal-title":"Biometrika"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1615\/Int.J.UncertaintyQuantification.2014006914","article-title":"Recursive co-kriging model for design of computer experiments with multiple levels of fidelity","volume":"4","author":"Garnier","year":"2014","journal-title":"Int. J. Uncertain. Quantif."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1137\/120884122","article-title":"Bayesian analysis of hierarchical multifidelity codes","volume":"1","year":"2013","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"100909","DOI":"10.1115\/1.4007573","article-title":"Improving identifiability in model calibration using multiple responses","volume":"134","author":"Arendt","year":"2012","journal-title":"J. Mech. Des."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1080\/0740817X.2015.1064554","article-title":"A preposterior analysis to predict identifiability in the experimental calibration of computer models","volume":"48","author":"Arendt","year":"2016","journal-title":"IIE Trans."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1137\/151005841","article-title":"A theoretical framework for calibration in computer models: Parametrization, estimation and convergence properties","volume":"4","author":"Tuo","year":"2016","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_53","unstructured":"Hoffman, M., Bach, F., and Blei, D. (2010, January 6\u20139). Online learning for latent Dirichlet allocation. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_54","first-page":"1","article-title":"Graphical models, exponential families, and variational inference","volume":"1","author":"Wainwright","year":"2008","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_55","first-page":"1005","article-title":"Variational Inference in Nonconjugate Models","volume":"14","author":"Wang","year":"2013","journal-title":"J. Mach. Learn. Res."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","article-title":"Natural gradient works efficiently in learning","volume":"10","author":"Amari","year":"1998","journal-title":"Neural Comput."},{"key":"ref_57","first-page":"1","article-title":"New insights and perspectives on the natural gradient method","volume":"21","author":"Martens","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Absil, P., Mahony, R., and Sepulchre, R. (2009). Optimization Algorithms on Matrix Manifolds, Princeton University Press.","DOI":"10.1515\/9781400830244"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Roy, S., and Harandi, M. (December, January 29). Constrained stochastic gradient descent: The good practice. Proceedings of the 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Sydney, Australia.","DOI":"10.1109\/DICTA.2017.8227420"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Savva, F., Anagnostopoulos, C., and Triantafillou, P. (2018, January 10\u201313). Explaining aggregates for exploratory analytics. Proceedings of the 2018 IEEE International Conference on Big Data (Big Data), Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8621953"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"2546","DOI":"10.1007\/s10489-017-1093-y","article-title":"Scalable aggregation predictive analytics","volume":"48","author":"Anagnostopoulos","year":"2018","journal-title":"Appl. Intell."},{"key":"ref_62","unstructured":"GPy (2022, January 01). GPy: A Gaussian Process Framework in Python. Since 2012. Available online: http:\/\/github.com\/SheffieldML\/GPy."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1291\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:30:44Z","timestamp":1760142644000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1291"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,13]]},"references-count":62,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["e24091291"],"URL":"https:\/\/doi.org\/10.3390\/e24091291","relation":{"has-preprint":[{"id-type":"doi","id":"10.20944\/preprints202207.0034.v1","asserted-by":"object"}]},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,13]]}}}