{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T08:12:27Z","timestamp":1766391147942,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,12,31]],"date-time":"2020-12-31T00:00:00Z","timestamp":1609372800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003629","name":"Korea Meteorological Administration","doi-asserted-by":"publisher","award":["KMA2018-00124"],"award-info":[{"award-number":["KMA2018-00124"]}],"id":[{"id":"10.13039\/501100003629","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2016R1A2B4014518","2020R1I1A3069260"],"award-info":[{"award-number":["2016R1A2B4014518","2020R1I1A3069260"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"name":"BK21 Plus, Ministry of Education, Korea","award":["21A20132212094"],"award-info":[{"award-number":["21A20132212094"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The approximated non-linear least squares (ALS) tunes or calibrates the computer model by minimizing the squared error between the computer output and real observations by using an emulator such as a Gaussian process (GP) model. A potential defect of the ALS method is that the emulator is constructed once and it is no longer re-built. An iterative method is proposed in this study to address this difficulty. In the proposed method, the tuning parameters of the simulation model are calculated by the conditional expectation (E-step), whereas the GP parameters are updated by the maximum likelihood estimation (M-step). These EM-steps are alternately repeated until convergence by using both computer and experimental data. For comparative purposes, another iterative method (the max-min algorithm) and a likelihood-based method are considered. Five toy models are tested for a comparative analysis of these methods. According to the toy model study, both the variance and bias of the estimates obtained from the proposed EM algorithm are smaller than those from the existing calibration methods. Finally, the application to a nuclear fusion simulator is demonstrated.<\/jats:p>","DOI":"10.3390\/e23010053","type":"journal-article","created":{"date-parts":[[2020,12,31]],"date-time":"2020-12-31T10:10:37Z","timestamp":1609409437000},"page":"53","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Expectation-Maximization Algorithm for the Calibration of Complex Simulator Using a Gaussian Process Emulator"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9283-4376","authenticated-orcid":false,"given":"Yun Am","family":"Seo","sequence":"first","affiliation":[{"name":"AI Weather Forecast Research Team, National Institute of Meteorological Sciences, Seogwipo 697-010, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8460-4869","authenticated-orcid":false,"given":"Jeong-Soo","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Statistics, Chonnam National University, Gwangju 61186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/S0167-9473(00)00057-8","article-title":"A statistical method for tuning a computer code to a data base","volume":"37","author":"Cox","year":"2001","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1111\/1467-9868.00294","article-title":"Bayesian Calibration of Computer Models","volume":"63","author":"Kennedy","year":"2001","journal-title":"J. R. Stat. Ser. B"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1198\/TECH.2009.08126","article-title":"Simultaneous Determination of Tuning and Calibration Parameters for Computer Experiments","volume":"51","author":"Han","year":"2009","journal-title":"Technometrics"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Seo, Y.A., Lee, Y., and Park, J.S. (2020). Iterative method for tuning complex simulation code. Commun. Stat-Simul. Comput.","DOI":"10.1080\/03610918.2020.1728317"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1016\/j.ress.2005.11.031","article-title":"Calibration, validation, and sensitivity analysis: What\u2019s what","volume":"91","author":"Trucano","year":"2006","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lee, Y., and Park, J.-S. (2020). Generalized Nonlinear Least Squares Method for the Calibration of Complex Computer Code Using a Gaussian Process Surrogate. Entropy, 22.","DOI":"10.3390\/e22090985"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1214\/15-AOAS850","article-title":"Calibrating a large computer experiment simulating radiative shock hydrodynamics","volume":"9","author":"Gramacy","year":"2015","journal-title":"Ann. Appl. Stat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1111\/rssb.12182","article-title":"A frequentist approach to computer model calibration","volume":"79","author":"Wong","year":"2017","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1080\/00401706.2013.838910","article-title":"Prediction and Computer Model Calibration Using Outputs From Multifidelity Simulators","volume":"55","author":"Goh","year":"2013","journal-title":"Technometrics"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1137\/15M1033162","article-title":"Adaptive numerical designs for the calibration of computer codes","volume":"6","author":"Damblin","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1080\/00949655.2013.823965","article-title":"Sequential tuning of complex computer models","volume":"85","author":"Kumar","year":"2015","journal-title":"J. Stat. Comput. Simul."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1198\/016214507000000888","article-title":"Computer Model Calibration Using High Dimensional Output","volume":"103","author":"Higdon","year":"2008","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_13","first-page":"743","article-title":"Prediction based om the Kennedy-O\u2019Hagan calibration model: Asymptotic consistency and other properties","volume":"28","author":"Tuo","year":"2018","journal-title":"Stat. Sin."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1080\/01621459.2016.1211016","article-title":"Bayesian calibration of inexact computer models","volume":"112","author":"Plumlee","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1555","DOI":"10.1137\/17M1159890","article-title":"Scaled Gaussian stochastic process for computer model calibration and prediction","volume":"6","author":"Gu","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1080\/00401706.2013.775897","article-title":"Fast sequential computer model calibration of large nonstationary spatial-temporal processes","volume":"55","author":"Pratola","year":"2013","journal-title":"Technometrics"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Pandit, R., and Infield, D. (2018). Gaussian Process Operational Curves for Wind Turbine Condition Monitoring. Energies, 11.","DOI":"10.3390\/en11071631"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Albert, C.G. (2019). Gaussian Processes for Data Fulfilling Linear Differential Equations. Proceedings, 33.","DOI":"10.3390\/proceedings2019033005"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Molada-Tebar, A., Riutort-Mayol, G., Marqu\u00e9s-Mateu, \u00c1., and Lerma, J.L. (2019). A Gaussian Process Model for Color Camera Characterization: Assessment in Outdoor Levantine Rock Art Scenes. Sensors, 19.","DOI":"10.3390\/s19214610"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kim, S., Jung, S., and Baek, S.-M. (2019). A Model for Predicting Energy Usage Pattern Types with Energy Consumption Information According to the Behaviors of Single-Person Households in South Korea. Sustainability, 11.","DOI":"10.3390\/su11010245"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Oladyshkin, S., Mohammadi, F., Kroeker, I., and Nowak, W. (2020). Bayesian3 Active Learning for the Gaussian Process Emulator Using Information Theory. Entropy, 22.","DOI":"10.3390\/e22080890"},{"key":"ref_22","first-page":"409","article-title":"Design and analysis of computer experiment (with discussion)","volume":"4","author":"Sacks","year":"1989","journal-title":"Stat. Sci."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liao, Y., Xie, J., Wang, Z., and Shen, X. (2019). Multisensor Estimation Fusion with Gaussian Process for Nonlinear Dynamic Systems. Entropy, 21.","DOI":"10.3390\/e21111126"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mahmoodian, M., Torres-Matallana, J.A., Leopold, U., Schutz, G., and Clemens, F.H.L.R. (2018). A Data-Driven Surrogate Modelling Approach for Acceleration of Short-Term Simulations of a Dynamic Urban Drainage Simulator. Water, 10.","DOI":"10.3390\/w10121849"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/j.csda.2019.05.006","article-title":"Emulating dynamic non-linear simulators using Gaussian processes","volume":"139","author":"Mohammadi","year":"2019","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Santner, T.J., Williams, B., and Notz, W. (2018). The Design and Analysis of Computer Experiments, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4939-8847-1"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"54","DOI":"10.18637\/jss.v051.i01","article-title":"DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization","volume":"51","author":"Roustant","year":"2012","journal-title":"J. Stat. Softw."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"15","DOI":"10.2307\/1269548","article-title":"Screening, predicting, and computer experiments","volume":"34","author":"Welch","year":"1992","journal-title":"Technometrics"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4731","DOI":"10.1016\/j.csda.2008.03.026","article-title":"An efficient methodology for modeling complex computer codes with Gaussian processes","volume":"52","author":"Marrel","year":"2008","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_30","first-page":"383","article-title":"Model selection algorithm in Gaussian process regression for computer experiments","volume":"24","author":"Lee","year":"2017","journal-title":"Commun. Stat. Appl. Methods"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gramacy, R.B. (2020). Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, CRC Press.","DOI":"10.1201\/9780367815493"},{"key":"ref_32","unstructured":"Wikipedia (2020, October 16). Expectation\u2013Maximization Algorithm. Available online: https:\/\/en.wikipedia.org\/wiki\/Expectation%E2%80%93maximization_algorithm."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum Likelihood from Incomplete Data via the EM Algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Ser. B."},{"key":"ref_34","unstructured":"Bilmes, J. (1997). A Gentle Tutorial of the EM Algorithm and Its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, University of Berkeley. Technical Report ICSI-TR-97-021."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1561\/2000000034","article-title":"Theory and Use of the EM Algorithm","volume":"4","author":"Gupta","year":"2010","journal-title":"Found. Trends Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"McLachlan, G.J., and Krishnan, T. (2008). The EM Algorithm and Extensions, Wiley. [2nd ed.].","DOI":"10.1002\/9780470191613"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/0771-050X(80)90039-X","article-title":"Remarks on algorithm 006: An Adaptive Algorithm for Numeric Integration over an N-Dimensional Rectangular Region","volume":"6","author":"Genz","year":"1980","journal-title":"J. Comp. Appl. Math"},{"key":"ref_38","unstructured":"Balasubramanian, N., Manuel, K., Steven, G., Thomas, H., Annie, B., Ki\u00ean, K., and Simen, G. (2020, October 13). Cubature: Adaptive Multivariate Integration over Hypercubes 2.0.4.1. Available online: Https:\/\/CRAN.R-project.org\/package=cubature."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/0378-3758(94)00035-T","article-title":"Exploratory designs for computational experiments","volume":"43","author":"Morris","year":"1995","journal-title":"J. Stat. Plan. Infer."},{"key":"ref_40","unstructured":"Carnell, R. (2020, October 13). lhs: Latin Hypercube Samples. R Package Version 0.16. Available online: Https:\/\/CRAN.R-project.org\/package=lhs."},{"key":"ref_41","unstructured":"Johnson, R.A., and Wichern, D.W. (2007). Applied Multivariate Statistical Analysis, Pearson Prentice Hall. [6th ed.]."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/0010-4655(88)90012-4","article-title":"BALDUR: A one-dimensional plasma transport code","volume":"49","author":"Singer","year":"1988","journal-title":"Comput. Phys. Commun."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1137\/140989613","article-title":"Sequential design with mutual information for computer experiments (MICE): Emulation of a tsunami model","volume":"4","author":"Beck","year":"2016","journal-title":"J. Uncertain. Quantif."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1002\/jcc.23475","article-title":"Calibration of Forcefields for Molecular Simulation: Sequential Design of Computer Experiments for Building Cost-Efficient Kriging Metamodels","volume":"35","author":"Cailliez","year":"2014","journal-title":"J. Comput. Chem."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/1\/53\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:48:18Z","timestamp":1760179698000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/1\/53"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,31]]},"references-count":44,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["e23010053"],"URL":"https:\/\/doi.org\/10.3390\/e23010053","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,12,31]]}}}