{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T03:38:58Z","timestamp":1773113938906,"version":"3.50.1"},"reference-count":38,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T00:00:00Z","timestamp":1599177600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2016R1A2B4014518"],"award-info":[{"award-number":["2016R1A2B4014518"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1I1A3069260"],"award-info":[{"award-number":["2020R1I1A3069260"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The approximated nonlinear least squares (ALS) method has been used for the estimation of unknown parameters in the complex computer code which is very time-consuming to execute. The ALS calibrates or tunes the computer code by minimizing the squared difference between real observations and computer output using a surrogate such as a Gaussian process model. When the differences (residuals) are correlated or heteroscedastic, the ALS may result in a distorted code tuning with a large variance of estimation. Another potential drawback of the ALS is that it does not take into account the uncertainty in the approximation of the computer model by a surrogate. To address these problems, we propose a generalized ALS (GALS) by constructing the covariance matrix of residuals. The inverse of the covariance matrix is multiplied to the residuals, and it is minimized with respect to the tuning parameters. In addition, we consider an iterative version for the GALS, which is called as the max-minG algorithm. In this algorithm, the parameters are re-estimated and updated by the maximum likelihood estimation and the GALS, by using both computer and experimental data repeatedly until convergence. Moreover, the iteratively re-weighted ALS method (IRWALS) was considered for a comparison purpose. Five test functions in different conditions are examined for a comparative analysis of the four methods. Based on the test function study, we find that both the bias and variance of estimates obtained from the proposed methods (the GALS and the max-minG) are smaller than those from the ALS and the IRWALS methods. Especially, the max-minG works better than others including the GALS for the relatively complex test functions. Lastly, an application to a nuclear fusion simulator is illustrated and it is shown that the abnormal pattern of residuals in the ALS can be resolved by the proposed methods.<\/jats:p>","DOI":"10.3390\/e22090985","type":"journal-article","created":{"date-parts":[[2020,9,4]],"date-time":"2020-09-04T11:24:24Z","timestamp":1599218664000},"page":"985","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Generalized Nonlinear Least Squares Method for the Calibration of Complex Computer Code Using a Gaussian Process Surrogate"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7457-2579","authenticated-orcid":false,"given":"Youngsaeng","family":"Lee","sequence":"first","affiliation":[{"name":"Data Science Lab, Korea Electric Power Corporation, Seoul 60732, Korea"},{"name":"Department of Statistics, Chonnam National University, Gwangju 61186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8460-4869","authenticated-orcid":false,"given":"Jeong-Soo","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Statistics, Chonnam National University, Gwangju 61186, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,4]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1198\/TECH.2009.08126","article-title":"Simultaneous Determination of Tuning and Calibration Parameters for Computer Experiments","volume":"51","author":"Han","year":"2009","journal-title":"Technometrics"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Seo, Y.A., Lee, Y., and Park, J.S. (2020). Iterative method for tuning complex simulation code. Comm. Stat-Simul. Comput.","DOI":"10.1080\/03610918.2020.1728317"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/S0167-9473(00)00057-8","article-title":"A statistical method for tuning a computer code to a data base","volume":"37","author":"Cox","year":"2001","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1111\/1467-9868.00294","article-title":"Bayesian Calibration of Computer Models","volume":"63","author":"Kennedy","year":"2001","journal-title":"J. R. Stat. Ser. B"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1016\/j.ress.2005.11.031","article-title":"Calibration, validation, and sensitivity analysis: What\u2019s what","volume":"91","author":"Trucano","year":"2006","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_6","first-page":"409","article-title":"Design and analysis computer experiment (with discussion)","volume":"4","author":"Sacks","year":"1989","journal-title":"Stat. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1080\/00401706.1989.10488474","article-title":"Designs for computer experiments","volume":"31","author":"Sacks","year":"1989","journal-title":"Technometrics"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yan, L., Duan, X., Liu, B., and Xu, J. (2018). Bayesian optimization based on K-optimality. Entropy, 20.","DOI":"10.3390\/e20080594"},{"key":"ref_9","unstructured":"Loeppky, D., Bingham, D., and Welch, W. (2006). Computer Model Calibration or Tuning in Practice, Technical Report; University of British Columbia."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1214\/15-AOAS850","article-title":"Calibrating a large computer experiment simulating radiative shock hydrodynamics","volume":"9","author":"Gramacy","year":"2015","journal-title":"Ann. Appl. Stat."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1111\/rssb.12182","article-title":"A frequentist approach to computer model calibration","volume":"79","author":"Wong","year":"2017","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1080\/00401706.2013.838910","article-title":"Prediction and Computer Model Calibration Using Outputs From Multifidelity Simulators","volume":"55","author":"Goh","year":"2013","journal-title":"Technometrics"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1137\/15M1033162","article-title":"Adaptive numerical designs for the calibration of computer codes","volume":"6","author":"Damblin","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1198\/016214507000000888","article-title":"Computer Model Calibration Using High Dimensional Output","volume":"103","author":"Higdon","year":"2008","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1080\/00949655.2013.823965","article-title":"Sequential tuning of complex computer models","volume":"85","author":"Kumar","year":"2015","journal-title":"J. Stat. Comput. Simul."},{"key":"ref_16","first-page":"743","article-title":"Prediction based om the Kennedy-O\u2019Hagan calibration model: Asymptotic consistency and other properties","volume":"28","author":"Tuo","year":"2018","journal-title":"Stat. Sin."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1080\/01621459.2016.1211016","article-title":"Bayesian calibration of inexact computer models","volume":"112","author":"Plumlee","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Majda, A.J., and Chen, N. (2018). Model error, information barriers, state estimation and prediction in complex multiscale systems. Entropy, 20.","DOI":"10.3390\/e20090644"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1555","DOI":"10.1137\/17M1159890","article-title":"Scaled Gaussian stochstic process for computer model calibration and prediction","volume":"6","author":"Gu","year":"2018","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1080\/00401706.2013.775897","article-title":"Fast sequential computer model calibration of large nonstationary spatial-temporal processes","volume":"55","author":"Pratola","year":"2013","journal-title":"Technometrics"},{"key":"ref_21","unstructured":"Gujarati, D.N. (2009). Basic Econometrics, Tata McGraw-Hill Education."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1080\/01621459.1975.10479877","article-title":"Estimating heteroscedastic variances in linear models","volume":"70","author":"Horn","year":"1975","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Santner, T.J., Williams, B., and Notz, W. (2018). The Design and Analysis of Computer Experiments, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4939-8847-1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"54","DOI":"10.18637\/jss.v051.i01","article-title":"DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization","volume":"51","author":"Roustant","year":"2012","journal-title":"J. Stat. Softw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"15","DOI":"10.2307\/1269548","article-title":"Screening, predicting, and computer experiments","volume":"34","author":"Welch","year":"1992","journal-title":"Technometrics"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4731","DOI":"10.1016\/j.csda.2008.03.026","article-title":"An efficient methodology for modeling complex computer codes with Gaussian processes","volume":"52","author":"Marrel","year":"2008","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_27","first-page":"383","article-title":"Model selection algorithm in Gaussian process regression for computer experiments","volume":"24","author":"Lee","year":"2017","journal-title":"Commun. Stat. Appl. Methods"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Gramacy, R.B. (2020). Surrogates: Gaussian Process Modeling, Design, and Optimization for the Applied Sciences, CRC Press.","DOI":"10.1201\/9780367815493"},{"key":"ref_29","unstructured":"(2017, December 20). The R Project for Statistical Computing. Available online: http:\/\/www.r-project.org."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1198\/TECH.2009.08019","article-title":"Diagnostics for GP emulators","volume":"51","author":"Bastos","year":"2009","journal-title":"Technometrics"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1016\/0378-3758(94)00035-T","article-title":"Exploratory designs for computational experiments","volume":"43","author":"Morris","year":"1995","journal-title":"J. Stat. Plan. Infer."},{"key":"ref_32","unstructured":"Carnell, R. (2020, July 03). lhs: Latin Hypercube Samples. R Package Version 0.16,. Available online: https:\/\/CRAN.R-project.org\/package=lhs."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Rubin, D.B. (2014). Iteratively reweighted least squares. Wiley Statsref.","DOI":"10.1002\/9781118445112.stat03199"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1016\/0010-4655(88)90012-4","article-title":"BALDUR: A one-dimensional plasma transport code","volume":"49","author":"Singer","year":"1988","journal-title":"Comput. Phys. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"739","DOI":"10.1137\/140989613","article-title":"Sequential design with mutual information for computer experiments (MICE): Emulation of a tsunami model","volume":"4","author":"Beck","year":"2016","journal-title":"J. Uncertain. Quantif."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1002\/jcc.23475","article-title":"Calibration of Forcefields for Molecular Simulation: Sequential Design of Computer Experiments for Building Cost-Efficient Kriging Metamodels","volume":"35","author":"Cailliez","year":"2014","journal-title":"J. Comput. Chem."},{"key":"ref_37","unstructured":"Liu, H., Cal, J., Wang, Y., and Ong, Y.S. (2018). Generalized robust Bayesian committee machine for large-scale Gaussian process regression. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Rao, C.R. (1973). Linear Statistical Inference and Its Applications, John Wiley. [2nd ed.].","DOI":"10.1002\/9780470316436"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/9\/985\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:06:44Z","timestamp":1760177204000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/9\/985"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,4]]},"references-count":38,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["e22090985"],"URL":"https:\/\/doi.org\/10.3390\/e22090985","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,4]]}}}