{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T15:01:04Z","timestamp":1767193264547,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,8,9]],"date-time":"2018-08-09T00:00:00Z","timestamp":1533772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004602","name":"Program for New Century Excellent Talents in University","doi-asserted-by":"publisher","award":["NCET 10-0893"],"award-info":[{"award-number":["NCET 10-0893"]}],"id":[{"id":"10.13039\/501100004602","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11771450","61573367"],"award-info":[{"award-number":["11771450","61573367"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Bayesian optimization (BO) based on the Gaussian process (GP) surrogate model has attracted extensive attention in the field of optimization and design of experiments (DoE). It usually faces two problems: the unstable GP prediction due to the ill-conditioned Gram matrix of the kernel and the difficulty of determining the trade-off parameter between exploitation and exploration. To solve these problems, we investigate the K-optimality, aiming at minimizing the condition number. Firstly, the Sequentially Bayesian K-optimal design (SBKO) is proposed to ensure the stability of the GP prediction, where the K-optimality is given as the acquisition function. We show that the SBKO reduces the integrated posterior variance and maximizes the hyper-parameters\u2019 information gain simultaneously. Secondly, a K-optimal enhanced Bayesian Optimization (KO-BO) approach is given for the optimization problems, where the K-optimality is used to define the trade-off balance parameters which can be output automatically. Specifically, we focus our study on the K-optimal enhanced Expected Improvement algorithm (KO-EI). Numerical examples show that the SBKO generally outperforms the Monte Carlo, Latin hypercube sampling, and sequential DoE approaches by maximizing the posterior variance with the highest precision of prediction. Furthermore, the study of the optimization problem shows that the KO-EI method beats the classical EI method due to its higher convergence rate and smaller variance.<\/jats:p>","DOI":"10.3390\/e20080594","type":"journal-article","created":{"date-parts":[[2018,8,9]],"date-time":"2018-08-09T10:36:31Z","timestamp":1533810991000},"page":"594","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Bayesian Optimization Based on K-Optimality"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2168-5470","authenticated-orcid":false,"given":"Liang","family":"Yan","sequence":"first","affiliation":[{"name":"College of Liberal Arts and Sciences, National University of Defense Technology, Changsha 410000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaojun","family":"Duan","sequence":"additional","affiliation":[{"name":"College of Liberal Arts and Sciences, National University of Defense Technology, Changsha 410000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bowen","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Liberal Arts and Sciences, National University of Defense Technology, Changsha 410000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Xu","sequence":"additional","affiliation":[{"name":"College of Liberal Arts and Sciences, National University of Defense Technology, Changsha 410000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,9]]},"reference":[{"key":"ref_1","unstructured":"Gould, H., Tobochnik, J., and Christian, W. (1988). An Introduction to Computer Simulation Methods, Addison-Wesley."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1007\/BF01532020","article-title":"Computer simulation of liquid crystals","volume":"3","author":"Allen","year":"1989","journal-title":"J. Comput. Aided Mol. Des."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Binder, K. (1986). Introduction: Theory and \u201ctechnical\u201d aspects of Monte Carlo simulations. Monte Carlo Methods in Statistical Physics, Springer.","DOI":"10.1007\/978-3-642-82803-4"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/BF02736747","article-title":"Monte Carlo techniques in computational stochastic mechanics","volume":"5","author":"Hurtado","year":"1998","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0962492900002804","article-title":"Monte carlo and quasi-monte carlo methods","volume":"7","author":"Caflisch","year":"1998","journal-title":"Acta Numer."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/S0951-8320(03)00058-9","article-title":"Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems","volume":"81","author":"Helton","year":"2003","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Papalambros, P.Y., and Wilde, D.J. (2000). Principles of Optimal Design: Modeling and Computation, Cambridge University Press.","DOI":"10.1017\/CBO9780511626418"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1020281327116","article-title":"An introduction to MCMC for machine learning","volume":"50","author":"Andrieu","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rasmussen, C.E. (2004). Gaussian processes in machine learning. Advanced Lectures on Machine Learning, Springer.","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"ref_10","unstructured":"Brochu, E., Cora, V.M., and De Freitas, N. (arXiv, 2010). A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1016\/j.ress.2008.07.007","article-title":"Hierarchical adaptive experimental design for Gaussian process emulators","volume":"94","author":"Busby","year":"2009","journal-title":"Reliab. Eng. Syst. Saf."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.strusafe.2011.01.002","article-title":"AK-MCS: An active learning reliability method combining Kriging and Monte Carlo simulation","volume":"33","author":"Echard","year":"2011","journal-title":"Struct. Saf."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1137\/110850268","article-title":"Minimizing the condition number to construct design points for polynomial regression models","volume":"23","author":"Ye","year":"2013","journal-title":"SIAM J. Optim."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.jspi.2017.02.003","article-title":"K-optimal designs for parameters of shifted Ornstein-Uhlenbeck processes and sheets","volume":"186","author":"Baran","year":"2017","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1007\/BF00941892","article-title":"Lipschitzian optimization without the Lipschitz constant","volume":"79","author":"Jones","year":"1993","journal-title":"J. Optim. Theory Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1137\/080740544","article-title":"Optimizing condition numbers","volume":"20","author":"Ye","year":"2009","journal-title":"SIAM J. Optim."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1137\/100786022","article-title":"Minimizing the condition number of a Gram matrix","volume":"21","author":"Chen","year":"2011","journal-title":"SIAM J. Optim."},{"key":"ref_18","first-page":"2951","article-title":"Practical bayesian optimization of machine learning algorithms","volume":"2","author":"Snoek","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Preuss, R., and von Toussaint, U. (2017). Sequential Batch Design for Gaussian Processes Employing Marginalization. Entropy, 19.","DOI":"10.3390\/e19020084"},{"key":"ref_20","first-page":"1732","article-title":"Slice sampling covariance hyperparameters of latent Gaussian models","volume":"2","author":"Murray","year":"2010","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1090\/S0002-9947-1950-0051437-7","article-title":"Theory of reproducing kernels","volume":"68","author":"Aronszajn","year":"1950","journal-title":"Trans. Am. Math. Soc."},{"key":"ref_22","unstructured":"Kullback, S. (1997). Information Theory and Statistics, Courier Corporation."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2465","DOI":"10.1162\/NECO_a_00654","article-title":"A hierarchical adaptive approach to optimal experimental design","volume":"26","author":"Kim","year":"2014","journal-title":"Neural Comput."},{"key":"ref_24","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1137\/15M1053529","article-title":"Universal prediction distribution for surrogate models","volume":"5","author":"Roustant","year":"2017","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1007\/s00158-013-0919-4","article-title":"A benchmark of kriging-based infill criteria for noisy optimization","volume":"48","author":"Picheny","year":"2013","journal-title":"Struct. Multidiscip. Optim."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1080\/00401706.2012.723572","article-title":"Sequential design and analysis of high-accuracy and low-accuracy computer codes","volume":"55","author":"Xiong","year":"2013","journal-title":"Technometrics"},{"key":"ref_28","unstructured":"Rasmussen, C., and Williams, C. (2015, October 25). GPML: Matlab Implementation of Gaussian Process Regression and Classification, 2007. Available online: http:\/\/www.GaussianProcess.org\/gpml\/code."},{"key":"ref_29","unstructured":"Johnson, S.G. (2016, August 13). The NLopt Nonlinear-Optimization Package, 2014. Available online: http:\/\/ab-initio.mit.edu\/nlopt."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/594\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:17:41Z","timestamp":1760195861000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,9]]},"references-count":29,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["e20080594"],"URL":"https:\/\/doi.org\/10.3390\/e20080594","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,8,9]]}}}