{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T01:53:52Z","timestamp":1773107632000,"version":"3.50.1"},"reference-count":75,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2024,9,6]],"date-time":"2024-09-06T00:00:00Z","timestamp":1725580800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Exascale Computing Project","award":["17-SC-20-SC"],"award-info":[{"award-number":["17-SC-20-SC"]}]},{"name":"National Energy Research Scientific Computing Center NERSC","award":["DE-AC02-05CH11231"],"award-info":[{"award-number":["DE-AC02-05CH11231"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:p> Tuning algorithmic parameters to optimize the performance of large, complicated computational codes is an important problem involving finding the optima and identifying regimes defined by non-smooth boundaries in black-box functions. Within the Bayesian optimization framework, the Gaussian process surrogate model produces smooth mean functions, but functions in the tuning problem are often non-smooth, which is exacerbated by the fact that we usually have limited sequential samples from the black-box function. Motivated by these issues encountered in tuning, we propose a novel Gaussian process model called a clustered Gaussian process (cGP), where the components are dynamically updated by clustering. In our studies, the performance of cGP can be better than stationary GPs in nearly 90% of the experiments and better than non-stationary GPs in nearly 70% of the repeated experiments while requiring less computational cost. cGP provides a novel approach for dynamic GP, computes more efficiently than recursive partitioning, and discovers non-smoothness regimes. We provide extensive experiments including high-performance computing (HPC) and industrial simulation functions to show the effectiveness of our methods. <\/jats:p>","DOI":"10.1177\/10943420241278981","type":"journal-article","created":{"date-parts":[[2024,9,7]],"date-time":"2024-09-07T05:11:38Z","timestamp":1725685898000},"page":"633-657","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["Non-smooth Bayesian optimization in tuning scientific applications"],"prefix":"10.1177","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9254-8342","authenticated-orcid":false,"given":"Hengrui","family":"Luo","sequence":"first","affiliation":[{"name":"Lawrence Berkeley National Laboratory, Berkeley, CA, USA"},{"name":"Department of Statistics, Rice University, Houston, TX, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6774-7668","authenticated-orcid":false,"given":"Younghyun","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of EECS, University of California, Berkeley, Berkeley, CA, USA"}]},{"given":"James W","family":"Demmel","sequence":"additional","affiliation":[{"name":"Department of EECS, University of California, Berkeley, Berkeley, CA, USA"}]},{"given":"Igor","family":"Kozachenko","sequence":"additional","affiliation":[{"name":"Department of EECS, University of California, Berkeley, Berkeley, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0747-698X","authenticated-orcid":false,"given":"Xiaoye S","family":"Li","sequence":"additional","affiliation":[{"name":"Lawrence Berkeley National Laboratory, Berkeley, CA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3750-1178","authenticated-orcid":false,"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Lawrence Berkeley National Laboratory, Berkeley, CA, USA"}]}],"member":"179","published-online":{"date-parts":[[2024,9,6]]},"reference":[{"key":"bibr1-10943420241278981","unstructured":"Adams RP, MacKay DJC (2007) Bayesian online changepoint detection. arXiv:0710.3742."},{"key":"bibr2-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2015.04.003"},{"key":"bibr3-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1006\/jcom.2001.0588"},{"key":"bibr4-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1993.10594323"},{"key":"bibr5-10943420241278981","volume-title":"Prentice Hall Information and System Sciences Series","author":"Basseville M","year":"1993"},{"key":"bibr6-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49956-7_2"},{"key":"bibr7-10943420241278981","first-page":"281","volume":"13","author":"Bergstra J","year":"2012","journal-title":"Journal of Machine Learning Research"},{"key":"bibr8-10943420241278981","unstructured":"Bhatia N (2010) Survey of nearest neighbor techniques. arXiv:1007.0085."},{"key":"bibr9-10943420241278981","first-page":"253","volume-title":"ACM International Conference on Supercomputing 25th Anniversary Volume","author":"Bilmes J","year":"1997"},{"key":"bibr10-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/567806.567807"},{"key":"bibr11-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/BF01197708"},{"key":"bibr12-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-10745-0_10"},{"issue":"10","key":"bibr13-10943420241278981","first-page":"1","volume":"12","author":"Bull AD","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"bibr14-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/2503210.2503277"},{"key":"bibr15-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1998.10473750"},{"key":"bibr16-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1214\/09-AOAS285"},{"key":"bibr17-10943420241278981","unstructured":"Cho Y, Demmel JW, Derezi\u0144ski M, et al. (2023a) Surrogate-based autotuning for randomized sketching algorithms in regression problems. ArXiv Preprint arXiv:2308.15720."},{"key":"bibr18-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS54959.2023.00069"},{"key":"bibr19-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8191(00)00087-9"},{"key":"bibr20-10943420241278981","volume-title":"Dover Books on Mathematics","author":"Cram\u00e9r H","year":"2013"},{"key":"bibr21-10943420241278981","volume-title":"A Probabilistic Theory of Pattern Recognition","volume":"31","author":"Devroye L","year":"2013"},{"key":"bibr22-10943420241278981","doi-asserted-by":"crossref","unstructured":"Fabregat-Traver D, Ismail AE, Bientinesi P (2018). Accelerating molecular dynamics codes by performance and accuracy modeling. Journal of Computational Science 27(2018): 77-90.","DOI":"10.1016\/j.jocs.2018.05.004"},{"key":"bibr23-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/1089014.1089018"},{"key":"bibr24-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-6226-1"},{"key":"bibr25-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1201\/9780367815493"},{"key":"bibr26-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2014.914442"},{"key":"bibr27-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1198\/016214508000000689"},{"key":"bibr28-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1137\/140980089"},{"key":"bibr29-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1198\/jcgs.2010.09171"},{"key":"bibr30-10943420241278981","unstructured":"Head T, MechCoder GL, Shcherbatyi I, et al. (2018) scikit-optimize\/scikit-optimize: v0. 5.2. Version v0 5."},{"key":"bibr31-10943420241278981","first-page":"1013","volume-title":"Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research","volume":"51","author":"Herlands W","year":"2016"},{"key":"bibr32-10943420241278981","first-page":"1","volume":"27","author":"Hern\u00e1ndez-Lobato JM","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"bibr33-10943420241278981","volume-title":"Proceedings of the Thirteenth Annual ACM Symposium on Theory of Computing - STOC \u201981","author":"Hong JW","year":"1981"},{"key":"bibr34-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1162\/evco.2007.15.1.1"},{"key":"bibr35-10943420241278981","first-page":"1","volume":"30","author":"Ke G","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"bibr36-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1002\/9781118763667"},{"key":"bibr37-10943420241278981","unstructured":"Klus J, Grunt P, Dobrovoln\u00fd M (2021) Hyper-optimization with Gaussian process and differential evolution algorithm."},{"key":"bibr38-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273553"},{"key":"bibr39-10943420241278981","unstructured":"Lamparth M, Bestehorn M, M\u00e4rkisch B (2022) Gaussian processes and Bayesian optimization for high precision experiments. ArXiv Preprint arXiv:2205.07625."},{"key":"bibr40-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/3577197"},{"issue":"1","key":"bibr41-10943420241278981","first-page":"2475","volume":"23","author":"Lindauer M","year":"2022","journal-title":"Journal of Machine Learning Research"},{"key":"bibr42-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2510000"},{"key":"bibr43-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/3437801.3441621"},{"key":"bibr44-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1109\/BigData52589.2021.9672024"},{"key":"bibr45-10943420241278981","doi-asserted-by":"crossref","unstructured":"Luo H, MacEachern SN, Peruggia M (2020) Asymptotics of lower dimensional zero-density regions. Statistics 57.6(2023): 1285-1316.","DOI":"10.1080\/02331888.2023.2262665"},{"issue":"1","key":"bibr46-10943420241278981","first-page":"1","volume":"23","author":"Luo H","year":"2022","journal-title":"Journal of Machine Learning Research"},{"key":"bibr47-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2024.2308216"},{"key":"bibr48-10943420241278981","first-page":"19274","volume":"34","author":"Maddox WJ","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"bibr49-10943420241278981","first-page":"4","volume":"7","author":"Martinez-Cantin R","year":"2015","journal-title":"Neural Information Processing Systems (NIPS) workshop on Bayesian Optimization"},{"key":"bibr50-10943420241278981","unstructured":"NERSC (2024) Perlmutter architecture. https:\/\/docs.nersc.gov\/systems\/perlmutter\/architecture."},{"key":"bibr51-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1163\/016918609X12529286896877"},{"key":"bibr52-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1063\/5.0176963"},{"key":"bibr53-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1093\/imaiai\/iau001"},{"key":"bibr54-10943420241278981","first-page":"273","volume-title":"Advances in Neural Information Processing Systems 16","author":"Paciorek CJ","year":"2004"},{"key":"bibr55-10943420241278981","first-page":"95","volume-title":"Proceedings of 2nd Asian Conference on Machine Learning, Proceedings of Machine Learning Research","volume":"13","author":"Park S","year":"2010"},{"key":"bibr56-10943420241278981","first-page":"1697","volume":"11","author":"Park C","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"bibr57-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-17248-4_10"},{"key":"bibr58-10943420241278981","doi-asserted-by":"crossref","unstructured":"Raponi E, Wang H, Bujny M, et al. (2020) High dimensional Bayesian optimization assisted by principal component analysis.","DOI":"10.1007\/978-3-030-58112-1_12"},{"key":"bibr59-10943420241278981","volume-title":"Gaussian Processes for Machine Learning","author":"Rasmussen CE","year":"2006"},{"key":"bibr60-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-012-9951-y"},{"key":"bibr61-10943420241278981","first-page":"927","volume-title":"Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML\u201910","author":"Saat\u00e7i Y","year":"2010"},{"key":"bibr62-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-021-00204-y"},{"key":"bibr63-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2494218"},{"key":"bibr64-10943420241278981","volume-title":"Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI)","author":"Slonim N","year":"2013"},{"key":"bibr65-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/62.2.407"},{"key":"bibr66-10943420241278981","volume-title":"Advances in Neural Information Processing Systems","volume":"25","author":"Snoek J","year":"2012"},{"key":"bibr67-10943420241278981","first-page":"1674","volume-title":"International Conference on Machine Learning","author":"Snoek J","year":"2014"},{"key":"bibr68-10943420241278981","volume-title":"Advances in Neural Information Processing Systems","volume":"15","author":"Solak E","year":"2003"},{"key":"bibr69-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-75426-0_8"},{"key":"bibr70-10943420241278981","doi-asserted-by":"publisher","DOI":"10.2514\/6.2017-1326"},{"key":"bibr71-10943420241278981","first-page":"1","volume":"22","author":"Sun D","year":"2021","journal-title":"Journal of Machine Learning Research"},{"key":"bibr72-10943420241278981","author":"Sung CL","year":"2019","journal-title":"Statistica Sinica"},{"key":"bibr73-10943420241278981","doi-asserted-by":"crossref","unstructured":"Teh YW (2010) Dirichlet process.","DOI":"10.1007\/978-0-387-30164-8_219"},{"key":"bibr74-10943420241278981","doi-asserted-by":"publisher","DOI":"10.1145\/2755561"},{"issue":"123","key":"bibr75-10943420241278981","first-page":"1","volume":"22","author":"Wynne G","year":"2021","journal-title":"Journal of Machine Learning Research"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420241278981","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10943420241278981","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10943420241278981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,27]],"date-time":"2025-02-27T18:25:10Z","timestamp":1740680710000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10943420241278981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,6]]},"references-count":75,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10.1177\/10943420241278981"],"URL":"https:\/\/doi.org\/10.1177\/10943420241278981","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,6]]}}}