{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T15:04:02Z","timestamp":1773068642720,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T00:00:00Z","timestamp":1654732800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T00:00:00Z","timestamp":1654732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Adv. Model. and Simul. in Eng. Sci."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Most real optimization problems are defined over a mixed search space where the variables are both discrete and continuous. In engineering applications, the objective function is typically calculated with a numerically costly black-box simulation. General mixed and costly optimization problems are therefore of a great practical interest, yet their resolution remains in a large part an open scientific question. In this article, costly mixed problems are approached through Gaussian processes where the discrete variables are relaxed into continuous latent variables. The continuous space is more easily harvested by classical Bayesian optimization techniques than a mixed space would. Discrete variables are recovered either subsequently to the continuous optimization, or simultaneously with an additional continuous-discrete compatibility constraint that is handled with augmented Lagrangians. Several possible implementations of such Bayesian mixed optimizers are compared. In particular, the reformulation of the problem with continuous latent variables is put in competition with searches working directly in the mixed space. Among the algorithms involving latent variables and an augmented Lagrangian, a particular attention is devoted to the Lagrange multipliers for which a local and a global estimation techniques are studied. The comparisons are based on the repeated optimization of three analytical functions and a beam design problem.<\/jats:p>","DOI":"10.1186\/s40323-022-00218-8","type":"journal-article","created":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T20:47:49Z","timestamp":1654807669000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["A comparison of mixed-variables Bayesian optimization approaches"],"prefix":"10.1186","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1365-0259","authenticated-orcid":false,"given":"Jhouben","family":"Cuesta Ramirez","sequence":"first","affiliation":[]},{"given":"Rodolphe","family":"Le Riche","sequence":"additional","affiliation":[]},{"given":"Olivier","family":"Roustant","sequence":"additional","affiliation":[]},{"given":"Guillaume","family":"Perrin","sequence":"additional","affiliation":[]},{"given":"C\u00e9dric","family":"Durantin","sequence":"additional","affiliation":[]},{"given":"Alain","family":"Gli\u00e8re","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,6,9]]},"reference":[{"issue":"3","key":"218_CR1","doi-asserted-by":"publisher","first-page":"573","DOI":"10.1137\/S1052623499352024","volume":"11","author":"C Audet","year":"2001","unstructured":"Audet C, Dennis Jr John E. Pattern search algorithms for mixed variable programming. SIAM J Optim. 2001;11(3):573\u201394.","journal-title":"SIAM J Optim"},{"key":"218_CR2","doi-asserted-by":"crossref","unstructured":"Bartz-Beielstein T, Filipi\u010d B, Koro\u0161ec P, Talbi EG. High-Performance Simulation-Based Optimization. Studies in Computational Intelligence. Springer International Publishing, 2019. ISBN 9783030187644. https:\/\/books.google.fr\/books?id=8yGbDwAAQBAJ.","DOI":"10.1007\/978-3-030-18764-4"},{"key":"218_CR3","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.asoc.2017.01.039","volume":"55","author":"T Bartz-Beielstein","year":"2017","unstructured":"Bartz-Beielstein T, Zaefferer Martin. Model-based methods for continuous and discrete global optimization. Appl Soft Comput. 2017;55:154\u201367.","journal-title":"Appl Soft Comput"},{"key":"218_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1017\/S0962492913000032","volume":"22","author":"P Belotti","year":"2013","unstructured":"Belotti P, Kirches C, Leyffer S, Linderoth J, Luedtke J, Mahajan Ashutosh. Mixed-integer nonlinear optimization. Acta Numer. 2013;22:1\u2013131.","journal-title":"Acta Numer"},{"key":"218_CR5","doi-asserted-by":"crossref","unstructured":"Bischl B, Richter J, Bossek J, Horn D, Thomas J, Lang M. mlrMBO: A modular framework for model-based optimization of expensive black-box functions, 2018.","DOI":"10.32614\/CRAN.package.mlrMBO"},{"issue":"12","key":"218_CR6","doi-asserted-by":"publisher","first-page":"931","DOI":"10.1016\/S0307-904X(00)00026-3","volume":"24","author":"YJ Cao","year":"2000","unstructured":"Cao YJ, Jiang L, Wu QH. An evolutionary programming approach to mixed-variable optimization problems. Appl Math Model. 2000;24(12):931\u201342.","journal-title":"Appl Math Model"},{"key":"218_CR7","unstructured":"Deville Y, Ginsbourger D, Roustant O, Durrande N. kergp. https:\/\/cran.r-project.org\/package=kergp, 2017\u20132021."},{"key":"218_CR8","unstructured":"Emmerich Michael, Zhang A, Li R, Flesch I, Lucas Peter\u00a0J. Mixed-integer Bayesian optimization utilizing a-priori knowledge on parameter dependences. J Phys Chem A. 2008;65\u201372."},{"key":"218_CR9","unstructured":"Frazier Peter\u00a0I. A Tutorial on Bayesian Optimization. arXiv e-prints, page arXiv:1807.02811, July 2018."},{"issue":"5","key":"218_CR10","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/BF00927673","volume":"4","author":"Magnus R Hestenes","year":"1969","unstructured":"Hestenes Magnus R. Multiplier and gradient methods. J Optim Theory Appl. 1969;4(5):303\u201320.","journal-title":"J Optim Theory Appl"},{"key":"218_CR11","doi-asserted-by":"crossref","unstructured":"Hutter F, Hoos HH, Leyton-Brown K. Sequential model-based optimization for general algorithm configuration. In: International Conference on Learning and Intelligent Optimization, pp. 507\u2013523. Springer, 2011.","DOI":"10.1007\/978-3-642-25566-3_40"},{"issue":"4","key":"218_CR12","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1023\/A:1008306431147","volume":"13","author":"DR Jones","year":"1998","unstructured":"Jones DR, Schonlau M, Welch WJ. Efficient global optimization of expensive black-box functions. J Glob Optim. 1998;13(4):455\u201392. https:\/\/doi.org\/10.1023\/A:1008306431147 (ISSN 1573-2916).","journal-title":"J Glob Optim."},{"key":"218_CR13","doi-asserted-by":"crossref","unstructured":"Le Riche R, Fr\u00e9d\u00e9ric G. Dual evolutionary optimization. Lecture Notes in Computer Science, (2310): 281\u2013294. selected papers of the 5th Int. Evolution Artificielle Conf; 2002.","DOI":"10.1007\/3-540-46033-0_23"},{"key":"218_CR14","doi-asserted-by":"crossref","unstructured":"Le\u00a0Riche R, Picheny V. Revisiting bayesian optimization in the light of the coco benchmark. Struct MultiDiscip Optim. 2021. to appear.","DOI":"10.1007\/s00158-021-02977-1"},{"issue":"1","key":"218_CR15","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1162\/EVCO_a_00059","volume":"21","author":"R Li","year":"2013","unstructured":"Li R, Emmerich MTM, Eggermont J, B\u00e4ck T, Sch\u00fctz M, Dijkstra J, Reiber Johan HC. Mixed integer evolution strategies for parameter optimization. Evol Comput. 2013;21(1):29\u201364.","journal-title":"Evol Comput"},{"key":"218_CR16","doi-asserted-by":"publisher","unstructured":"Lin Y, Liu Y, Chen W-N, Zhang J. A hybrid differential evolution algorithm for mixed-variable optimization problems. Inform Sci. 2018;466:170\u2013188. ISSN 00200255. https:\/\/doi.org\/10.1016\/j.ins.2018.07.035. URL https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0020025516318163.","DOI":"10.1016\/j.ins.2018.07.035"},{"key":"218_CR17","unstructured":"Minoux M. Mathematical programming: theory and algorithms. Wiley: A Wiley-Interscience publication; 1986."},{"issue":"117\u2013129","key":"218_CR18","first-page":"2","volume":"2","author":"J Mockus","year":"1978","unstructured":"Mockus J, Tiesis V, Zilinskas Antanas. The application of Bayesian methods for seeking the extremum. Towards Glob Optim. 1978;2(117\u2013129):2.","journal-title":"Towards Glob Optim"},{"key":"218_CR19","unstructured":"Nocedal J, Wright SJ. Numerical optimization. Springer series in operations research. Springer, New York, 2nd edn, 2006. ISBN 978-0-387-30303-1. OCLC: ocm68629100."},{"key":"218_CR20","unstructured":"Ocenasek J, Schwarz J. Estimation of distribution algorithm for mixed continuous-discrete optimization problems. In: 2nd Euro-International Symposium on Computational Intelligence. pp. 227\u2013232. IOS Press Kosice, Slovakia, 2002."},{"issue":"3","key":"218_CR21","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1007\/s10898-018-0715-1","volume":"73","author":"J Pelamatti","year":"2019","unstructured":"Pelamatti J, Brevault L, Balesdent M, Talbi E-G, Guerin Y. Efficient global optimization of constrained mixed variable problems. J Glob Optim. 2019;73(3):583\u2013613. https:\/\/doi.org\/10.1007\/s10898-018-0715-1 (ISSN 0925-5001, 1573-2916).","journal-title":"J Glob Optim."},{"key":"218_CR22","unstructured":"Picheny V, Gramacy RB, Wild S, Le\u00a0Digabel S. Bayesian optimization under mixed constraints with a slack-variable augmented Lagrangian. In Lee D, Sugiyama M, Luxburg, Guyon I, Garnett R. editors, Advances in Neural Information Processing Systems, volume\u00a029. Curran Associates, Inc., 2016. URL https:\/\/proceedings.neurips.cc\/paper\/2016\/file\/31839b036f63806cba3f47b93af8ccb5-Paper.pdf."},{"key":"218_CR23","doi-asserted-by":"publisher","unstructured":"Powell MJD. A direct search optimization method that models the objective and constraint functions by linear interpolation, pp. 51\u201367. Netherlands, Dordrecht: Springer; 1994. ISBN 978-94-015-8330-5. https:\/\/doi.org\/10.1007\/978-94-015-8330-5_4.","DOI":"10.1007\/978-94-015-8330-5_4"},{"key":"218_CR24","doi-asserted-by":"crossref","unstructured":"Rockafellar TR. Lagrange multipliers and optimality. SIAM Rev. 1993;35(2):183\u2013238. URL http:\/\/www.jstor.org\/stable\/2133143.","DOI":"10.1137\/1035044"},{"issue":"2","key":"218_CR25","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1137\/18M1209386","volume":"8","author":"O Roustant","year":"2020","unstructured":"Roustant O, Padonou E, Deville Y, Cl\u00e9ment A, Perrin G, Giorla J, Wynn Henry. Group kernels for gaussian process metamodels with categorical inputs. SIAM\/ASA J Uncertainty Quantif. 2020;8(2):775\u2013806. https:\/\/doi.org\/10.1137\/18M1209386.","journal-title":"SIAM\/ASA J Uncertainty Quantif"},{"key":"218_CR26","unstructured":"Thi HAL, Le HM, Dinh TP. Optimization of complex systems: theory, models, algorithms and applications. In: Advances in intelligent systems and computing. Springer International Publishing, 2019. ISBN 9783030218034. URL https:\/\/books.google.fr\/books?id=R46dDwAAQBAJ."},{"key":"218_CR27","doi-asserted-by":"publisher","unstructured":"Vazquez E, Bect J. Convergence properties of the expected improvement algorithm with fixed mean and covariance functions. J Statis Plan Inference, 2010;140(11):3088\u20133095. ISSN 03783758. https:\/\/doi.org\/10.1016\/j.jspi.2010.04.018. URL https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0378375810001850.","DOI":"10.1016\/j.jspi.2010.04.018"},{"key":"218_CR28","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1613\/jair.4806","volume":"55","author":"Z Wang","year":"2016","unstructured":"Wang Z, Hutter F, Zoghi M, Matheson D, de Feitas Nando. Bayesian optimization in a billion dimensions via random embeddings. J Artif Intell Res. 2016;55:361\u201387.","journal-title":"J Artif Intell Res"},{"key":"218_CR29","unstructured":"Wilson JT, Hutter F, Deisenroth MP. Maximizing acquisition functions for bayesian optimization. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS\u201918, page 9906-9917, Red Hook, NY, USA, 2018. Curran Associates Inc."},{"key":"218_CR30","unstructured":"Zaefferer M. CEGO. https:\/\/cran.r-project.org\/package=CEGO, 2014\u20132021."},{"key":"218_CR31","doi-asserted-by":"publisher","unstructured":"Zhang Y, Tao S, Chen W, Apley DW. A latent variable approach to Gaussian Process modeling with qualitative and quantitative factors. Technometrics, 2019;1\u201312. ISSN 0040-1706, 1537-2723. https:\/\/doi.org\/10.1080\/00401706.2019.1638834.","DOI":"10.1080\/00401706.2019.1638834"},{"key":"218_CR32","doi-asserted-by":"publisher","unstructured":"Zhang Y, Apley DW, Chen W. Bayesian optimization for materials design with mixed quantitative and qualitative variables. Sci Rep. 2020;10(1). ISSN 2045-2322. https:\/\/doi.org\/10.1038\/s41598-020-60652-9. URL http:\/\/www.nature.com\/articles\/s41598-020-60652-9.","DOI":"10.1038\/s41598-020-60652-9"},{"issue":"2","key":"218_CR33","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1080\/03155986.2020.1730677","volume":"58","author":"MM Zuniga","year":"2020","unstructured":"Zuniga MM, Sinoquet Delphine. Global optimization for mixed categorical-continuous variables based on gaussian process models with a randomized categorical space exploration step. INFOR Inform Syst Opera Res. 2020;58(2):310\u201341. https:\/\/doi.org\/10.1080\/03155986.2020.1730677.","journal-title":"INFOR Inform Syst Opera Res"}],"container-title":["Advanced Modeling and Simulation in Engineering Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40323-022-00218-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40323-022-00218-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40323-022-00218-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T18:42:44Z","timestamp":1727376164000},"score":1,"resource":{"primary":{"URL":"https:\/\/amses-journal.springeropen.com\/articles\/10.1186\/s40323-022-00218-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,9]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["218"],"URL":"https:\/\/doi.org\/10.1186\/s40323-022-00218-8","relation":{},"ISSN":["2213-7467"],"issn-type":[{"value":"2213-7467","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,9]]},"assertion":[{"value":"4 November 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 March 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 June 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"6"}}