{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T14:50:29Z","timestamp":1774363829736,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T00:00:00Z","timestamp":1676419200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["1654579"],"award-info":[{"award-number":["1654579"]}]},{"name":"NSF","award":["2113642"],"award-info":[{"award-number":["2113642"]}]},{"name":"NSF","award":["EP\/W021595\/1"],"award-info":[{"award-number":["EP\/W021595\/1"]}]},{"name":"NSF","award":["EP\/X5257161\/1"],"award-info":[{"award-number":["EP\/X5257161\/1"]}]},{"name":"EPSRC","award":["1654579"],"award-info":[{"award-number":["1654579"]}]},{"name":"EPSRC","award":["2113642"],"award-info":[{"award-number":["2113642"]}]},{"name":"EPSRC","award":["EP\/W021595\/1"],"award-info":[{"award-number":["EP\/W021595\/1"]}]},{"name":"EPSRC","award":["EP\/X5257161\/1"],"award-info":[{"award-number":["EP\/X5257161\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes and utilizing the uncertainty in that surrogate by deriving an acquisition function. This acquisition function represents the probability of improvement based on the kernel of the Gaussian process, which guides the search in the optimization process. The critical challenge for designing Bayesian optimization algorithms on manifolds lies in the difficulty of constructing valid covariance kernels for Gaussian processes on general manifolds. Our approach is to employ extrinsic Gaussian processes by first embedding the manifold onto some higher dimensional Euclidean space via equivariant embeddings and then constructing a valid covariance kernel on the image manifold after the embedding. This leads to efficient and scalable algorithms for optimization over complex manifolds. Simulation study and real data analyses are carried out to demonstrate the utilities of our eBO framework by applying the eBO to various optimization problems over manifolds such as the sphere, the Grassmannian, and the manifold of positive definite matrices.<\/jats:p>","DOI":"10.3390\/a16020117","type":"journal-article","created":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T03:37:24Z","timestamp":1676432244000},"page":"117","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Extrinsic Bayesian Optimization on Manifolds"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4576-367X","authenticated-orcid":false,"given":"Yihao","family":"Fang","sequence":"first","affiliation":[{"name":"Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA"}]},{"given":"Mu","family":"Niu","sequence":"additional","affiliation":[{"name":"School of Mathematics and Statistics, University of Glasgow, Glasgow G12 8QQ, UK"}]},{"given":"Pokman","family":"Cheung","sequence":"additional","affiliation":[{"name":"London, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7913-2780","authenticated-orcid":false,"given":"Lizhen","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, Notre Dame, IN 46556, USA"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1016\/j.nurt.2007.05.011","article-title":"Diffusion Tensor Imaging of the Brain","volume":"4","author":"Alexander","year":"2007","journal-title":"Neurotherapeutics"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"428","DOI":"10.2307\/1426091","article-title":"The diffusion of shape","volume":"9","author":"Kendall","year":"1977","journal-title":"Adv. Appl. Probab."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Nishimori, Y., Akaho, S., and Plumbley, M. (2008, January 3\u20136). Natural Conjugate Gradient on Complex Flag Manifolds for Complex Independent Subspace Analysis. Proceedings of the ICANN 2008: 18th International Conference, Prague, Czech Republic. Part I.","DOI":"10.1007\/978-3-540-87536-9_18"},{"key":"ref_4","unstructured":"St. Thomas, B., Lin, L., Lim, L., and Mukherjee, S. (2014). Learning subspaces of different dimension. arXiv."},{"key":"ref_5","first-page":"216","article-title":"Statistical methods for vectorcardiogram orientations","volume":"2","author":"Downs","year":"1971","journal-title":"Vectorcardiography"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bhattacharya, A., and Bhattacharya, R. (2012). Nonparametric Inference on Manifolds: With Applications to Shape Spaces, Cambridge University Press.","DOI":"10.1017\/CBO9781139094764"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1090\/proc\/13216","article-title":"Omnibus CLTs for Fr\u00e9chet means and nonparametric inference on non-Euclidean spaces","volume":"145","author":"Bhattacharya","year":"2017","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_8","first-page":"215","article-title":"Les \u00e9l\u00e9ments al\u00e9atoires de nature quelconque dans un espace distanci\u00e9","volume":"10","year":"1948","journal-title":"Annales De L\u2019Institut Henri Poincar\u00e9"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1137\/S0036144500378648","article-title":"A Grassmann\u2013Rayleigh quotient iteration for computing invariant subspaces","volume":"44","author":"Absil","year":"2002","journal-title":"SIAM Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1023\/B:ACAP.0000013855.14971.91","article-title":"Riemannian geometry of Grassmann manifolds with a view on algorithmic computation","volume":"80","author":"Absil","year":"2004","journal-title":"Acta Appl. Math."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Absil, P., Mahony, R., and Sepulchre, R. (2009). Optimization Algorithms On Matrix Manifolds, Princeton University Press.","DOI":"10.1515\/9781400830244"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2284","DOI":"10.3182\/20110828-6-IT-1002.00542","article-title":"A discrete regression method on manifolds and its application to data on SO (n)","volume":"44","author":"Boumal","year":"2011","journal-title":"IFAC Proc. Vol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.laa.2015.02.027","article-title":"Low-rank matrix completion via preconditioned optimization on the Grassmann manifold","volume":"475","author":"Boumal","year":"2015","journal-title":"Linear Algebra Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"S440","DOI":"10.1137\/15M1025153","article-title":"Robust low-rank matrix completion by Riemannian optimization","volume":"38","author":"Cambier","year":"2016","journal-title":"SIAM J. Sci. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"A54","DOI":"10.1051\/0004-6361\/201527387","article-title":"Low-rank plus sparse decomposition for exoplanet detection in direct-imaging ADI sequences-The LLSG algorithm","volume":"589","author":"Gonzalez","year":"2016","journal-title":"Astron. Astrophys."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2327","DOI":"10.1137\/080731359","article-title":"Low-rank optimization on the cone of positive semidefinite matrices","volume":"20","author":"Bach","year":"2010","journal-title":"SIAM J. Optim."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1137\/18M1231389","article-title":"Quotient geometry with simple geodesics for the manifold of fixed-rank positive-semidefinite matrices","volume":"41","author":"Massart","year":"2020","journal-title":"SIAM J. Matrix Anal. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.neucom.2014.02.018","article-title":"Two algorithms for orthogonal nonnegative matrix factorization with application to clustering","volume":"141","author":"Pompili","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_19","unstructured":"Theis, F., Cason, T., and Absil, P. (2009). International Conference on Independent Component Analysis And Signal Separation, Proceedings of the 8th International Conference, ICA 2009, Springer."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Vandereycken, B., Absil, P., and Vandewalle, S. (September, January 31). Embedded geometry of the set of symmetric positive semidefinite matrices of fixed rank. Proceedings of the 2009 IEEE\/SP 15th Workshop on Statistical Signal Processing, Cardiff, UK.","DOI":"10.1109\/SSP.2009.5278558"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1093\/imanum\/drs006","article-title":"A Riemannian geometry with complete geodesics for the set of positive semidefinite matrices of fixed rank","volume":"33","author":"Vandereycken","year":"2013","journal-title":"IMA J. Numer. Anal."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1137\/11082885X","article-title":"Optimization Methods on Riemannian Manifolds and Their Application to Shape Space","volume":"22","author":"Ring","year":"2012","journal-title":"SIAM J. Optim."},{"key":"ref_23","unstructured":"Smith, S. (2014). Optimization Techniques on Riemannian Manifolds. arXiv."},{"key":"ref_24","unstructured":"Absil, P., Mahony, R., and Sepulchre, R. (2010). Recent Advances in Optimization and its Applications in Engineering, Proceedings of the 14th Belgian-French-German Conference on Optimization, Springer."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1007\/s10208-005-0179-9","article-title":"Trust-Region Methods on Riemannian Manifolds","volume":"7","author":"Absil","year":"2007","journal-title":"Found. Comput. Math."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1093\/imanum\/drn029","article-title":"An implicit trust-region method on Riemannian manifolds","volume":"28","author":"Baker","year":"2008","journal-title":"IMA J. Numer. Anal."},{"key":"ref_27","first-page":"1","article-title":"RTRMC: A Riemannian trust-region method for low-rank matrix completion","volume":"24","author":"Boumal","year":"2011","journal-title":"Adv. Neural Inf. Process. Syst.."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1137\/090764827","article-title":"Best low multilinear rank approximation of higher-order tensors, based on the Riemannian trust-region scheme","volume":"32","author":"Ishteva","year":"2011","journal-title":"SIAM J. Matrix Anal. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1137\/040605266","article-title":"Convergence of the iterates of descent methods for analytic cost functions","volume":"16","author":"Absil","year":"2005","journal-title":"SIAM J. Optim."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/imanum\/drx080","article-title":"Global rates of convergence for nonconvex optimization on manifolds","volume":"39","author":"Boumal","year":"2019","journal-title":"IMA J. Numer. Anal."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Lin, L., Saparbayeva, B., Zhang, M., and Dunson, D. (2020). Accelerated Algorithms for Convex and Non-Convex Optimization on Manifolds. arXiv.","DOI":"10.1007\/978-981-15-2910-8_3"},{"key":"ref_32","unstructured":"Qi, C., Gallivan, K., and Absil, P. (2010). Recent Advances in Optimization and its Applications in Engineering, Proceedings of the 14th Belgian-French-German Conference on Optimization, Springer."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1007\/s10208-011-9091-7","article-title":"A gradient-descent method for curve fitting on Riemannian manifolds","volume":"12","author":"Samir","year":"2012","journal-title":"Found. Comput. Math."},{"key":"ref_34","first-page":"1","article-title":"Practical bayesian optimization of machine learning algorithms","volume":"25","author":"Snoek","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1023\/A:1012771025575","article-title":"A Taxonomy of Global Optimization Methods Based on Response Surfaces","volume":"21","author":"Jones","year":"2001","journal-title":"J. Glob. Optim."},{"key":"ref_36","first-page":"97","article-title":"A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise","volume":"86","author":"Kushner","year":"1964","journal-title":"J. Fluids Eng."},{"key":"ref_37","unstructured":"Mo\u010dkus, J. (1975). Optimization Techniques IFIP Technical Conference, Springer."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1023\/A:1008306431147","article-title":"Efficient global optimization of expensive black-box functions","volume":"13","author":"Jones","year":"1998","journal-title":"J. Glob. Optim."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s00158-005-0587-0","article-title":"Sequential kriging optimization using multiple-fidelity evaluations","volume":"32","author":"Huang","year":"2006","journal-title":"Struct. Multidiscip. Optim."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"879","DOI":"10.2514\/1.16875","article-title":"Statistical improvement criteria for use in multiobjective design optimization","volume":"44","author":"Keane","year":"2006","journal-title":"AIAA J."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1214\/aoap\/1034801250","article-title":"Average performance of a class of adaptive algorithms for global optimization","volume":"7","author":"Calvin","year":"1997","journal-title":"Ann. Appl. Probab."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Packwood, D. (2017). Bayesian Optimization for Materials Science, Springer.","DOI":"10.1007\/978-981-10-6781-5"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1623\/hysj.52.3.450","article-title":"Watershed calibration using multistart local optimization and evolutionary optimization with radial basis function approximation","volume":"52","author":"Shoemaker","year":"2007","journal-title":"Hydrol. Sci. J."},{"key":"ref_44","unstructured":"Brochu, E., Cora, V., and De Freitas, N. (2010). A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1214\/18-BA1135","article-title":"Extrinsic Gaussian processes for regression and classification on manifolds","volume":"14","author":"Lin","year":"2019","journal-title":"Bayesian Anal."},{"key":"ref_46","first-page":"20939","article-title":"High-dimensional Bayesian optimization via nested Riemannian manifolds","volume":"33","author":"Jaquier","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_47","first-page":"12426","article-title":"Mat\u00e9rn Gaussian processes on Riemannian manifolds","volume":"33","author":"Borovitskiy","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_48","unstructured":"Jaquier, N., Borovitskiy, V., Smolensky, A., Terenin, A., Asfour, T., and Rozo, L. (2022, January 14\u201318). Geometry-aware Bayesian Optimization in Robotics using Riemannian Mat\u00e9rn Kernels. Proceedings of the Conference on Robot Learning, Auckland, NZ, USA."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Frazier, P. (2018). A tutorial on Bayesian optimization. arXiv.","DOI":"10.1287\/educ.2018.0188"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1111\/rssb.12320","article-title":"Intrinsic Gaussian processes on complex constrained domains","volume":"81","author":"Niu","year":"2019","journal-title":"J. R. Stat. Soc. Ser. B (Stat. Methodol.)"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/2\/117\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:36:12Z","timestamp":1760121372000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/2\/117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,15]]},"references-count":50,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["a16020117"],"URL":"https:\/\/doi.org\/10.3390\/a16020117","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,15]]}}}