{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T17:02:49Z","timestamp":1773334969253,"version":"3.50.1"},"reference-count":25,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T00:00:00Z","timestamp":1773100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004772","name":"Natural Science Foundation of Ningxia","doi-asserted-by":"crossref","award":["2025AAC030018"],"award-info":[{"award-number":["2025AAC030018"]}],"id":[{"id":"10.13039\/501100004772","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Social Science Fund of China","award":["1246010681"],"award-info":[{"award-number":["1246010681"]}]},{"name":"National Social Science Fund of China","award":["23BMZ062"],"award-info":[{"award-number":["23BMZ062"]}]},{"name":"North Minzu University Research Initiative","award":["2022ZLGTTYS12"],"award-info":[{"award-number":["2022ZLGTTYS12"]}]},{"name":"North Minzu University Research Initiative","award":["ZDZX201805"],"award-info":[{"award-number":["ZDZX201805"]}]},{"name":"First-Class Discipline Construction Program of Ningxia","award":["NXYLXK2017B09"],"award-info":[{"award-number":["NXYLXK2017B09"]}]},{"name":"Ningxia Youth Talent Support Program"},{"name":"Leading Talent Program of North Minzu University"},{"name":"Governance and Social Management Research Center of Northwest Ethnic Regions"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>Recently, the adaptive regularized proximal quasi-Newton (ARPQN) method has demonstrated a strong performance in solving composite optimization problems over the Stiefel manifold. However, its reliance on first-order information limits its applicability to scenarios where gradient and Hessian evaluations are unavailable or costly. In this paper, we propose a zeroth-order adaptive regularized proximal quasi-Newton method (ZO-ARPQN) for black-box composite optimization over Riemannian manifolds, particularly the Stiefel and symmetric positive definite (SPD) manifolds. The proposed method estimates the Riemannian gradient and curvature information through randomized one-point finite-difference approximations and adaptively updates a regularized quasi-Newton matrix to capture the local manifold geometry. Theoretically, we established global convergence and complex analyses under mild assumptions. More importantly, by incorporating curvature-aware regularization and random perturbations in the proximal quasi-Newton framework, we proved that ZO-ARPQN can escape strict saddle points with a high probability. This guarantees convergence to a stationary point, even in the absence of explicit gradients. Extensive numerical experiments were conducted on manifold-constrained problems, including sparse PCA and robot stiffness tuning. These demonstrated that ZO-ARPQN shows a competitive convergence behavior compared with other state-of-the-art Riemannian optimization methods, while requiring only function evaluations.<\/jats:p>","DOI":"10.3390\/axioms15030203","type":"journal-article","created":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T08:43:31Z","timestamp":1773132211000},"page":"203","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Zeroth-Order Riemannian Adaptive Regularized Proximal Quasi-Newton Optimization Method"],"prefix":"10.3390","volume":"15","author":[{"given":"Yinpu","family":"Ma","sequence":"first","affiliation":[{"name":"School of Mathematics and Information Science, North Minzu University, Yinchuan 750021, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cunlin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Mathematics and Information Science, North Minzu University, Yinchuan 750021, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhichao","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, WA 6102, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qian","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University, Perth, WA 6102, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1137\/110845768","article-title":"Low-rank matrix completion by Riemannian optimization","volume":"23","author":"Vandereycken","year":"2013","journal-title":"SIAM J. Optim."},{"key":"ref_2","first-page":"517","article-title":"Generalized power method for sparse principal component analysis","volume":"11","author":"Nesterov","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_3","unstructured":"Li, J., Ma, S., and Srivastava, T. (2022). A riemannian admm. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Absil, P.-A., Mahony, R., and Sepulchre, R. (2008). Optimization Algorithms on Matrix Manifolds, Princeton University Press.","DOI":"10.1515\/9781400830244"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Boumal, N. (2023). An Introduction to Optimization on Smooth Manifolds, Cambridge University Press.","DOI":"10.1017\/9781009166164"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1080\/02331930290019413","article-title":"Proximal point algorithm on Riemannian manifolds","volume":"51","author":"Ferreira","year":"2002","journal-title":"Optimization"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1137\/15M1052858","article-title":"A parallel Douglas\u2013Rachford algorithm for minimizing ROF-like functionals on images with values in symmetric Hadamard manifolds","volume":"9","author":"Bergmann","year":"2016","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kovnatsky, A., Glashoff, K., and Bronstein, M.M. (2016). MADMM: A Generic Algorithm for Non-Smooth Optimization on Manifolds. Computer Vision\u2014ECCV 2016, Springer.","DOI":"10.1007\/978-3-319-46454-1_41"},{"key":"ref_9","unstructured":"Reimherr, M., Bharath, K., and Soto, C. (2021, January 6\u201314). Differential privacy over riemannian manifolds. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Virtual."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1137\/21M1395648","article-title":"A variational formulation of accelerated optimization on Riemannian manifolds","volume":"4","author":"Duruisseaux","year":"2022","journal-title":"SIAM J. Math. Data Sci."},{"key":"ref_11","first-page":"1554","article-title":"Riemannian accelerated gradient methods via extrapolation","volume":"206","author":"Han","year":"2023","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_12","unstructured":"Zhang, H., Reddi, S.J., and Sra, S. (2016, January 5\u201310,). Riemannian SVRG: Fast stochastic optimization on Riemannian manifolds. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_13","unstructured":"Xu, Y., Jin, R., and Yang, T. (2017). NEON+: Accelerated gradient methods for extracting negative curvature for non-convex optimization. arXiv."},{"key":"ref_14","unstructured":"Jensen, R., and Zimmermann, R. (2024). Riemannian optimization on the symplectic Stiefel manifold using second-order information. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1137\/11082885X","article-title":"Optimization methods on Riemannian manifolds and their application to shape space","volume":"22","author":"Ring","year":"2012","journal-title":"SIAM J. Optim."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1007\/s10589-024-00595-3","article-title":"An adaptive regularized proximal Newton-type methods for composite optimization over the Stiefel manifold","volume":"89","author":"Wang","year":"2024","journal-title":"Comput. Optim. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Frazier, P.I. (2018). A tutorial on Bayesian optimization. arXiv.","DOI":"10.1287\/educ.2018.0188"},{"key":"ref_18","unstructured":"Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I. (2017). Evolution strategies as a scalable alternative to reinforcement learning. arXiv."},{"key":"ref_19","first-page":"138","article-title":"Faster first-order methods for stochastic non-convex optimization on Riemannian manifolds","volume":"89","author":"Zhou","year":"2019","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1287\/moor.2022.1302","article-title":"Stochastic zeroth-order Riemannian derivative estimation and optimization","volume":"48","author":"Li","year":"2023","journal-title":"Math. Oper. Res."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1007\/s10208-021-09499-8","article-title":"Zeroth-order nonconvex stochastic optimization: Handling constraints, high dimensionality, and saddle points","volume":"22","author":"Balasubramanian","year":"2022","journal-title":"Found. Comput. Math."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1007\/s10915-023-02165-x","article-title":"Proximal quasi-Newton method for composite optimization over the Stiefel manifold","volume":"95","author":"Wang","year":"2023","journal-title":"J. Sci. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1137\/18M122457X","article-title":"Proximal gradient method for nonsmooth optimization over the Stiefel manifold","volume":"30","author":"Chen","year":"2020","journal-title":"SIAM J. Optim."},{"key":"ref_24","unstructured":"Li, J., Balasubramanian, K., and Ma, S. (2020). Zeroth-order optimization on Riemannian manifolds. arXiv."},{"key":"ref_25","first-page":"233","article-title":"Bayesian optimization meets Riemannian manifolds in robot learning","volume":"100","author":"Jaquier","year":"2020","journal-title":"Proc. Mach. Learn. Res."}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/15\/3\/203\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T05:23:10Z","timestamp":1773292990000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/15\/3\/203"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,10]]},"references-count":25,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["axioms15030203"],"URL":"https:\/\/doi.org\/10.3390\/axioms15030203","relation":{},"ISSN":["2075-1680"],"issn-type":[{"value":"2075-1680","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,10]]}}}