{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:23:01Z","timestamp":1760145781332,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T00:00:00Z","timestamp":1724371200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["11671205"],"award-info":[{"award-number":["11671205"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>In this paper, we propose a modified RFedSVRG method by incorporating the Barzilai\u2013Borwein (BB) method to approximate second-order information on the manifold for Federated Learning (FL). Moreover, we use the BB strategy to obtain self-adjustment of step size. We show the convergence of our methods under some assumptions. The numerical experiments on both synthetic and real datasets demonstrate that the proposed methods outperform some used methods in FL in some test problems.<\/jats:p>","DOI":"10.3390\/sym16091101","type":"journal-article","created":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T12:58:07Z","timestamp":1724417887000},"page":"1101","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Riemannian SVRG Using Barzilai\u2013Borwein Method as Second-Order Approximation for Federated Learning"],"prefix":"10.3390","volume":"16","author":[{"given":"He","family":"Xiao","sequence":"first","affiliation":[{"name":"School of Mathematics and Statistics, Nanjing University of Science and Technology, Nanjing 210094, China"}]},{"given":"Tao","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Mathematics and Statistics, Nanjing University of Science and Technology, Nanjing 210094, China"}]},{"given":"Kai","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mathematics and Statistics, Nanjing University of Science and Technology, Nanjing 210094, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,23]]},"reference":[{"key":"ref_1","first-page":"1273","article-title":"Communication-Efficient Learning of Deep Networks from Decentralized Data","volume":"Volume 54","author":"McMahan","year":"2017","journal-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics"},{"key":"ref_2","unstructured":"Konecn\u00fd, J., McMahan, H., Ramage, D., and Richt\u00e1rik, P. (2016). Federated Optimization: Distributed Machine Learning for On-Device Intelligence. arXiv."},{"key":"ref_3","first-page":"213","article-title":"Federated Learning on Riemannian Manifolds","volume":"5","author":"Li","year":"2023","journal-title":"Appl. Set-Valued Anal. Optim."},{"key":"ref_4","first-page":"1617","article-title":"First-order Methods for Geodesically Convex Optimization","volume":"Volume 49","author":"Zhang","year":"2016","journal-title":"Proceedings of the 29th Annual Conference on Learning Theory"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"H\u00e4rdle, W., and Simar, L. (2019). Applied Multivariate Statistical Analysis, Springer.","DOI":"10.1007\/978-3-030-26006-4"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cheung, Y., Lou, J., and Yu, F. (2021). Vertical Federated Principal Component Analysis on Feature-Wise Distributed Data. Web Information Systems Engineering, Proceedings of the 22nd International Conference on Web Information Systems Engineering, WISE 2021, Melbourne, VIC, Australia, 26\u201329 October 2021, WISE.","DOI":"10.1007\/978-3-030-90888-1_14"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.laa.2015.02.027","article-title":"Low-rank matrix completion via preconditioned optimization on the Grassmann manifold","volume":"475","author":"Boumal","year":"2015","journal-title":"Linear Algebra Its Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1007\/s11263-005-3222-z","article-title":"A Riemannian Framework for Tensor Computing","volume":"66","author":"Pennec","year":"2005","journal-title":"Int. J. Comput. Vis."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.sigpro.2005.12.018","article-title":"Riemannian geometry for the statistical analysis of diffusion tensor data","volume":"87","author":"Fletcher","year":"2007","journal-title":"Signal Process."},{"key":"ref_10","unstructured":"Rentmeesters, Q., and Absil, P. (September, January 29). Algorithm comparison for Karcher mean computation of rotation matrices and diffusion tensors. Proceedings of the 19th European Signal Processing Conference, Barcelona, Spain."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1023\/A:1007335407097","article-title":"Averaging anisotropic elastic constant data","volume":"46","author":"Cowi","year":"1997","journal-title":"J. Elast."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Massart, E., and Chevallier, S. (2017). Inductive Means and Sequences Applied to Online Classification of EEG. Geometric Science of Information, Proceedings of the Third International Conference, GSI 2017, Paris, France, 7\u20139 November 2017, Springer International Publishing.","DOI":"10.1007\/978-3-319-68445-1_88"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Magai, G. (2023, January 18\u201320). Deep Neural Networks Architectures from the Perspective of Manifold Learning. Proceedings of the 2023 IEEE 6th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), Haikou, China.","DOI":"10.1109\/PRAI59366.2023.10331992"},{"key":"ref_14","first-page":"24103","article-title":"Learning Efficient Coding of Natural Images with Maximum Manifold Capacity Representations","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1137\/18M122457X","article-title":"Proximal Gradient Method for Nonsmooth Optimization over the Stiefel Manifold","volume":"30","author":"Chen","year":"2020","journal-title":"SIAM J. Optim."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Boumal, N. (2022). An Introduction to Optimization on Smooth Manifolds, Cambridge University Press.","DOI":"10.1017\/9781009166164"},{"key":"ref_17","first-page":"4599","article-title":"Riemannian svrg: Fast stochastic optimization on riemannian manifolds","volume":"29","author":"Zhang","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_18","first-page":"429","article-title":"Federated Optimization in Heterogeneous Networks","volume":"2","author":"Li","year":"2020","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_19","first-page":"7057","article-title":"FedSplit: An algorithmic framework for fast federated optimization","volume":"33","author":"Pathak","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","unstructured":"Wang, J., Liu, Q., Liang, H., Joshi, G., and Poor, H.V. (2020, January 6\u201312). Tackling the objective inconsistency problem in heterogeneous federated optimization. Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS \u201920, Vancouver, BC, Canada."},{"key":"ref_21","first-page":"14606","article-title":"Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients","volume":"34","author":"Mitra","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","unstructured":"Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S.J., Stich, S.U., and Suresh, A.T. (2020, January 13\u201318). SCAFFOLD: Stochastic controlled averaging for federated learning. Proceedings of the 37th International Conference on Machine Learning, ICML\u201920, Virtual Event."},{"key":"ref_23","first-page":"12253","article-title":"Federated Composite Optimization","volume":"Volume 139","author":"Meila","year":"2021","journal-title":"Proceedings of the Machine Learning Research, 38th International Conference on Machine Learning"},{"key":"ref_24","first-page":"1508","article-title":"Fast Composite Optimization and Statistical Recovery in Federated Learning","volume":"Volume 162","author":"Chaudhuri","year":"2022","journal-title":"Proceedings of the Machine Learning Research, 39th International Conference on Machine Learning"},{"key":"ref_25","first-page":"30326","article-title":"FedDR\u2013Randomized Douglas-Rachford Splitting Algorithms for Nonconvex Federated Composite Optimization","volume":"34","author":"Pham","year":"2021","journal-title":"Neural Inf. Process. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, J., Hu, J., and Johansson, M. (2023, January 14\u201319). Composite Federated Learning with Heterogeneous Data. Proceedings of the ICASSP 2024\u20142024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.","DOI":"10.1109\/ICASSP48485.2024.10447718"},{"key":"ref_27","unstructured":"Zhang, J., Hu, J., So, A.M., and Johansson, M. (2024). Nonconvex Federated Learning on Compact Smooth Submanifolds With Heterogeneous Data. arXiv."},{"key":"ref_28","first-page":"2575","article-title":"Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learning","volume":"Volume 130","author":"Banerjee","year":"2021","journal-title":"Proceedings of the 24th International Conference on Artificial Intelligence and Statistics"},{"key":"ref_29","first-page":"315","article-title":"Accelerating stochastic gradient descent using predictive variance reduction","volume":"Volume 1","author":"Johnson","year":"2013","journal-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems, NIPS\u201913"},{"key":"ref_30","unstructured":"Huang, Z., Huang, W., Jawanpuria, P., and Mishra, B. (2024). Federated Learning on Riemannian Manifolds with Differential Privacy. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nguyen, T.A., Le, L.T., Nguyen, T.D., Bao, W., Seneviratne, S., Hong, C.S., and Tran, N.H. (2024). Federated PCA on Grassmann Manifold for IoT Anomaly Detection. IEEE\/ACM Trans. Netw., 1\u201316.","DOI":"10.1109\/TNET.2024.3423780"},{"key":"ref_32","first-page":"707","article-title":"Tracking the gradients using the Hessian: A new look at variance reducing stochastic methods","volume":"Volume 84","author":"Storkey","year":"2018","journal-title":"Proceedings of the Machine Learning Research, Twenty-First International Conference on Artificial Intelligence and Statistics"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"525","DOI":"10.3934\/jimo.2023089","article-title":"A stochastic variance reduced gradient using Barzilai-Borwein techniques as second order information","volume":"20","author":"Tankaria","year":"2024","journal-title":"J. Ind. Manag. Optim."},{"key":"ref_34","first-page":"2663","article-title":"A Stochastic Gradient Method with an Exponential Convergence Rate for Finite Training Sets","volume":"Volume 2","author":"Roux","year":"2012","journal-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS\u201912"},{"key":"ref_35","unstructured":"Tan, C., Ma, S., Dai, Y., and Qian, Y. (2016, January 5\u201310). Barzilai-Borwein step size for stochastic gradient descent. Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS\u201916, Barcelona, Spain."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1093\/imanum\/8.1.141","article-title":"Two-Point Step Size Gradient Methods","volume":"8","author":"Barzilai","year":"1988","journal-title":"IMA J. Numer. Anal."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2717","DOI":"10.1016\/j.cam.2012.01.014","article-title":"Nonmonotone algorithm for minimization on closed sets with applications to minimization on Stiefel manifolds","volume":"236","author":"Francisco","year":"2012","journal-title":"J. Comput. Appl. Math."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1007\/s10107-014-0816-7","article-title":"A framework of constraint preserving update schemes for optimization on Stiefel manifold","volume":"153","author":"Jiang","year":"2013","journal-title":"Math. Program."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1007\/s10107-012-0584-1","article-title":"A feasible method for optimization with orthogonality constraints","volume":"142","author":"Wen","year":"2013","journal-title":"Math. Program."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1093\/imanum\/drx015","article-title":"The Riemannian Barzilai\u2013Borwein method with nonmonotone line search and the matrix geometric mean computation","volume":"38","author":"Iannazzo","year":"2017","journal-title":"IMA J. Numer. Anal."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Absil, P., Mahony, R., and Sepulchre, R. (2007). Optimization Algorithms on Matrix Manifolds, Princeton University Press.","DOI":"10.1515\/9781400830244"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Lee, J.M. (2018). Introduction to Riemannian Manifolds, Springer International Publishing. [2nd ed.].","DOI":"10.1007\/978-3-319-91755-9_3"},{"key":"ref_43","unstructured":"Petersen, P. (2006). Riemannian Geometry, Springer."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tu, L. (2011). An Introduction to Manifolds, Springer.","DOI":"10.1007\/978-1-4419-7400-6"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Nocedal, J., and Wright, S. (1999). Numerical Optimization, Springer.","DOI":"10.1007\/b98874"},{"key":"ref_46","first-page":"4755","article-title":"Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation","volume":"17","author":"Townsend","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1137\/100802529","article-title":"Projection-like Retractions on Matrix Manifolds","volume":"22","author":"Absil","year":"2012","journal-title":"SIAM J. Optim."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1109\/TSP.2012.2226167","article-title":"Empirical Arithmetic Averaging Over the Compact Stiefel Manifold","volume":"61","author":"Kaneko","year":"2013","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1515\/jnum-2013-0013","article-title":"Angles between subspaces and their tangents","volume":"21","author":"Zhu","year":"2013","journal-title":"J. Numer. Math."},{"key":"ref_50","first-page":"191","article-title":"PARVUS: An Extendable Package of Programs for Data Exploration","volume":"4","author":"Forina","year":"1990","journal-title":"J. Chemom."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1117\/12.148698","article-title":"Nuclear feature extraction for breast tumor diagnosis","volume":"Volume 1905","author":"Acharya","year":"1993","journal-title":"Proceedings of the Biomedical Image Processing and Biomedical Visualization"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/9\/1101\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:42:23Z","timestamp":1760110943000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/9\/1101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,23]]},"references-count":52,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["sym16091101"],"URL":"https:\/\/doi.org\/10.3390\/sym16091101","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2024,8,23]]}}}