{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T17:55:30Z","timestamp":1791395730960,"version":"4.3.3"},"reference-count":45,"publisher":"Institute of Mathematical Statistics","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2024,12,1]]},"abstract":"This paper considers decentralized Federated Learning (FL) under heterogeneous distributions among distributed clients or data blocks for the M-estimation. The mean squared error and consensus error across the estimators from different clients via the decentralized stochastic gradient descent algorithm are derived. The asymptotic normality of the Polyak\u2013Ruppert (PR) averaged estimator in the decentralized distributed setting is attained, which shows that its statistical efficiency comes at a cost as it is more restrictive on the number of clients than that in the distributed M-estimation. To overcome the restriction, a one-step estimator is proposed which permits a much larger number of clients while still achieving the same efficiency as the original PR-averaged estimator in the nondistributed setting. The confidence regions based on both the PR-averaged estimator and the proposed one-step estimator are constructed to facilitate statistical inference for decentralized FL.<\/jats:p>","DOI":"10.1214\/24-aos2452","type":"journal-article","created":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T01:24:59Z","timestamp":1734571499000},"page":"2931-2955","source":"Crossref","is-referenced-by-count":6,"title":["Statistical inference for decentralized federated learning"],"prefix":"10.1214","volume":"52","author":[{"given":"Jia","family":"Gu","sequence":"first","affiliation":[{"name":"Center for Data Science, Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Song Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, Tsinghua University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"6","doi-asserted-by":"publisher","unstructured":"BOTTOU, L.<\/small>, CURTIS, F. E.<\/small> and NOCEDAL, J.<\/small> (2018). Optimization methods for large-scale machine learning. SIAM Rev.<\/i> 60<\/b> 223\u2013311.","DOI":"10.1137\/16M1080173"},{"key":"44","unstructured":"Zhang, Y.<\/small>, Duchi, J. C.<\/small> and Wainwright, M. J.<\/small> (2013). Communication-efficient algorithms for statistical optimization. J. Mach. Learn. Res.<\/i> 14<\/b> 3321\u20133363."},{"key":"22","doi-asserted-by":"publisher","unstructured":"LAI, T. L.<\/small> (2003). Stochastic approximation. Ann. Statist.<\/i> 31<\/b> 391\u2013406.","DOI":"10.1214\/aos\/1051027873"},{"key":"32","doi-asserted-by":"publisher","unstructured":"Polyak, B. T.<\/small> and Juditsky, A. B.<\/small> (1992). Acceleration of stochastic approximation by averaging. SIAM J. Control Optim.<\/i> 30<\/b> 838\u2013855.","DOI":"10.1137\/0330046"},{"key":"34","doi-asserted-by":"publisher","unstructured":"Robbins, H.<\/small> and Monro, S.<\/small> (1951). A stochastic approximation method. Ann. Math. Stat.<\/i> 22<\/b> 400\u2013407.","DOI":"10.1214\/aoms\/1177729586"},{"key":"4","unstructured":"Bach, F.<\/small> (2014). Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression. J. Mach. Learn. Res.<\/i> 15<\/b> 595\u2013627."},{"key":"1","doi-asserted-by":"publisher","unstructured":"ALGHUNAIM, S. A.<\/small> and YUAN, K.<\/small> (2022). A unified and refined convergence analysis for non-convex decentralized learning. IEEE Trans. Signal Process.<\/i> 70<\/b> 3264\u20133279.","DOI":"10.1109\/tsp.2022.3184770"},{"key":"2","unstructured":"APPLE<\/small> (2019). Private federated learning. NeurIPS 2019 Expo Talk."},{"key":"3","doi-asserted-by":"publisher","unstructured":"BACH, F.<\/small> (2010). Self-concordant analysis for logistic regression. Electron. J. Stat.<\/i> 4<\/b> 384\u2013414.","DOI":"10.1214\/09-EJS521"},{"key":"5","doi-asserted-by":"crossref","unstructured":"BICKEL, P. J.<\/small> (1975). One-step Huber estimates in the linear model. J. Amer. Statist. Assoc.<\/i> 70<\/b> 428\u2013434.","DOI":"10.1080\/01621459.1975.10479884"},{"key":"7","doi-asserted-by":"publisher","unstructured":"BOYD, S.<\/small>, GHOSH, A.<\/small>, PRABHAKAR, B.<\/small> and SHAH, D.<\/small> (2006). Randomized gossip algorithms. IEEE Trans. Inf. Theory<\/i> 52<\/b> 2508\u20132530.","DOI":"10.1109\/TIT.2006.874516"},{"key":"8","doi-asserted-by":"publisher","unstructured":"CHEN, M.<\/small> and CUI, S.<\/small> (2024). Federated learning for autonomous vehicles control. In Communication Efficient Federated Learning for Wireless Networks<\/i> 129\u2013150. Springer, Switzerland.","DOI":"10.1007\/978-3-031-51266-7_6"},{"key":"9","doi-asserted-by":"publisher","unstructured":"CHEN, X.<\/small>, LAI, Z.<\/small>, LI, H.<\/small> et al. (2024). Online statistical inference for stochastic optimization via Kiefer\u2013Wolfowitz methods. J. Amer. Statist. Assoc.<\/i> 1\u201324.","DOI":"10.1080\/01621459.2023.2296703"},{"key":"10","doi-asserted-by":"publisher","unstructured":"CHEN, X.<\/small>, LEE, J. D.<\/small>, TONG, X. T.<\/small> and ZHANG, Y.<\/small> (2020a). Statistical inference for model parameters in stochastic gradient descent. Ann. Statist.<\/i> 48<\/b> 251\u2013273.","DOI":"10.1214\/18-AOS1801"},{"key":"11","doi-asserted-by":"publisher","unstructured":"CHEN, Y.<\/small>, QIN, X.<\/small>, WANG, J.<\/small> et al. (2020b). FedHealth: A federated transfer learning framework for wearable healthcare. IEEE Intell. Syst.<\/i> 35<\/b> 83\u201393.","DOI":"10.1109\/MIS.2020.2988604"},{"key":"12","unstructured":"CHOI, W.<\/small> and KIM, J.<\/small> (2022). On the convergence of decentralized gradient descent with diminishing stepsize. Revisited."},{"key":"13","doi-asserted-by":"publisher","unstructured":"CHUNG, K. L.<\/small> (1954). On a stochastic approximation method. Ann. Math. Stat.<\/i> 25<\/b> 463\u2013483.","DOI":"10.1214\/aoms\/1177728716"},{"key":"14","unstructured":"FANG, Y.<\/small>, XU, J.<\/small> and YANG, L.<\/small> (2018). Online bootstrap confidence intervals for the stochastic gradient descent estimator. J. Mach. Learn. Res.<\/i> 19<\/b> 1\u201321."},{"key":"15","unstructured":"GU, J.<\/small> and CHEN, S. X.<\/small> (2023). Distributed statistical inference under heterogeneity. J. Mach. Learn. Res.<\/i> 24<\/b> 1\u201356."},{"key":"16","doi-asserted-by":"crossref","unstructured":"GU, J.<\/small> and CHEN, S. X.<\/small> (2024). Supplement to \u201cStatistical Inference for Decentralized Federated Learning.\u201d https:\/\/doi.org\/10.1214\/24-AOS2452SUPP<\/a>","DOI":"10.1214\/24-AOS2452SUPP"},{"key":"17","unstructured":"HALL, P.<\/small> and HEYDE, C. C.<\/small> (1980). Martingale Limit Theory and Its Application<\/i>. Probability and Mathematical Statistics<\/i>. Academic Press, San Diego."},{"key":"18","unstructured":"HARD, A.<\/small>, RAO, K.<\/small>, MATHEWS, R.<\/small> et al. (2018). Federated learning for mobile keyboard prediction."},{"key":"19","doi-asserted-by":"crossref","unstructured":"JOHNSON, R.<\/small> and ZHANG, T.<\/small> (2013). Accelerating stochastic gradient descent using predictive variance reduction. Adv. Neural Inf. Process. Syst.<\/i> 1<\/b> 315\u2013323.","DOI":"10.1007\/BF00393811"},{"key":"20","doi-asserted-by":"publisher","unstructured":"KAIROUZ, P.<\/small>, MCMAHAN, H. B.<\/small>, AVENT, B.<\/small> et al. (2021). Advances and open problems in federated learning. Found. Trends Mach. Learn.<\/i> 14<\/b> 1\u2013210.","DOI":"10.1561\/2200000083"},{"key":"21","unstructured":"KOLOSKOVA, A.<\/small>, STICH, S.<\/small> and JAGGI, M.<\/small> (2019). Decentralized stochastic optimization and gossip algorithms with compressed communication. In Proceedings of the<\/i> 36th International Conference on Machine Learning<\/i> 97<\/b> 3478\u20133487."},{"key":"23","unstructured":"LEE, S.<\/small>, LIAO, Y.<\/small>, SEO, M. H.<\/small> et al. (2022). Fast and robust online inference with stochastic gradient descent via random scaling. In The AAAI Conference on Artificial Intelligence<\/i>."},{"key":"24","doi-asserted-by":"publisher","unstructured":"LI, T.<\/small>, SAHU, A.<\/small>, TALWALKAR, A.<\/small> et al. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Process. Mag.<\/i> 37<\/b> 50\u201360.","DOI":"10.1109\/MSP.2020.2975749"},{"key":"25","unstructured":"LI, X.<\/small>, LIANG, J.<\/small>, CHANG, X.<\/small> et al. (2022). Statistical estimation and online inference via local SGD. In Conference on Learning Theory<\/i> (COLT<\/i>) 178<\/b> 1613\u20131661."},{"key":"26","unstructured":"LIAN, X.<\/small>, ZHANG, C.<\/small>, ZHANG, H.<\/small> et al. (2017). Can decentralized algorithms outperform centralized algorithms? A case study for decentralized parallel stochastic gradient descent. In Advances in Neural Information Processing Systems<\/i> 30<\/b>."},{"key":"27","unstructured":"MCMAHAN, B.<\/small>, MOORE, E.<\/small>, RAMAGE, D.<\/small> et al. (2017). Communication-efficient learning of deep networks from decentralized data. Int. Conf. Artif. Intell. Stat.<\/i> 54<\/b> 1273\u20131282."},{"key":"28","doi-asserted-by":"publisher","unstructured":"NEDI\u0106, A.<\/small>, OLSHEVSKY, A.<\/small> and RABBAT, M. G.<\/small> (2018). Network topology and communication-computation tradeoffs in decentralized optimization. Proc. IEEE<\/i> 106<\/b> 953\u2013976.","DOI":"10.1109\/JPROC.2018.2817461"},{"key":"29","doi-asserted-by":"publisher","unstructured":"NEDI\u0106, A.<\/small>, OLSHEVSKY, A.<\/small> and SHI, W.<\/small> (2017). Achieving geometric convergence for distributed optimization over time-varying graphs. SIAM J. Optim.<\/i> 27<\/b> 2597\u20132633.","DOI":"10.1137\/16M1084316"},{"key":"30","unstructured":"NGUYEN, L. M.<\/small>, NGUYEN, P. H.<\/small>, RICHT\u00c1RIK, P.<\/small>, SCHEINBERG, K.<\/small>, TAK\u00c1\u010c, M.<\/small> and VAN DIJK, M.<\/small> (2019). New convergence aspects of stochastic gradient algorithms. J. Mach. Learn. Res.<\/i> 20<\/b> Paper No. 176, 49."},{"key":"31","doi-asserted-by":"publisher","unstructured":"PANTELOPOULOS, A.<\/small> and BOURBAKIS, N. G.<\/small> (2010). A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans. Syst. Man Cybern<\/i>., Part C Appl. Rev.<\/i> 40<\/b> 1\u201312.","DOI":"10.1109\/TSMCC.2009.2032660"},{"key":"33","doi-asserted-by":"publisher","unstructured":"QIAN, N.<\/small> (1999). On the momentum term in gradient descent learning algorithms. Neural Netw.<\/i> 12<\/b> 145\u2013151.","DOI":"10.1016\/S0893-6080(98)00116-6"},{"key":"35","doi-asserted-by":"crossref","unstructured":"ROBBINS, H.<\/small> and SIEGMUND, D.<\/small> (1971). A convergence theorem for non negative almost supermartingales and some applications. In Optimizing Methods in Statistics<\/i> (Proc. Sympos<\/i>., Ohio State Univ<\/i>., Columbus, Ohio<\/i>, 1971) 233\u2013257. Academic Press, New York.","DOI":"10.1016\/B978-0-12-604550-5.50015-8"},{"key":"36","unstructured":"RUPPERT, D.<\/small> (1988). Efficient estimations from a slowly convergent Robbins\u2013Monro process Technical report, Cornell Univ. Operations Research and Industrial Engineering."},{"key":"37","doi-asserted-by":"publisher","unstructured":"SACKS, J.<\/small> (1958). Asymptotic distribution of stochastic approximation procedures. Ann. Math. Stat.<\/i> 29<\/b> 373\u2013405.","DOI":"10.1214\/aoms\/1177706619"},{"key":"38","unstructured":"STICH, S. U.<\/small> (2019). Local SGD converges fast and communicates little. In International Conference on Learning Representations<\/i>."},{"key":"39","unstructured":"SU, W. J.<\/small> and ZHU, Y.<\/small> (2018). Uncertainty Quantification for Online Learning and Stochastic Approximation via Hierarchical Incremental Gradient Descent. arXiv. Available at arXiv:10.48550\/ARXIV.1802.04876<\/a>."},{"key":"40","unstructured":"WANG, J.<\/small>, CHARLES, Z.<\/small>, XU, Z.<\/small> et al. (2021). A field guide to federated optimization."},{"key":"41","unstructured":"WANG, J.<\/small> and JOSHI, G.<\/small> (2021). Cooperative SGD: A unified framework for the design and analysis of local-update SGD algorithms. J. Mach. Learn. Res.<\/i> 22<\/b> 1\u201350."},{"key":"42","doi-asserted-by":"publisher","unstructured":"YUAN, K.<\/small>, ALGHUNAIM, S. A.<\/small>, YING, B.<\/small> and SAYED, A. H.<\/small> (2020). On the influence of bias-correction on distributed stochastic optimization. IEEE Trans. Signal Process.<\/i> 68<\/b> 4352\u20134367.","DOI":"10.1109\/TSP.2020.3008605"},{"key":"43","doi-asserted-by":"publisher","unstructured":"YUAN, K.<\/small>, LING, Q.<\/small> and YIN, W.<\/small> (2016). On the convergence of decentralized gradient descent. SIAM J. Optim.<\/i> 26<\/b> 1835\u20131854.","DOI":"10.1137\/130943170"},{"key":"45","doi-asserted-by":"publisher","unstructured":"ZHU, W.<\/small>, CHEN, X.<\/small> and WU, W. B.<\/small> (2023). Online covariance matrix estimation in stochastic gradient descent. J. Amer. Statist. Assoc.<\/i> 118<\/b> 393\u2013404.","DOI":"10.1080\/01621459.2021.1933498"}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T17:18:56Z","timestamp":1791393536000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-52\/issue-6\/Statistical-inference-for-decentralized-federated-learning\/10.1214\/24-AOS2452.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,1]]},"references-count":45,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,12,1]]}},"URL":"https:\/\/doi.org\/10.1214\/24-aos2452","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,1]]}}}