{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:55:53Z","timestamp":1791478553355,"version":"4.3.4"},"reference-count":50,"publisher":"Institute of Mathematical Statistics","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2019,12,1]]},"abstract":"<jats:p>Principal component analysis (PCA) is fundamental to statistical machine learning. It extracts latent principal factors that contribute to the most variation of the data. When data are stored across multiple machines, however, communication cost can prohibit the computation of PCA in a central location and distributed algorithms for PCA are thus needed. This paper proposes and studies a distributed PCA algorithm: each node machine computes the top $K$ eigenvectors and transmits them to the central server; the central server then aggregates the information from all the node machines and conducts a PCA based on the aggregated information. We investigate the bias and variance for the resulting distributed estimator of the top $K$ eigenvectors. In particular, we show that for distributions with symmetric innovation, the empirical top eigenspaces are unbiased, and hence the distributed PCA is \u201cunbiased.\u201d We derive the rate of convergence for distributed PCA estimators, which depends explicitly on the effective rank of covariance, eigengap, and the number of machines. We show that when the number of machines is not unreasonably large, the distributed PCA performs as well as the whole sample PCA, even without full access of whole data. The theoretical results are verified by an extensive simulation study. We also extend our analysis to the heterogeneous case where the population covariance matrices are different across local machines but share similar top eigenstructures.<\/jats:p>","DOI":"10.1214\/18-aos1713","type":"journal-article","created":{"date-parts":[[2019,10,30]],"date-time":"2019-10-30T22:04:00Z","timestamp":1572473040000},"page":"3009-3031","source":"Crossref","is-referenced-by-count":119,"title":["Distributed estimation of principal eigenspaces"],"prefix":"10.1214","volume":"47","author":[{"given":"Jianqing","family":"Fan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dong","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaizheng","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziwei","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"18","doi-asserted-by":"publisher","unstructured":"Johnstone, I. M. and Lu, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. <i>J. Amer. Statist. Assoc.<\/i> <b>104<\/b> 682\u2013693.","DOI":"10.1198\/jasa.2009.0121"},{"key":"7","doi-asserted-by":"crossref","unstructured":"Chen, X. and Xie, M. (2014). A split-and-conquer approach for analysis of extraordinarily large data. <i>Statist. Sinica<\/i> <b>24<\/b> 1655\u20131684.","DOI":"10.5705\/ss.2013.088"},{"key":"2","doi-asserted-by":"publisher","unstructured":"Baik, J., Ben Arous, G. and P\u00e9ch\u00e9, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. <i>Ann. Probab.<\/i> <b>33<\/b> 1643\u20131697.","DOI":"10.1214\/009117905000000233"},{"key":"17","doi-asserted-by":"publisher","unstructured":"Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. <i>Ann. Statist.<\/i> <b>29<\/b> 295\u2013327.","DOI":"10.1214\/aos\/1009210544"},{"key":"33","unstructured":"Paul, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. <i>Statist. Sinica<\/i> <b>17<\/b> 1617\u20131642."},{"key":"23","doi-asserted-by":"publisher","unstructured":"Kneip, A. and Utikal, K. J. (2001). Inference for density families using functional principal component analysis. <i>J. Amer. Statist. Assoc.<\/i> <b>96<\/b> 519\u2013542.","DOI":"10.1198\/016214501753168235"},{"key":"6","doi-asserted-by":"publisher","unstructured":"Cai, T. T., Ma, Z. and Wu, Y. (2013). Sparse PCA: Optimal rates and adaptive estimation. <i>Ann. Statist.<\/i> <b>41<\/b> 3074\u20133110.","DOI":"10.1214\/13-AOS1178"},{"key":"16","doi-asserted-by":"crossref","unstructured":"Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. <i>J. Educ. Psychol.<\/i> <b>24<\/b> 417\u2013441.","DOI":"10.1037\/h0071325"},{"key":"44","doi-asserted-by":"publisher","unstructured":"Vu, V. Q. and Lei, J. (2013). Minimax sparse principal subspace estimation in high dimensions. <i>Ann. Statist.<\/i> <b>41<\/b> 2905\u20132947.","DOI":"10.1214\/13-AOS1151"},{"key":"19","doi-asserted-by":"publisher","unstructured":"Jung, S. and Marron, J. S. (2009). PCA consistency in high dimension, low sample size context. <i>Ann. Statist.<\/i> <b>37<\/b> 4104\u20134130.","DOI":"10.1214\/09-AOS709"},{"key":"15","doi-asserted-by":"publisher","unstructured":"Halko, N., Martinsson, P. G. and Tropp, J. A. (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. <i>SIAM Rev.<\/i> <b>53<\/b> 217\u2013288.","DOI":"10.1137\/090771806"},{"key":"39","doi-asserted-by":"publisher","unstructured":"Shen, D., Shen, H. and Marron, J. S. (2013). Consistency of sparse PCA in high dimension, low sample size contexts. <i>J. Multivariate Anal.<\/i> <b>115<\/b> 317\u2013333.","DOI":"10.1016\/j.jmva.2012.10.007"},{"key":"31","doi-asserted-by":"publisher","unstructured":"Nadler, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. <i>Ann. Statist.<\/i> <b>36<\/b> 2791\u20132817.","DOI":"10.1214\/08-AOS618"},{"key":"34","unstructured":"Paul, D. and Johnstone, I. M. (2012). Augmented sparse principal component analysis for high dimensional data. Preprint. Available at <a href=\"arXiv:1202.1242\">arXiv:1202.1242<\/a>."},{"key":"49","doi-asserted-by":"publisher","unstructured":"Yu, Y., Wang, T. and Samworth, R. J. (2015). A useful variant of the Davis\u2013Kahan theorem for statisticians. <i>Biometrika<\/i> <b>102<\/b> 315\u2013323.","DOI":"10.1093\/biomet\/asv008"},{"key":"32","doi-asserted-by":"publisher","unstructured":"Onatski, A. (2012). Asymptotics of the principal components estimator of large factor models with weakly influential factors. <i>J. Econometrics<\/i> <b>168<\/b> 244\u2013258.","DOI":"10.1016\/j.jeconom.2012.01.034"},{"key":"1","doi-asserted-by":"publisher","unstructured":"Anderson, T. W. (1963). Asymptotic theory for principal component analysis. <i>Ann. Math. Stat.<\/i> <b>34<\/b> 122\u2013148.","DOI":"10.1214\/aoms\/1177704248"},{"key":"25","doi-asserted-by":"publisher","unstructured":"Koltchinskii, V. and Lounici, K. (2017). Concentration inequalities and moment bounds for sample covariance operators. <i>Bernoulli<\/i> <b>23<\/b> 110\u2013133.","DOI":"10.3150\/15-BEJ730"},{"key":"40","doi-asserted-by":"crossref","unstructured":"Shen, D., Shen, H., Zhu, H. and Marron, J. S. (2016). The statistics and mathematics of high dimension low sample size asymptotics. <i>Statist. Sinica<\/i> <b>26<\/b> 1747\u20131770.","DOI":"10.5705\/ss.202015.0088"},{"key":"45","doi-asserted-by":"publisher","unstructured":"Wang, R. (2015). Singular vector perturbation under Gaussian noise. <i>SIAM J. Matrix Anal. Appl.<\/i> <b>36<\/b> 158\u2013177.","DOI":"10.1137\/130938177"},{"key":"46","doi-asserted-by":"publisher","unstructured":"Wang, W. and Fan, J. (2017). Asymptotics of empirical eigenstructure for high dimensional spiked covariance. <i>Ann. Statist.<\/i> <b>45<\/b> 1342\u20131374.","DOI":"10.1214\/16-AOS1487"},{"key":"24","doi-asserted-by":"publisher","unstructured":"Koltchinskii, V. and Lounici, K. (2016). Asymptotics and concentration bounds for bilinear forms of spectral projectors of sample covariance. <i>Ann. Inst. Henri Poincar\u00e9 Probab. Stat.<\/i> <b>52<\/b> 1976\u20132013.","DOI":"10.1214\/15-AIHP705"},{"key":"3","unstructured":"Battey, H., Fan, J., Liu, H., Lu, J. and Zhu, Z. (2015). Distributed estimation and inference with statistical guarantees. Preprint. Available at <a href=\"arXiv:1509.05457\">arXiv:1509.05457<\/a>."},{"key":"4","doi-asserted-by":"crossref","unstructured":"Bertrand, A. and Moonen, M. (2014). Distributed adaptive estimation of covariance matrix eigenvectors in wireless sensor networks with application to distributed PCA. <i>Signal Process.<\/i> <b>104<\/b> 120\u2013135.","DOI":"10.1016\/j.sigpro.2014.03.037"},{"key":"8","unstructured":"Chen, T.-L., Chang, D. D., Huang, S.-Y., Chen, H., Lin, C. and Wang, W. (2016). Integrating multiple random sketches for singular value decomposition. Preprint. Available at <a href=\"arXiv:1608.08285\">arXiv:1608.08285<\/a>."},{"key":"9","doi-asserted-by":"publisher","unstructured":"El Karoui, N. and d\u2019Aspremont, A. (2010). Second order accurate distributed eigenvector computation for extremely large matrices. <i>Electron. J. Stat.<\/i> <b>4<\/b> 1345\u20131385.","DOI":"10.1214\/10-EJS577"},{"key":"10","unstructured":"Fan, J., Wang, W. and Zhu, Z. (2016). Robust low-rank matrix recovery. Preprint. Available at <a href=\"arXiv:1603.08315\">arXiv:1603.08315<\/a>."},{"key":"11","doi-asserted-by":"publisher","unstructured":"Fan, J., Wang, D., Wang, K. and Zhu, Z. (2019). Supplement to \u201cDistributed estimation of principal eigenspaces.\u201d <a href=\"DOI:10.1214\/18-AOS1713SUPP\">DOI:10.1214\/18-AOS1713SUPP<\/a>.","DOI":"10.1214\/18-AOS1713"},{"key":"12","unstructured":"Garber, D., Shamir, O. and Srebro, N. (2017). Communication-efficient algorithms for distributed stochastic principal component analysis. Preprint. Available at <a href=\"arXiv:1702.08169\">arXiv:1702.08169<\/a>."},{"key":"14","doi-asserted-by":"crossref","unstructured":"Guo, Z.-C., Lin, S.-B. and Zhou, D.-X. (2017). Learning theory of distributed spectral algorithms. <i>Inverse Probl.<\/i> <b>33<\/b> 074009, 29.","DOI":"10.1088\/1361-6420\/aa72b2"},{"key":"21","doi-asserted-by":"publisher","unstructured":"Kargupta, H., Huang, W., Sivakumar, K. and Johnson, E. (2001). Distributed clustering using collective principal component analysis. <i>Knowledge and Information Systems<\/i> <b>3<\/b> 422\u2013448.","DOI":"10.1007\/PL00011677"},{"key":"26","unstructured":"Lee, J. D., Liu, Q., Sun, Y. and Taylor, J. E. (2017). Communication-efficient sparse regression. <i>J. Mach. Learn. Res.<\/i> <b>18<\/b> 1\u201330."},{"key":"27","doi-asserted-by":"crossref","unstructured":"Li, L., Scaglione, A. and Manton, J. H. (2011). Distributed principal subspace estimation in wireless sensor networks. <i>IEEE J. Sel. Top. Signal Process.<\/i> <b>5<\/b> 725\u2013738.","DOI":"10.1109\/JSTSP.2011.2118742"},{"key":"29","doi-asserted-by":"publisher","unstructured":"Minsker, S. (2018). Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries. <i>Ann. Statist.<\/i> <b>46<\/b> 2871\u20132903.","DOI":"10.1214\/17-AOS1642"},{"key":"30","unstructured":"M\u00fccke, N. and Blanchard, G. (2018). Parallelizing spectrally regularized kernel algorithms. <i>J. Mach. Learn. Res.<\/i> <b>19<\/b> Paper No. 30, 29."},{"key":"37","doi-asserted-by":"publisher","unstructured":"Reiss, M. and Wahl, M. (2016). Non-asymptotic upper bounds for the reconstruction error of PCA. Preprint. Available at <a href=\"arXiv:1609.03779\">arXiv:1609.03779<\/a>.","DOI":"10.1214\/19-AOS1839"},{"key":"38","doi-asserted-by":"publisher","unstructured":"Schizas, I. D. and Aduroja, A. (2015). A distributed framework for dimensionality reduction and denoising. <i>IEEE Trans. Signal Process.<\/i> <b>63<\/b> 6379\u20136394.","DOI":"10.1109\/TSP.2015.2465300"},{"key":"41","doi-asserted-by":"publisher","unstructured":"Tropp, J. A., Yurtsever, A., Udell, M. and Cevher, V. (2016). Randomized single-view algorithms for low-rank matrix approximation. Preprint. Available at <a href=\"arXiv:1609.00048\">arXiv:1609.00048<\/a>.","DOI":"10.1137\/17M1111590"},{"key":"42","doi-asserted-by":"publisher","unstructured":"Vaccaro, R. J. (1994). A second-order perturbation expansion for the SVD. <i>SIAM J. Matrix Anal. Appl.<\/i> <b>15<\/b> 661\u2013671.","DOI":"10.1137\/S0895479891224245"},{"key":"48","doi-asserted-by":"crossref","unstructured":"Xu, Z. (2002). Perturbation analysis for subspace decomposition with applications in subspace-based algorithms. <i>IEEE Trans. Signal Process.<\/i> <b>50<\/b> 2820\u20132830.","DOI":"10.1109\/TSP.2002.804084"},{"key":"43","doi-asserted-by":"publisher","unstructured":"Vershynin, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In <i>Compressed Sensing<\/i> 210\u2013268. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511794308.006"},{"key":"5","doi-asserted-by":"crossref","unstructured":"Boutsidis, C., Woodruff, D. P. and Zhong, P. (2016). Optimal principal component analysis in distributed and streaming models. In <i>STOC\u2019<\/i>16<i>\u2014Proceedings of the<\/i> 48<i>th Annual ACM SIGACT Symposium on Theory of Computing<\/i> 236\u2013249. ACM, New York.","DOI":"10.1145\/2897518.2897646"},{"key":"13","unstructured":"Golub, G. H. and Van Loan, C. F. (2012). <i>Matrix Computations<\/i>, 3rd ed. <i>Johns Hopkins Studies in the Mathematical Sciences<\/i>. Johns Hopkins Univ. Press, Baltimore, MD."},{"key":"20","unstructured":"Kannan, R., Vempala, S. and Woodruff, D. (2014). Principal component analysis and higher correlations for distributed data. In <i>Conference on Learning Theory<\/i> 1040\u20131057."},{"key":"22","unstructured":"Kato, T. (1966). <i>Perturbation Theory for Linear Operators. Die Grundlehren der Mathematischen Wissenschaften<\/i>, <i>Band<\/i> 132. Springer, New York."},{"key":"28","unstructured":"Liang, Y., Balcan, M.-F. F., Kanchanapally, V. and Woodruff, D. (2014). Improved distributed principal component analysis. In <i>Advances in Neural Information Processing Systems<\/i> 3113\u20133121."},{"key":"35","doi-asserted-by":"publisher","unstructured":"Pearson, K. (1901). On lines and planes of closest fit to systems of point in space. <i>Philosophical Magazine Series<\/i> 6 <b>2<\/b> 559\u2013572.","DOI":"10.1080\/14786440109462720"},{"key":"36","unstructured":"Qu, Y., Ostrouchov, G., Samatova, N. and Geist, A. (2002). Principal component analysis for dimension reduction in massive distributed data sets. In <i>IEEE International Conference on Data Mining<\/i> (<i>ICDM<\/i>)."},{"key":"47","unstructured":"Wei, X. and Minsker, S. (2017). Estimation of the covariance structure of heavy-tailed distributions. In <i>Advances in Neural Information Processing Systems<\/i> 2855\u20132864."},{"key":"50","unstructured":"Zhang, Y., Duchi, J. C. and Wainwright, M. J. (2013). Divide and conquer kernel ridge regression. In <i>COLT<\/i> 592\u2013617."}],"container-title":["The Annals of Statistics"],"original-title":[],"link":[{"URL":"https:\/\/projecteuclid.org\/download\/pdfview_1\/euclid.aos\/1572487381","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:18:18Z","timestamp":1791476298000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-47\/issue-6\/Distributed-estimation-of-principal-eigenspaces\/10.1214\/18-AOS1713.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12,1]]},"references-count":50,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2019,12,1]]}},"URL":"https:\/\/doi.org\/10.1214\/18-aos1713","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12,1]]}}}