{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T21:50:27Z","timestamp":1791323427064,"version":"4.3.1"},"reference-count":34,"publisher":"Institute of Mathematical Statistics","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2026,10,1]]},"abstract":"We study the spectral properties of the Jacobian matrix in artificial neural networks and establish both its limiting spectral distribution and the second-order fluctuations of its spectrum. Our analysis uncovers fundamental phenomena, including universality properties with respect to the weight distribution and a phase transition in spectral behavior driven by the choice of activation function. Leveraging tools from random matrix theory, we first analyze a class of nonlinear covariance ensembles, which may be of independent interest in high-dimensional statistics, and subsequently characterize the spectral fluctuations of the Jacobian matrix, highlighting their dependence on weight initialization, layer widths, activation functions and network depth. These findings provide finite-width distributional calibration for Jacobian-based stability diagnostics and quantify initialization-induced uncertainty in signal propagation, gradient stability and local sensitivity of high-dimensional neural networks.<\/jats:p>","DOI":"10.1214\/26-aos2654","type":"journal-article","created":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T12:52:45Z","timestamp":1790945565000},"page":"2481-2506","source":"Crossref","is-referenced-by-count":0,"title":["Spectral asymptotics of neural network Jacobians: Convergency, universality, and phase transition"],"prefix":"10.1214","volume":"54","author":[{"given":"Huiqin","family":"Li","sequence":"first","affiliation":[{"name":"School of Statistics and Data Science, Nanjing Audit University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangming","family":"Pan","sequence":"additional","affiliation":[{"name":"Division of Mathematical Sciences, Nanyang Technological University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4911-3159","authenticated-orcid":false,"given":"Yanqing","family":"Yin","sequence":"additional","affiliation":[{"name":"School of Statistics and Data Science, Nanjing Audit University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"29","doi-asserted-by":"publisher","unstructured":"SILVERSTEIN, J. W.<\/small> and CHOI, S.-I.<\/small> (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal.<\/i> 54<\/b> 295\u2013309.","DOI":"10.1006\/jmva.1995.1058"},{"key":"6","doi-asserted-by":"publisher","unstructured":"BAO, Z.<\/small>, PAN, G.<\/small> and ZHOU, W.<\/small> (2012). Tracy\u2013Widom law for the extreme eigenvalues of sample correlation matrices. Electron. J. Probab.<\/i> 17<\/b> no. 88, 32 pp.","DOI":"10.1214\/EJP.v17-1962"},{"key":"4","unstructured":"BAI, Z.<\/small> and ZHOU, W.<\/small> (2008). Large sample covariance matrices without independence structures in columns. Statist. Sinica<\/i> 18<\/b> 425\u2013442."},{"key":"17","doi-asserted-by":"publisher","unstructured":"KOHLER, M.<\/small> and LANGER, S.<\/small> (2021). On the rate of convergence of fully connected deep neural network regression estimates. Ann. Statist.<\/i> 49<\/b> 2231\u20132249.","DOI":"10.1214\/20-aos2034"},{"key":"3","doi-asserted-by":"publisher","unstructured":"Bai, Z.<\/small> and Silverstein, J. W.<\/small> (2010). Spectral Analysis of Large Dimensional Random Matrices<\/i>, 2nd ed. Springer Series in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-1-4419-0661-8"},{"key":"2","unstructured":"Anderson, G. W.<\/small>, Guionnet, A.<\/small> and Zeitouni, O.<\/small> (2010). An Introduction to Random Matrices<\/i>. Cambridge Studies in Advanced Mathematics<\/i> 118<\/b>. Cambridge Univ. Press, Cambridge."},{"key":"24","doi-asserted-by":"publisher","unstructured":"Mingo, J. A.<\/small> and Speicher, R.<\/small> (2006). Second order freeness and fluctuations of random matrices. I. Gaussian and Wishart matrices and cyclic Fock spaces. J. Funct. Anal.<\/i> 235<\/b> 226\u2013270.","DOI":"10.1016\/j.jfa.2005.10.007"},{"key":"10","doi-asserted-by":"publisher","unstructured":"El Karoui, N.<\/small> (2010). The spectrum of kernel random matrices. Ann. Statist.<\/i> 38<\/b> 1\u201350.","DOI":"10.1214\/08-AOS648"},{"key":"5","doi-asserted-by":"publisher","unstructured":"Bai, Z. D.<\/small> and Silverstein, J. W.<\/small> (2004). CLT for linear spectral statistics of large-dimensional sample covariance matrices. Ann. Probab.<\/i> 32<\/b> 553\u2013605.","DOI":"10.1214\/aop\/1078415845"},{"key":"11","doi-asserted-by":"publisher","unstructured":"Gao, J.<\/small>, Han, X.<\/small>, Pan, G.<\/small> and Yang, Y.<\/small> (2017). High dimensional correlation matrices: The central limit theorem and its applications. J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> 79<\/b> 677\u2013693.","DOI":"10.1111\/rssb.12189"},{"key":"25","doi-asserted-by":"publisher","unstructured":"Pan, G. M.<\/small> and Zhou, W.<\/small> (2008). Central limit theorem for signal-to-interference ratio of reduced rank linear receiver. Ann. Appl. Probab.<\/i> 18<\/b> 1232\u20131270.","DOI":"10.1214\/07-AAP477"},{"key":"9","doi-asserted-by":"publisher","unstructured":"El Karoui, N.<\/small> (2009). Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond. Ann. Appl. Probab.<\/i> 19<\/b> 2362\u20132405.","DOI":"10.1214\/08-AAP548"},{"key":"15","doi-asserted-by":"publisher","unstructured":"Jiang, T.<\/small> (2004). The asymptotic distributions of the largest entries of sample correlation matrices. Ann. Appl. Probab.<\/i> 14<\/b> 865\u2013880.","DOI":"10.1214\/105051604000000143"},{"key":"21","doi-asserted-by":"publisher","unstructured":"LOUART, C.<\/small>, LIAO, Z.<\/small> and COUILLET, R.<\/small> (2018). A random matrix approach to neural networks. Ann. Appl. Probab.<\/i> 28<\/b> 1190\u20131248.","DOI":"10.1214\/17-AAP1328"},{"key":"27","unstructured":"PENNINGTON, J.<\/small>, SCHOENHOLZ, S.<\/small> and GANGULI, S.<\/small> (2018). The emergence of spectral universality in deep networks. In International Conference on Artificial Intelligence and Statistics<\/i> 1924\u20131932. PMLR."},{"key":"22","unstructured":"MAR\u010cENKO, V. A.<\/small> and PASTUR, L. A.<\/small> (1967). Distribution of eigenvalues in certain sets of random matrices. Mat. Sb<\/i>. (N.S<\/i>.) 114<\/b> 507\u2013536."},{"key":"13","doi-asserted-by":"publisher","unstructured":"HU, J.<\/small>, LI, W.<\/small>, LIU, Z.<\/small> and ZHOU, W.<\/small> (2019). High-dimensional covariance matrices in elliptical distributions with application to spherical test. Ann. Statist.<\/i> 47<\/b> 527\u2013555.","DOI":"10.1214\/18-AOS1699"},{"key":"33","doi-asserted-by":"publisher","unstructured":"YIN, Y.<\/small>, ZHENG, S.<\/small> and ZOU, T.<\/small> (2023). Central limit theorem of linear spectral statistics of high-dimensional sample correlation matrices. Bernoulli<\/i> 29<\/b> 984\u20131006.","DOI":"10.3150\/22-bej1487"},{"key":"1","unstructured":"AKHIEZER, N. I.<\/small> and GLAZMAN, I. M.<\/small> (2013). Theory of Linear Operators in Hilbert Space<\/i>. Courier Corporation."},{"key":"7","unstructured":"DING, X.<\/small>, XIE, J.<\/small>, YU, L.<\/small> and ZHOU, W.<\/small> (2023). Extreme eigenvalues of sample covariance matrices under generalized elliptical models with applications. Preprint. Available at arXiv:2303.03532<\/a>."},{"key":"8","doi-asserted-by":"crossref","unstructured":"DOSHI, D.<\/small>, HE, T.<\/small> and GROMOV, A.<\/small> (2023). Critical initialization of wide and deep neural networks using partial Jacobians: General theory and applications. Adv. Neural Inf. Process. Syst.<\/i> 36<\/b> 40054\u201340095.","DOI":"10.52202\/075280-1743"},{"key":"12","doi-asserted-by":"publisher","unstructured":"G\u00d6TZE, F.<\/small>, NAUMOV, A.<\/small> and TIKHOMIROV, A.<\/small> (2017). Distribution of linear statistics of singular values of the product of random matrices. Bernoulli<\/i> 23<\/b> 3067\u20133113.","DOI":"10.3150\/16-BEJ837"},{"key":"14","doi-asserted-by":"crossref","unstructured":"ITO, T.<\/small>, KLINGER, T.<\/small>, SCHULTZ, D.<\/small>, MURRAY, J.<\/small>, COLE, M.<\/small> and RIGOTTI, M.<\/small> (2022). Compositional generalization through abstract representations in human and artificial neural networks. Adv. Neural Inf. Process. Syst.<\/i> 35<\/b> 32225\u201332239.","DOI":"10.52202\/068431-2335"},{"key":"16","unstructured":"KOGLER, C.<\/small>, SCHWARZ, T.<\/small> and KITTLE, S.<\/small> (2026). Optimal initialization in depth: Lyapunov initialization and limit theorems for deep leaky ReLU networks. Preprint. Available at arXiv:2602.10949<\/a>."},{"key":"18","doi-asserted-by":"crossref","unstructured":"LI, H.<\/small>, PAN, G.<\/small> and YIN, Y.<\/small> (2026). Supplement to \u201cSpectral asymptotics of neural network Jacobians: Convergency, universality, and phase transition.\u201d https:\/\/doi.org\/10.1214\/26-AOS2654SUPP<\/a>","DOI":"10.1214\/26-AOS2654SUPP"},{"key":"19","doi-asserted-by":"publisher","unstructured":"LI, H.<\/small>, PAN, G.<\/small>, YIN, Y.<\/small> and ZHOU, W.<\/small> (2024). Separable sample covariance matrices under elliptical populations with applications. Trans. Amer. Math. Soc.<\/i> 377<\/b> 3805\u20133843.","DOI":"10.1090\/tran\/9070"},{"key":"20","doi-asserted-by":"publisher","unstructured":"LI, H.<\/small>, PAN, G.<\/small>, YIN, Y.<\/small> and ZHOU, W.<\/small> (2024). Spectral analysis of Gram matrices with missing at random observations: Convergence, central limit theorems, and applications in statistical inference. Ann. Statist.<\/i> 52<\/b> 1254\u20131275.","DOI":"10.1214\/24-aos2392"},{"key":"23","doi-asserted-by":"publisher","unstructured":"MARINAI, S.<\/small>, GORI, M.<\/small> and SODA, G.<\/small> (2005). Artificial neural networks for document analysis and recognition. IEEE Trans. Pattern Anal. Mach. Intell.<\/i> 27<\/b> 23\u201335.","DOI":"10.1109\/TPAMI.2005.4"},{"key":"26","doi-asserted-by":"publisher","unstructured":"PASTUR, L.<\/small> and SLAVIN, V.<\/small> (2023). On random matrices arising in deep neural networks: General i.i.d. case. Random Matrices Theory Appl.<\/i> 12<\/b> Paper No. 2250046, 41 pp.","DOI":"10.1142\/S2010326322500460"},{"key":"28","doi-asserted-by":"publisher","unstructured":"PENNINGTON, J.<\/small> and WORAH, P.<\/small> (2019). Nonlinear random matrix theory for deep learning. J. Stat. Mech. Theory Exp.<\/i> 2019<\/b> 124005, 14 pp.","DOI":"10.1088\/1742-5468\/ab3bc3"},{"key":"30","doi-asserted-by":"crossref","unstructured":"WISHART, J.<\/small> (1928). The generalised product moment distribution in samples from a normal multivariate population. Biometrika<\/i> 20<\/b> 32\u201352.","DOI":"10.1093\/biomet\/20A.1-2.32"},{"key":"31","doi-asserted-by":"crossref","unstructured":"YAO, X.<\/small> (1999). Evolving artificial neural networks. Proc. IEEE<\/i> 87<\/b> 1423\u20131447.","DOI":"10.1109\/5.784219"},{"key":"32","doi-asserted-by":"publisher","unstructured":"YIN, Y.<\/small> and MA, Y.<\/small> (2022). Properties of eigenvalues and eigenvectors of large-dimensional sample correlation matrices. Ann. Appl. Probab.<\/i> 32<\/b> 4763\u20134802.","DOI":"10.1214\/22-aap1802"},{"key":"34","unstructured":"ZHANG, L.<\/small> (2006). Spectral analysis of large dimensional random matrices. PhD Thesis, National Univ. Singapore."}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T21:05:40Z","timestamp":1791320740000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-54\/issue-5\/Spectral-asymptotics-of-neural-network-Jacobians--Convergency-universality-and\/10.1214\/26-AOS2654.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,1]]},"references-count":34,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,10,1]]}},"URL":"https:\/\/doi.org\/10.1214\/26-aos2654","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,1]]}}}