{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T16:56:36Z","timestamp":1791392196532,"version":"4.3.3"},"reference-count":80,"publisher":"Institute of Mathematical Statistics","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2025,12,1]]},"abstract":"<jats:p>We obtain upper bounds for the estimation error of Kernel Ridge Regression (KRR) for all nonnegative regularization parameters, offering a geometric perspective on various phenomena in KRR. As applications: 1. We address the Multiple Descents problem, unifying the proofs of (In Proceedings of Thirty Third Conference on Learning Theory (2020) 2683\u20132711 PMLR) and (Ann. Statist. 49 (2021) 1029\u20131054) for polynomial kernels in nonasymptotic regime and we establish Multiple Descents for the generalization error of KRR for polynomial kernel under sub-Gaussian design in asymptotic regimes. 2. In the nonasymptotic regime, we have established a one-sided isomorphic version of the Gaussian Equivalence Conjecture for sub-Gaussian design vectors. 3. We offer a novel perspective on the linearization of kernel matrices of nonlinear kernels, extending it to the power regime for polynomial kernels. 4. Our theory extends the results in (J. Mach. Learn. Res. 24 (2023) Paper No. [123]) under weak moment assumption. Our proof is based on three mathematical tools developed in this paper that can be of independent interest: 1. Dvoretzky\u2013Milman theorem for ellipsoids under (very) weak moment assumptions. 2. Restricted isomorphic property in reproducing kernel Hilbert spaces with embedding index conditions. 3. A concentration inequality for finite-degree polynomial kernel functions.<\/jats:p>","DOI":"10.1214\/25-aos2556","type":"journal-article","created":{"date-parts":[[2025,12,23]],"date-time":"2025-12-23T06:17:06Z","timestamp":1766470626000},"page":"2592-2616","source":"Crossref","is-referenced-by-count":4,"title":["A geometrical analysis of kernel ridge regression and its applications"],"prefix":"10.1214","volume":"53","author":[{"given":"Georgios","family":"Gavrilopoulos","sequence":"first","affiliation":[{"name":"ETH Zurich, Seminar for Statistics"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guillaume","family":"Lecu\u00e9","sequence":"additional","affiliation":[{"name":"ESSEC, Business School"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zong","family":"Shang","sequence":"additional","affiliation":[{"name":"CREST, ENSAE, IPParis"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"58","doi-asserted-by":"publisher","unstructured":"<small>PISIER, G.<\/small> (1989). <i>The Volume of Convex Bodies and Banach Space Geometry<\/i>. <i>Cambridge Tracts in Mathematics<\/i> <b>94<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511662454"},{"key":"50","doi-asserted-by":"publisher","unstructured":"<small>MEI, S.<\/small>, <small>MISIAKIEWICZ, T.<\/small> and <small>MONTANARI, A.<\/small> (2022). Generalization error of random feature and kernel methods: Hypercontractivity and kernel matrix concentration. <i>Appl. Comput. Harmon. Anal.<\/i> <b>59<\/b> 3\u201384.","DOI":"10.1016\/j.acha.2021.12.003"},{"key":"62","doi-asserted-by":"publisher","unstructured":"<small>SAITOH, S.<\/small> and <small>SAWANO, Y.<\/small> (2016). <i>Theory of Reproducing Kernels and Applications<\/i>. <i>Developments in Mathematics<\/i> <b>44<\/b>. Springer, Singapore.","DOI":"10.1007\/978-981-10-0530-5"},{"key":"52","doi-asserted-by":"publisher","unstructured":"<small>MENDELSON, S.<\/small> and <small>NEEMAN, J.<\/small> (2010). Regularization in kernel learning. <i>Ann. Statist.<\/i> <b>38<\/b> 526\u2013565.","DOI":"10.1214\/09-AOS728"},{"key":"25","doi-asserted-by":"crossref","unstructured":"<small>DO, Y.<\/small> and <small>VU, V.<\/small> (2013). The spectrum of random kernel matrices: Universality results for rough and varying kernels. <i>Random Matrices Theory Appl.<\/i> <b>2<\/b> 1350005.","DOI":"10.1142\/S2010326313500056"},{"key":"64","doi-asserted-by":"publisher","unstructured":"<small>Schmidt-Hieber, J.<\/small> (2020). Nonparametric regression using deep neural networks with ReLU activation function. <i>Ann. Statist.<\/i> <b>48<\/b> 1875\u20131897.","DOI":"10.1214\/19-AOS1875"},{"key":"74","doi-asserted-by":"publisher","unstructured":"<small>Wainwright, M. J.<\/small> (2019). <i>High-Dimensional Statistics<\/i>: <i>A Non-Asymptotic Viewpoint<\/i>. <i>Cambridge Series in Statistical and Probabilistic Mathematics<\/i> <b>48<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/9781108627771"},{"key":"4","doi-asserted-by":"publisher","unstructured":"<small>BARTLETT, P. L.<\/small>, <small>LONG, P. M.<\/small>, <small>LUGOSI, G.<\/small> and <small>TSIGLER, A.<\/small> (2020). Benign overfitting in linear regression. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>117<\/b> 30063\u201330070.","DOI":"10.1073\/pnas.1907378117"},{"key":"31","doi-asserted-by":"publisher","unstructured":"<small>FOUCART, S.<\/small> and <small>RAUHUT, H.<\/small> (2013). <i>A Mathematical Introduction to Compressive Sensing<\/i>. <i>Applied and Numerical Harmonic Analysis<\/i>. Birkh\u00e4user\/Springer, New York.","DOI":"10.1007\/978-0-8176-4948-7"},{"key":"5","doi-asserted-by":"publisher","unstructured":"<small>BARTLETT, P. L.<\/small>, <small>MONTANARI, A.<\/small> and <small>RAKHLIN, A.<\/small> (2021). Deep learning: A statistical viewpoint. <i>Acta Numer.<\/i> <b>30<\/b> 87\u2013201.","DOI":"10.1017\/S0962492921000027"},{"key":"17","doi-asserted-by":"publisher","unstructured":"<small>CHENG, X.<\/small> and <small>SINGER, A.<\/small> (2013). The spectrum of random inner-product kernel matrices. <i>Random Matrices Theory Appl.<\/i> <b>2<\/b> 1350010.","DOI":"10.1142\/S201032631350010X"},{"key":"29","doi-asserted-by":"publisher","unstructured":"<small>FAN, Z.<\/small> and <small>MONTANARI, A.<\/small> (2019). The spectral norm of random inner-product kernel matrices. <i>Probab. Theory Related Fields<\/i> <b>173<\/b> 27\u201385.","DOI":"10.1007\/s00440-018-0830-4"},{"key":"33","doi-asserted-by":"publisher","unstructured":"<small>GHORBANI, B.<\/small>, <small>MEI, S.<\/small>, <small>MISIAKIEWICZ, T.<\/small> and <small>MONTANARI, A.<\/small> (2021). Linearized two-layers neural networks in high dimension. <i>Ann. Statist.<\/i> <b>49<\/b> 1029\u20131054.","DOI":"10.1214\/20-aos1990"},{"key":"78","doi-asserted-by":"publisher","unstructured":"<small>YAROTSKY, D.<\/small> (2017). Error bounds for approximations with deep ReLU networks. <i>Neural Netw.<\/i> <b>94<\/b> 103\u2013114.","DOI":"10.1016\/j.neunet.2017.07.002"},{"key":"37","doi-asserted-by":"publisher","unstructured":"<small>HASTIE, T.<\/small>, <small>MONTANARI, A.<\/small>, <small>ROSSET, S.<\/small> and <small>TIBSHIRANI, R. J.<\/small> (2022). Surprises in high-dimensional ridgeless least squares interpolation. <i>Ann. Statist.<\/i> <b>50<\/b> 949\u2013986.","DOI":"10.1214\/21-aos2133"},{"key":"51","doi-asserted-by":"publisher","unstructured":"<small>MEI, S.<\/small> and <small>MONTANARI, A.<\/small> (2022). The generalization error of random features regression: Precise asymptotics and the double descent curve. <i>Comm. Pure Appl. Math.<\/i> <b>75<\/b> 667\u2013766.","DOI":"10.1002\/cpa.22008"},{"key":"71","doi-asserted-by":"publisher","unstructured":"<small>TIKHOMIROV, K.<\/small> (2018). Sample covariance matrices of heavy-tailed distributions. <i>Int. Math. Res. Not. IMRN<\/i> <b>20<\/b> 6254\u20136289.","DOI":"10.1093\/imrn\/rnx067"},{"key":"28","doi-asserted-by":"publisher","unstructured":"<small>El Karoui, N.<\/small> (2010). The spectrum of kernel random matrices. <i>Ann. Statist.<\/i> <b>38<\/b> 1\u201350.","DOI":"10.1214\/08-AOS648"},{"key":"67","doi-asserted-by":"crossref","unstructured":"<small>Shawe-Taylor, J.<\/small> and <small>Cristianini, N.<\/small> (2004). <i>Kernel Methods for Pattern Analysis<\/i>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511809682"},{"key":"38","doi-asserted-by":"publisher","unstructured":"<small>Hastie, T.<\/small>, <small>Tibshirani, R.<\/small> and <small>Friedman, J.<\/small> (2009). <i>The Elements of Statistical Learning<\/i>: <i>Data Mining, Inference, and Prediction<\/i>, 2nd ed. <i>Springer Series in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"61","unstructured":"<small>Rasmussen, C. E.<\/small> and <small>Williams, C. K. I.<\/small> (2006). <i>Gaussian Processes for Machine Learning<\/i>. <i>Adaptive Computation and Machine Learning<\/i>. MIT Press, Cambridge, MA."},{"key":"7","doi-asserted-by":"publisher","unstructured":"<small>BAUER, F.<\/small>, <small>PEREVERZEV, S.<\/small> and <small>ROSASCO, L.<\/small> (2007). On regularization algorithms in learning theory. <i>J. Complexity<\/i> <b>23<\/b> 52\u201372.","DOI":"10.1016\/j.jco.2006.07.001"},{"key":"23","doi-asserted-by":"publisher","unstructured":"<small>DEVORE, R.<\/small>, <small>HANIN, B.<\/small> and <small>PETROVA, G.<\/small> (2021). Neural network approximation. <i>Acta Numer.<\/i> <b>30<\/b> 327\u2013444.","DOI":"10.1017\/S0962492921000052"},{"key":"8","doi-asserted-by":"publisher","unstructured":"<small>BELKIN, M.<\/small> (2021). Fit without fear: Remarkable mathematical phenomena of deep learning through the prism of interpolation. <i>Acta Numer.<\/i> <b>30<\/b> 203\u2013248.","DOI":"10.1017\/S0962492921000039"},{"key":"54","unstructured":"<small>MISIAKIEWICZ, T.<\/small> (2022). Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2204.10425\">2204.10425<\/a> [math, Stat]."},{"key":"1","unstructured":"<small>AUBIN, B.<\/small>, <small>KRZAKALA, F.<\/small>, <small>LU, Y.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> (2020). Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization. In <i>Advances in Neural Information Processing Systems<\/i> <b>33<\/b> 12199\u201312210. Curran Associates, Red Hook."},{"key":"2","unstructured":"<small>BA, J.<\/small>, <small>ERDOGDU, M. A.<\/small>, <small>SUZUKI, T.<\/small>, <small>WANG, Z.<\/small>, <small>WU, D.<\/small> and <small>YANG, G.<\/small> (2022). High-dimensional asymptotics of feature learning: How one gradient step improves the representation. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2205.01445\">2205.01445<\/a> [cs, math, stat]."},{"key":"3","unstructured":"<small>BACH, F.<\/small> (2024). <i>Learning Theory from First Principles<\/i>. MIT Press, Cambridge."},{"key":"6","unstructured":"<small>BARZILAI, D.<\/small> and <small>SHAMIR, O.<\/small> (2024). Generalization in kernel regression under realistic assumptions. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2312.15995\">2312.15995<\/a> [cs, stat]."},{"key":"9","unstructured":"<small>BELKIN, M.<\/small>, <small>MA, S.<\/small> and <small>MANDAL, S.<\/small> (2018). To understand deep learning we need to understand kernel learning. In <i>Proceedings of the<\/i> 35<i>th International Conference on Machine Learning<\/i> 541\u2013549. PMLR. ISSN: 2640-3498."},{"key":"10","unstructured":"<small>BIETTI, A.<\/small> and <small>BACH, F. R.<\/small> (2021). Deep equals shallow for ReLU networks in kernel regimes. In 9<i>th International Conference on Learning Representations, ICLR<\/i> 2021, <i>Virtual Event, Austria, May<\/i> 3<i>\u2013<\/i>7, 2021 OpenReview.net."},{"key":"11","unstructured":"<small>BIETTI, A.<\/small>, <small>BRUNA, J.<\/small>, <small>SANFORD, C.<\/small> and <small>SONG, M. J.<\/small> (2022). Learning single-index models with shallow neural networks. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2210.15651\">2210.15651<\/a> [cs, math, stat]."},{"key":"12","unstructured":"<small>BIETTI, A.<\/small> and <small>MAIRAL, J.<\/small> (2019). On the inductive bias of neural tangent kernels. In <i>Advances in Neural Information Processing Systems<\/i> <b>32<\/b>. Curran Associates, Red Hook."},{"key":"13","unstructured":"<small>BLANCHARD, G.<\/small>, <small>MATH\u00c9, P.<\/small> and <small>M\u00dcCKE, N.<\/small> (2019). Lepskii principle in supervised learning. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1905.10764\">1905.10764<\/a> [math, stat]."},{"key":"14","unstructured":"<small>BLANCHARD, G.<\/small> and <small>M\u00dcCKE, N.<\/small> (2016). Kernel regression, minimax rates and effective dimensionality: Beyond the regular case. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1611.03979\">1611.03979<\/a> [stat]."},{"key":"15","unstructured":"<small>BOYER, C.<\/small> (2022). Living la vida loca: Learning in interpolation regimes."},{"key":"16","unstructured":"<small>BUCHHOLZ, S.<\/small> (2022). Kernel interpolation in Sobolev spaces is not consistent in low dimensions. In <i>Proceedings of Thirty Fifth Conference on Learning Theory<\/i> 3410\u20133440. PMLR. ISSN: 2640-3498."},{"key":"18","doi-asserted-by":"publisher","unstructured":"<small>CHINOT, G.<\/small>, <small>L\u00d6FFLER, M.<\/small> and <small>VAN DE GEER, S.<\/small> (2022). On the robustness of minimum norm interpolators and regularized empirical risk minimizers. <i>Ann. Statist.<\/i> <b>50<\/b> 2306\u20132333.","DOI":"10.1214\/22-aos2190"},{"key":"19","doi-asserted-by":"publisher","unstructured":"<small>CHIZAT, L.<\/small>, <small>COLOMBO, M.<\/small>, <small>FERN\u00c1NDEZ-REAL, X.<\/small> and <small>FIGALLI, A.<\/small> (2024). Infinite-width limit of deep linear neural networks. <i>Comm. Pure Appl. Math.<\/i> <b>77<\/b> 3958\u20134007.","DOI":"10.1002\/cpa.22200"},{"key":"20","unstructured":"<small>CHIZAT, L.<\/small>, <small>OYALLON, E.<\/small> and <small>BACH, F.<\/small> (2019). On lazy training in differentiable programming. In <i>Proceedings of the<\/i> 33<i>rd International Conference on Neural Information Processing Systems<\/i> <b>264<\/b> 2937\u20132947. Curran Associates, Red Hook, NY, USA."},{"key":"21","unstructured":"<small>DAMIAN, A.<\/small>, <small>LEE, J. D.<\/small> and <small>SOLTANOLKOTABI, M.<\/small> (2022). Neural networks can learn representations with gradient descent. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2206.15144\">2206.15144<\/a> [cs, math, stat]."},{"key":"22","unstructured":"<small>DANDI, Y.<\/small>, <small>KRZAKALA, F.<\/small>, <small>LOUREIRO, B.<\/small>, <small>PESCE, L.<\/small> and <small>STEPHAN, L.<\/small> (2024). How two-layer neural networks learn, one (giant) step at a time. <i>J. Mach. Learn. Res.<\/i> <b>25<\/b> Paper No. [349]."},{"key":"24","unstructured":"<small>DHIFALLAH, O.<\/small> and <small>LU, Y. M.<\/small> (2020). A precise performance analysis of learning with random features. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2008.11904\">2008.11904<\/a> [cs, math]."},{"key":"26","unstructured":"<small>DONHAUSER, K.<\/small>, <small>RUGGERI, N.<\/small>, <small>STOJANOVIC, S.<\/small> and <small>YANG, F.<\/small> (2022). Fast rates for noisy interpolation require rethinking the effect of inductive bias. In <i>Proceedings of the<\/i> 39<i>th International Conference on Machine Learning<\/i> 5397\u20135428. PMLR. ISSN: 2640-3498."},{"key":"27","unstructured":"<small>DONHAUSER, K.<\/small>, <small>WU, M.<\/small> and <small>YANG, F.<\/small> (2021). How rotational invariance of common kernels prevents generalization in high dimensions. In <i>Proceedings of the<\/i> 38<i>th International Conference on Machine Learning<\/i> 2804\u20132814. PMLR. ISSN: 2640-3498."},{"key":"30","unstructured":"<small>FISCHER, S.<\/small> and <small>STEINWART, I.<\/small> (2020). Sobolev norm learning rates for regularized least-squares algorithms. <i>J. Mach. Learn. Res.<\/i> <b>21<\/b> Paper No. 205."},{"key":"32","doi-asserted-by":"crossref","unstructured":"<small>GAVRILOPOULOS, G.<\/small>, <small>LECU\u00c9, G.<\/small> and <small>SHANG, Z.<\/small> (2025). Supplement to \u201cA geometrical analysis of kernel ridge regression and its applications.\u201d <a href=\"https:\/\/doi.org\/10.1214\/25-AOS2556SUPP\">https:\/\/doi.org\/10.1214\/25-AOS2556SUPP<\/a>","DOI":"10.1214\/25-AOS2556SUPP"},{"key":"34","doi-asserted-by":"crossref","unstructured":"<small>GOLDT, S.<\/small>, <small>M\u00c9ZARD, M.<\/small>, <small>KRZAKALA, F.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> (2020). Modelling the influence of data structure on learning in neural networks: The hidden manifold model. <i>Phys. Rev. X<\/i> <b>10<\/b> 041044. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1909.11500\">1909.11500<\/a> [cond-mat, stat].","DOI":"10.1103\/PhysRevX.10.041044"},{"key":"35","doi-asserted-by":"publisher","unstructured":"<small>GU\u00c9DON, O.<\/small>, <small>LITVAK, A. E.<\/small>, <small>PAJOR, A.<\/small> and <small>TOMCZAK-JAEGERMANN, N.<\/small> (2017). On the interval of fluctuation of the singular values of random matrices. <i>J. Eur. Math. Soc<\/i>. (<i>JEMS<\/i>) <b>19<\/b> 1469\u20131505.","DOI":"10.4171\/JEMS\/697"},{"key":"36","unstructured":"<small>HAAS, M.<\/small>, <small>HOLZM\u00dcLLER, D.<\/small>, <small>VON LUXBURG, U.<\/small> and <small>STEINWART, I.<\/small> (2023). Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2305.14077\">2305.14077<\/a> [cs, math, stat]."},{"key":"39","unstructured":"<small>JACOT, A.<\/small>, <small>GABRIEL, F.<\/small> and <small>HONGLER, C.<\/small> (2018). Neural tangent kernel: Convergence and generalization in neural networks. In <i>Proceedings of the<\/i> 32<i>nd International Conference on Neural Information Processing Systems<\/i>. <i>NIPS\u2019<\/i>18 8580\u20138589. Curran Associates, Red Hook, NY, USA."},{"key":"40","unstructured":"<small>KINGMA, D. P.<\/small> and <small>BA, J.<\/small> (2017). Adam: A method for stochastic optimization. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1412.6980\">1412.6980<\/a> [cs]."},{"key":"41","unstructured":"<small>KOEHLER, F.<\/small>, <small>ZHOU, L.<\/small>, <small>SUTHERLAND, D. J.<\/small> and <small>SREBRO, N.<\/small> (2021). Uniform convergence of interpolators: Gaussian width, norm bounds and benign overfitting."},{"key":"42","unstructured":"<small>LAX, P. D.<\/small> (2002). <i>Funct. Anal.<\/i>. Wiley\u2013Blackwell, New York."},{"key":"43","doi-asserted-by":"publisher","unstructured":"<small>LECU\u00c9, G.<\/small> and <small>SHANG, Z.<\/small> (2025). A geometrical viewpoint on the benign overfitting property of the minimum <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_1\"><msub><mrow><mi>\u2113<\/mi><\/mrow><mrow><mn>2<\/mn><\/mrow><\/msub><\/math>-norm interpolant estimator and its universality. <i>Probab. Theory Related Fields<\/i> <b>191<\/b> 1401\u20131484.","DOI":"10.1007\/s00440-024-01336-7"},{"key":"44","unstructured":"<small>LECU\u00c9, G.<\/small> and <small>SHANG, Z.<\/small> (2025). A geometric viewpoint on the benign overfitting property of the minimum norm interpolant estimator in regression and classification. In preparation."},{"key":"45","unstructured":"<small>LI, Y.<\/small>, <small>ZHANG, H.<\/small> and <small>LIN, Q.<\/small> (2023). On the asymptotic learning curves of kernel ridge regression under power-law decay. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2309.13337\">2309.13337<\/a> [cs, math, stat]."},{"key":"46","unstructured":"<small>LI, Z.<\/small> Personal communication."},{"key":"47","unstructured":"<small>LIANG, T.<\/small>, <small>RAKHLIN, A.<\/small> and <small>ZHAI, X.<\/small> (2020). On the multiple descent of minimum-norm interpolants and restricted lower isometry of kernels. In <i>Proceedings of Thirty Third Conference on Learning Theory<\/i> 2683\u20132711. PMLR. ISSN: 2640-3498."},{"key":"48","unstructured":"<small>LONG, P. M.<\/small> (2021). Properties of the after kernel. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2105.10585\">2105.10585<\/a> [cs]."},{"key":"49","unstructured":"<small>MCRAE, A. D.<\/small>, <small>KARNIK, S.<\/small>, <small>DAVENPORT, M.<\/small> and <small>MUTHUKUMAR, V. K.<\/small> (2022). Harmless interpolation in regression and classification with structured features. In <i>Proceedings of the<\/i> 25<i>th International Conference on Artificial Intelligence and Statistics<\/i> 5853\u20135875. PMLR ISSN: 2640-3498."},{"key":"53","doi-asserted-by":"publisher","unstructured":"<small>MILMAN, V.<\/small> (2023). Asymptotic geometric analysis: Achievements and perspective. In <i>Geometric Aspects of Functional Analysis<\/i> (R. Eldan, B. Klartag, A. Litvak and E. Milman, eds.). <i>Lecture Notes in Math.<\/i> <b>2327<\/b> 1\u201355. Springer, Cham.","DOI":"10.1007\/978-3-031-26300-2_1"},{"key":"55","unstructured":"<small>MONIRI, B.<\/small>, <small>LEE, D.<\/small>, <small>HASSANI, H.<\/small> and <small>DOBRIBAN, E.<\/small> (2023). A theory of non-linear feature learning with one gradient step in two-layer neural networks. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2310.07891\">2310.07891<\/a> [cs, stat]."},{"key":"56","doi-asserted-by":"publisher","unstructured":"<small>MOURTADA, J.<\/small> and <small>ROSASCO, L.<\/small> (2022). An elementary analysis of ridge regression with random design. <i>C. R. Math. Acad. Sci. Paris<\/i> <b>360<\/b> 1055\u20131063.","DOI":"10.5802\/crmath.367"},{"key":"57","unstructured":"<small>PILLAUD-VIVIEN, L.<\/small> and <small>BACH, F.<\/small> (2023). Kernelized diffusion maps. In <i>Proceedings of Thirty Sixth Conference on Learning Theory<\/i> 5236\u20135259. PMLR. ISSN: 2640-3498."},{"key":"59","unstructured":"<small>RADHAKRISHNAN, A.<\/small>, <small>BEAGLEHOLE, D.<\/small>, <small>PANDIT, P.<\/small> and <small>BELKIN, M.<\/small> (2022). Feature learning in neural networks and kernel machines that recursively learn features. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2212.13881\">2212.13881<\/a> [cs]."},{"key":"60","unstructured":"<small>RAKHLIN, A.<\/small> and <small>ZHAI, X.<\/small> (2019). Consistency of interpolation with Laplace kernels is a high-dimensional phenomenon. In <i>Conference on Learning Theory, COLT<\/i> 2019, 25<i>-<\/i>28 <i>June<\/i> 2019, <i>Phoenix, AZ, USA<\/i> (A. Beygelzimer and D. Hsu, eds.). <i>Proceedings of Machine Learning Research<\/i> <b>99<\/b> 2595\u20132623. PMLR."},{"key":"63","unstructured":"<small>SAXE, A. M.<\/small>, <small>MCCLELLAND, J. L.<\/small> and <small>GANGULI, S.<\/small> (2014). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1312.6120\">1312.6120<\/a> [cond-mat, q-bio, stat]."},{"key":"65","unstructured":"<small>SEDDIK, M. E. A.<\/small>, <small>LOUART, C.<\/small>, <small>TAMAAZOUSTI, M.<\/small> and <small>COUILLET, R.<\/small> (2020). Random matrix theory proves that deep learning representations of GAN-data behave as Gaussian mixtures. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2001.08370\">2001.08370<\/a> [cs, stat]."},{"key":"66","unstructured":"<small>SHAMIR, O.<\/small> (2022). The implicit bias of benign overfitting. In <i>Proceedings of Thirty Fifth Conference on Learning Theory<\/i> 448\u2013478. PMLR ISSN: 2640-3498."},{"key":"68","doi-asserted-by":"crossref","unstructured":"<small>STEINWART, I.<\/small> and <small>CHRISTMANN, A.<\/small> (2008). <i>Support Vector Machines<\/i>. <i>Information Science and Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-77242-4"},{"key":"69","unstructured":"<small>STEINWART, I.<\/small>, <small>HUSH, D. R.<\/small> and <small>SCOVEL, C.<\/small> (2009). Optimal rates for regularized least squares regression. In <i>COLT<\/i> 79\u201393."},{"key":"70","unstructured":"<small>SUZUKI, T.<\/small> (2018). Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: Optimal rate and curse of dimensionality."},{"key":"72","unstructured":"<small>TSIGLER, A.<\/small> and <small>BARTLETT, P. L.<\/small> (2023). Benign overfitting in ridge regression. <i>J. Mach. Learn. Res.<\/i> <b>24<\/b> Paper No. [123]."},{"key":"73","doi-asserted-by":"publisher","unstructured":"<small>VERSHYNIN, R.<\/small> (2018). <i>High-Dimensional Probability<\/i>: <i>An Introduction with Applications in Data Science, with a Foreword by Sara van de Geer<\/i>. <i>Cambridge Series in Statistical and Probabilistic Mathematics<\/i> <b>47<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/9781108231596"},{"key":"75","unstructured":"<small>WANG, G.<\/small>, <small>DONHAUSER, K.<\/small> and <small>YANG, F.<\/small> (2022). Tight bounds for minimum <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_2\"><msub><mrow><mi>\u2113<\/mi><\/mrow><mrow><mn>1<\/mn><\/mrow><\/msub><\/math>-norm interpolation of noisy data. In <i>Proceedings of the<\/i> 25<i>th International Conference on Artificial Intelligence and Statistics<\/i> 10572\u201310602. PMLR. ISSN: 2640-3498."},{"key":"76","unstructured":"<small>WOODWORTH, B.<\/small>, <small>GUNASEKAR, S.<\/small>, <small>LEE, J. D.<\/small>, <small>MOROSHKO, E.<\/small>, <small>SAVARESE, P.<\/small>, <small>GOLAN, I.<\/small>, <small>SOUDRY, D.<\/small> and <small>SREBRO, N.<\/small> (2020). Kernel and rich regimes in overparametrized models. In <i>Proceedings of Thirty Third Conference on Learning Theory<\/i> 3635\u20133673. PMLR. ISSN: 2640-3498."},{"key":"77","doi-asserted-by":"crossref","unstructured":"<small>YANG, G.<\/small> (2020). Tensor programs II: Neural tangent kernel for any architecture. <i>CoRR<\/i>. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2006.14548\">2006.14548<\/a>.","DOI":"10.1200\/jco.2006.24.18_suppl.14548"},{"key":"79","unstructured":"<small>ZHANG, C.<\/small>, <small>BENGIO, S.<\/small>, <small>HARDT, M.<\/small>, <small>RECHT, B.<\/small> and <small>VINYALS, O.<\/small> (2017). Understanding deep learning requires rethinking generalization. In 5<i>th International Conference on Learning Representations, ICLR<\/i> 2017, <i>Toulon, France, April<\/i> 24<i>-<\/i>26, 2017, <i>Conference Track Proceedings<\/i> OpenReview.net."},{"key":"80","unstructured":"<small>ZHANG, H.<\/small>, <small>LI, Y.<\/small> and <small>LIN, Q.<\/small> (2024). On the optimality of misspecified spectral algorithms. <i>J. Mach. Learn. Res.<\/i> <b>25<\/b> Paper No. [188]."}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T16:22:55Z","timestamp":1791390175000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-53\/issue-6\/A-geometrical-analysis-of-kernel-ridge-regression-and-its-applications\/10.1214\/25-AOS2556.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":80,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,12,1]]}},"URL":"https:\/\/doi.org\/10.1214\/25-aos2556","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,1]]}}}