{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T14:13:35Z","timestamp":1792246415430,"version":"4.4.0"},"reference-count":94,"publisher":"Institute of Mathematical Statistics","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2025,4,1]]},"abstract":"<jats:p>Modern machine learning classifiers often exhibit vanishing classification error on the training set. They achieve this by learning nonlinear representations of the inputs that map the data into linearly separable classes. Motivated by these phenomena, we revisit high-dimensional maximum margin classification for linearly separable data. We consider a stylized setting in which data (yi,xi), i\u2264n are i.i.d. with xi\u223cN(0,\u03a3) a p-dimensional Gaussian feature vector, and yi\u2208{+1,\u22121} a label whose distribution depends on a linear combination of the covariates \u27e8\u03b8\u2217,xi\u27e9. Recent universality results can be used to show that the results derived in the Gaussian setting also apply when xi=\u03d5(zi) for standard Gaussian zi and \u03d5 a nonlinear featurization map. We consider the proportional asymptotics n,p\u2192\u221e with p\/n\u2192\u03c8, and derive exact expressions for the limiting generalization error. We use this theory to derive two results of independent interest: (i) Sufficient conditions on (\u03a3,\u03b8\u2217) for \u201cbenign overfitting\u201d that are parallel to previously derived conditions in the case of linear regression; (ii) An asymptotically exact expression for the generalization error when max-margin classification is used in conjunction with feature vectors produced by random one-layer neural networks.<\/jats:p>","DOI":"10.1214\/25-aos2489","type":"journal-article","created":{"date-parts":[[2025,4,30]],"date-time":"2025-04-30T01:55:09Z","timestamp":1745978109000},"page":"822-853","source":"Crossref","is-referenced-by-count":10,"title":["The generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regime"],"prefix":"10.1214","volume":"53","author":[{"given":"Andrea","family":"Montanari","sequence":"first","affiliation":[{"name":"Department of Statistics and Department of Electrical Engineering, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feng","family":"Ruan","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, Northwestern University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youngtak","family":"Sohn","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Massachusetts Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Yan","sequence":"additional","affiliation":[{"name":"D. E. Shaw & Co., L.P"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"8","doi-asserted-by":"publisher","unstructured":"<small>Barbier, J.<\/small> and <small>Macris, N.<\/small> (2019). The adaptive interpolation method: A simple scheme to prove replica formulas in Bayesian inference. <i>Probab. Theory Related Fields<\/i> <b>174<\/b> 1133\u20131185.","DOI":"10.1007\/s00440-018-0879-0"},{"key":"14","doi-asserted-by":"publisher","unstructured":"<small>Bayati, M.<\/small> and <small>Montanari, A.<\/small> (2011). The dynamics of message passing on dense graphs, with applications to compressed sensing. <i>IEEE Trans. Inf. Theory<\/i> <b>57<\/b> 764\u2013785.","DOI":"10.1109\/TIT.2010.2094817"},{"key":"28","doi-asserted-by":"publisher","unstructured":"<small>Deshpande, Y.<\/small>, <small>Abbe, E.<\/small> and <small>Montanari, A.<\/small> (2017). Asymptotic mutual information for the balanced binary stochastic block model. <i>Inf. Inference<\/i> <b>6<\/b> 125\u2013170.","DOI":"10.1093\/imaiai\/iaw017"},{"key":"64","doi-asserted-by":"publisher","unstructured":"<small>M\u00e9zard, M.<\/small> and <small>Montanari, A.<\/small> (2009). <i>Information, Physics, and Computation<\/i>. <i>Oxford Graduate Texts<\/i>. Oxford Univ. Press, Oxford.","DOI":"10.1093\/acprof:oso\/9780198570837.001.0001"},{"key":"65","unstructured":"<small>M\u00e9zard, M.<\/small>, <small>Parisi, G.<\/small> and <small>Virasoro, M. A.<\/small> (1987). <i>Spin Glass Theory and Beyond<\/i>. <i>World Scientific Lecture Notes in Physics<\/i> <b>9<\/b>. World Scientific Co., Inc., Teaneck, NJ."},{"key":"46","doi-asserted-by":"publisher","unstructured":"<small>HOFMANN, T.<\/small>, <small>SCH\u00d6LKOPF, B.<\/small> and <small>SMOLA, A. J.<\/small> (2008). Kernel methods in machine learning. <i>Ann. Statist.<\/i> <b>36<\/b> 1171\u20131220.","DOI":"10.1214\/009053607000000677"},{"key":"67","doi-asserted-by":"crossref","unstructured":"<small>MONTANARI, A.<\/small> (2018). Mean field asymptotics in high-dimensional statistics: From exact results to efficient algorithms. In <i>Proceedings of the International Congress of Mathematicians\u2014Rio de Janeiro<\/i> 2018. <i>Vol. IV. Invited Lectures<\/i> 2973\u20132994. World Sci. Publ., Hackensack, NJ.","DOI":"10.1142\/9789813272880_0168"},{"key":"81","doi-asserted-by":"publisher","unstructured":"<small>Shcherbina, M.<\/small> and <small>Tirozzi, B.<\/small> (2003). Rigorous solution of the Gardner problem. <i>Comm. Math. Phys.<\/i> <b>234<\/b> 383\u2013422.","DOI":"10.1007\/s00220-002-0783-3"},{"key":"1","doi-asserted-by":"publisher","unstructured":"<small>Amelunxen, D.<\/small>, <small>Lotz, M.<\/small>, <small>McCoy, M. B.<\/small> and <small>Tropp, J. A.<\/small> (2014). Living on the edge: Phase transitions in convex programs with random data. <i>Inf. Inference<\/i> <b>3<\/b> 224\u2013294.","DOI":"10.1093\/imaiai\/iau005"},{"key":"15","doi-asserted-by":"publisher","unstructured":"<small>Bayati, M.<\/small> and <small>Montanari, A.<\/small> (2012). The LASSO risk for Gaussian matrices. <i>IEEE Trans. Inf. Theory<\/i> <b>58<\/b> 1997\u20132017.","DOI":"10.1109\/TIT.2011.2174612"},{"key":"32","doi-asserted-by":"publisher","unstructured":"<small>El Karoui, N.<\/small> (2018). On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators. <i>Probab. Theory Related Fields<\/i> <b>170<\/b> 95\u2013175.","DOI":"10.1007\/s00440-016-0754-9"},{"key":"85","doi-asserted-by":"publisher","unstructured":"<small>Sur, P.<\/small> and <small>Cand\u00e8s, E. J.<\/small> (2019). A modern maximum-likelihood theory for high-dimensional logistic regression. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>116<\/b> 14516\u201314525.","DOI":"10.1073\/pnas.1810420116"},{"key":"87","doi-asserted-by":"publisher","unstructured":"<small>Thrampoulidis, C.<\/small>, <small>Abbasi, E.<\/small> and <small>Hassibi, B.<\/small> (2018). Precise error analysis of regularized <i>M<\/i>-estimators in high dimensions. <i>IEEE Trans. Inf. Theory<\/i> <b>64<\/b> 5592\u20135628.","DOI":"10.1109\/TIT.2018.2840720"},{"key":"88","unstructured":"<small>Thrampoulidis, C.<\/small>, <small>Oymak, S.<\/small> and <small>Hassibi, B.<\/small> (2015). Regularized linear regression: A precise analysis of the estimation error. In <i>Conference on Learning Theory<\/i> 1683\u20131709."},{"key":"13","doi-asserted-by":"publisher","unstructured":"<small>Bayati, M.<\/small>, <small>Lelarge, M.<\/small> and <small>Montanari, A.<\/small> (2015). Universality in polytope phase transitions and message passing algorithms. <i>Ann. Appl. Probab.<\/i> <b>25<\/b> 753\u2013822.","DOI":"10.1214\/14-AAP1010"},{"key":"7","doi-asserted-by":"publisher","unstructured":"<small>BARBIER, J.<\/small>, <small>KRZAKALA, F.<\/small>, <small>MACRIS, N.<\/small>, <small>MIOLANE, L.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> (2019). Optimal errors and phase transitions in high-dimensional generalized linear models. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>116<\/b> 5451\u20135460.","DOI":"10.1073\/pnas.1802705116"},{"key":"21","doi-asserted-by":"publisher","unstructured":"<small>CHANDRASEKARAN, V.<\/small>, <small>RECHT, B.<\/small>, <small>PARRILO, P. A.<\/small> and <small>WILLSKY, A. S.<\/small> (2012). The convex geometry of linear inverse problems. <i>Found. Comput. Math.<\/i> <b>12<\/b> 805\u2013849.","DOI":"10.1007\/s10208-012-9135-7"},{"key":"66","doi-asserted-by":"publisher","unstructured":"<small>MIOLANE, L.<\/small> and <small>MONTANARI, A.<\/small> (2021). The distribution of the Lasso: Uniform control over sparse balls and adaptive parameter tuning. <i>Ann. Statist.<\/i> <b>49<\/b> 2313\u20132335.","DOI":"10.1214\/20-aos2038"},{"key":"11","doi-asserted-by":"publisher","unstructured":"<small>BARTLETT, P. L.<\/small>, <small>LONG, P. M.<\/small>, <small>LUGOSI, G.<\/small> and <small>TSIGLER, A.<\/small> (2020). Benign overfitting in linear regression. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>117<\/b> 30063\u201330070.","DOI":"10.1073\/pnas.1907378117"},{"key":"17","doi-asserted-by":"publisher","unstructured":"<small>BELKIN, M.<\/small>, <small>HSU, D.<\/small> and <small>XU, J.<\/small> (2020). Two models of double descent for weak features. <i>SIAM J. Math. Data Sci.<\/i> <b>2<\/b> 1167\u20131180.","DOI":"10.1137\/20M1336072"},{"key":"35","doi-asserted-by":"publisher","unstructured":"<small>FAN, Z.<\/small> and <small>MONTANARI, A.<\/small> (2019). The spectral norm of random inner-product kernel matrices. <i>Probab. Theory Related Fields<\/i> <b>173<\/b> 27\u201385.","DOI":"10.1007\/s00440-018-0830-4"},{"key":"10","doi-asserted-by":"publisher","unstructured":"<small>BARTLETT, P. L.<\/small>, <small>JORDAN, M. I.<\/small> and <small>MCAULIFFE, J. D.<\/small> (2006). Convexity, classification, and risk bounds. <i>J. Amer. Statist. Assoc.<\/i> <b>101<\/b> 138\u2013156.","DOI":"10.1198\/016214505000000907"},{"key":"20","doi-asserted-by":"publisher","unstructured":"<small>CAND\u00c8S, E. J.<\/small> and <small>SUR, P.<\/small> (2020). The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. <i>Ann. Statist.<\/i> <b>48<\/b> 27\u201342.","DOI":"10.1214\/18-AOS1789"},{"key":"26","doi-asserted-by":"crossref","unstructured":"<small>COVER, T. M.<\/small> (1965). Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. <i>IEEE Trans. Electron. Comput.<\/i> <b>3<\/b> 326\u2013334.","DOI":"10.1109\/PGEC.1965.264137"},{"key":"38","doi-asserted-by":"crossref","unstructured":"<small>GARDNER, E.<\/small> (1988). The space of interactions in neural network models. <i>J. Phys. A<\/i> <b>21<\/b> 257\u2013270.","DOI":"10.1088\/0305-4470\/21\/1\/030"},{"key":"56","doi-asserted-by":"publisher","unstructured":"<small>KOLTCHINSKII, V.<\/small> and <small>PANCHENKO, D.<\/small> (2002). Empirical margin distributions and bounding the generalization error of combined classifiers. <i>Ann. Statist.<\/i> <b>30<\/b> 1\u201350.","DOI":"10.1214\/aos\/1015362182"},{"key":"45","doi-asserted-by":"publisher","unstructured":"<small>HASTIE, T.<\/small>, <small>MONTANARI, A.<\/small>, <small>ROSSET, S.<\/small> and <small>TIBSHIRANI, R. J.<\/small> (2022). Surprises in high-dimensional ridgeless least squares interpolation. <i>Ann. Statist.<\/i> <b>50<\/b> 949\u2013986.","DOI":"10.1214\/21-aos2133"},{"key":"62","doi-asserted-by":"publisher","unstructured":"<small>MEI, S.<\/small> and <small>MONTANARI, A.<\/small> (2022). The generalization error of random features regression: Precise asymptotics and the double descent curve. <i>Comm. Pure Appl. Math.<\/i> <b>75<\/b> 667\u2013766.","DOI":"10.1002\/cpa.22008"},{"key":"63","doi-asserted-by":"publisher","unstructured":"<small>MEI, S.<\/small> and <small>MONTANARI, A.<\/small> (2022). The generalization error of random features regression: Precise asymptotics and the double descent curve. <i>Comm. Pure Appl. Math.<\/i> <b>75<\/b> 667\u2013766.","DOI":"10.1002\/cpa.22008"},{"key":"22","unstructured":"<small>CHATTERJI, N. S.<\/small> and <small>LONG, P. M.<\/small> (2021). Finite-sample analysis of interpolating linear classifiers in the overparameterized regime. <i>J. Mach. Learn. Res.<\/i> <b>22<\/b> Paper No. 129, 30 pp."},{"key":"61","doi-asserted-by":"publisher","unstructured":"<small>LIANG, T.<\/small> and <small>SUR, P.<\/small> (2022). A precise high-dimensional asymptotic theory for boosting and minimum-<i>\u2113<\/i>1-norm interpolated classifiers. <i>Ann. Statist.<\/i> <b>50<\/b> 1669\u20131695.","DOI":"10.1214\/22-aos2170"},{"key":"72","unstructured":"<small>MUTHUKUMAR, V.<\/small>, <small>NARANG, A.<\/small>, <small>SUBRAMANIAN, V.<\/small>, <small>BELKIN, M.<\/small>, <small>HSU, D.<\/small> and <small>SAHAI, A.<\/small> (2021). Classification vs regression in overparameterized regimes: Does the loss function matter? <i>J. Mach. Learn. Res.<\/i> <b>22<\/b> Paper No. 222, 69 pp."},{"key":"82","unstructured":"<small>SOUDRY, D.<\/small>, <small>HOFFER, E.<\/small>, <small>NACSON, M. S.<\/small>, <small>GUNASEKAR, S.<\/small> and <small>SREBRO, N.<\/small> (2018). The implicit bias of gradient descent on separable data. <i>J. Mach. Learn. Res.<\/i> <b>19<\/b> Paper No. 70, 57 pp."},{"key":"27","doi-asserted-by":"publisher","unstructured":"<small>DENG, Z.<\/small>, <small>KAMMOUN, A.<\/small> and <small>THRAMPOULIDIS, C.<\/small> (2022). A model of double descent for high-dimensional binary linear classification. <i>Inf. Inference<\/i> <b>11<\/b> 435\u2013495.","DOI":"10.1093\/imaiai\/iaab002"},{"key":"50","doi-asserted-by":"publisher","unstructured":"<small>JAVANMARD, A.<\/small> and <small>SOLTANOLKOTABI, M.<\/small> (2022). Precise statistical analysis of classification accuracies for adversarial training. <i>Ann. Statist.<\/i> <b>50<\/b> 2127\u20132156.","DOI":"10.1214\/22-aos2180"},{"key":"12","doi-asserted-by":"publisher","unstructured":"<small>BARTLETT, P. L.<\/small> and <small>MENDELSON, S.<\/small> (2002). Rademacher and Gaussian complexities: Risk bounds and structural results. <i>J. Mach. Learn. Res.<\/i> <b>3<\/b> 463\u2013482.","DOI":"10.1162\/153244303321897690"},{"key":"48","doi-asserted-by":"publisher","unstructured":"<small>HU, H.<\/small> and <small>LU, Y. M.<\/small> (2023). Universality laws for high-dimensional learning with random features. <i>IEEE Trans. Inf. Theory<\/i> <b>69<\/b> 1932\u20131964.","DOI":"10.1109\/tit.2022.3217698"},{"key":"4","doi-asserted-by":"publisher","unstructured":"<small>Bai, Z.<\/small> and <small>Silverstein, J. W.<\/small> (2010). <i>Spectral Analysis of Large Dimensional Random Matrices<\/i>, 2nd ed. <i>Springer Series in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-1-4419-0661-8"},{"key":"80","doi-asserted-by":"crossref","unstructured":"<small>Shalev-Shwartz, S.<\/small> and <small>Ben-David, S.<\/small> (2014). <i>Understanding Machine Learning<\/i>: <i>From Theory to Algorithms<\/i>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9781107298019"},{"key":"2","doi-asserted-by":"publisher","unstructured":"<small>Anthony, M.<\/small> and <small>Bartlett, P. L.<\/small> (2009). <i>Neural Network Learning<\/i>: <i>Theoretical Foundations<\/i>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511624216"},{"key":"9","doi-asserted-by":"publisher","unstructured":"<small>Bartlett, P. L.<\/small> (1998). The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network. <i>IEEE Trans. Inf. Theory<\/i> <b>44<\/b> 525\u2013536.","DOI":"10.1109\/18.661502"},{"key":"16","doi-asserted-by":"publisher","unstructured":"<small>Belkin, M.<\/small>, <small>Hsu, D.<\/small>, <small>Ma, S.<\/small> and <small>Mandal, S.<\/small> (2019). Reconciling modern machine-learning practice and the classical bias-variance trade-off. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>116<\/b> 15849\u201315854.","DOI":"10.1073\/pnas.1903070116"},{"key":"58","doi-asserted-by":"publisher","unstructured":"<small>Lelarge, M.<\/small> and <small>Miolane, L.<\/small> (2019). Fundamental limits of symmetric low-rank matrix estimation. <i>Probab. Theory Related Fields<\/i> <b>173<\/b> 859\u2013929.","DOI":"10.1007\/s00440-018-0845-x"},{"key":"29","doi-asserted-by":"publisher","unstructured":"<small>Donoho, D.<\/small> and <small>Montanari, A.<\/small> (2016). High dimensional robust M-estimation: Asymptotic variance via approximate message passing. <i>Probab. Theory Related Fields<\/i> <b>166<\/b> 935\u2013969.","DOI":"10.1007\/s00440-015-0675-z"},{"key":"33","doi-asserted-by":"crossref","unstructured":"<small>El Karoui, N.<\/small>, <small>Bean, D.<\/small>, <small>Bickel, P. J.<\/small>, <small>Lim, C.<\/small> and <small>Yu, B.<\/small> (2013). On robust regression with high-dimensional predictors. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>110<\/b> 14557\u201314562.","DOI":"10.1073\/pnas.1307842110"},{"key":"60","doi-asserted-by":"publisher","unstructured":"<small>Liang, T.<\/small> and <small>Rakhlin, A.<\/small> (2020). Just interpolate: Kernel \u201cRidgeless\u201d regression can generalize. <i>Ann. Statist.<\/i> <b>48<\/b> 1329\u20131347.","DOI":"10.1214\/19-AOS1849"},{"key":"55","doi-asserted-by":"publisher","unstructured":"<small>Koltchinskii, V.<\/small> (2011). <i>Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems<\/i>. <i>Lecture Notes in Math.<\/i> <b>2033<\/b>. Springer, Heidelberg.","DOI":"10.1007\/978-3-642-22147-7"},{"key":"30","doi-asserted-by":"publisher","unstructured":"<small>Donoho, D. L.<\/small>, <small>Johnstone, I.<\/small> and <small>Montanari, A.<\/small> (2013). Accurate prediction of phase transitions in compressed sensing via a connection to minimax denoising. <i>IEEE Trans. Inf. Theory<\/i> <b>59<\/b> 3396\u20133433.","DOI":"10.1109\/TIT.2013.2239356"},{"key":"90","doi-asserted-by":"publisher","unstructured":"<small>Vershynin, R.<\/small> (2018). <i>High-Dimensional Probability<\/i>: <i>An Introduction with Applications in Data Science<\/i>. <i>Cambridge Series in Statistical and Probabilistic Mathematics<\/i> <b>47<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/9781108231596"},{"key":"91","doi-asserted-by":"publisher","unstructured":"<small>Villani, C.<\/small> (2009). <i>Optimal Transport<\/i>: <i>Old and New<\/i>. <i>Grundlehren der Mathematischen Wissenschaften<\/i> [<i>Fundamental Principles of Mathematical Sciences<\/i>] <b>338<\/b>. Springer, Berlin.","DOI":"10.1007\/978-3-540-71050-9"},{"key":"92","doi-asserted-by":"publisher","unstructured":"<small>WAHBA, G.<\/small> (2002). Soft and hard classification by reproducing kernel Hilbert space methods. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>99<\/b> 16524\u201316530.","DOI":"10.1073\/pnas.242574899"},{"key":"71","doi-asserted-by":"publisher","unstructured":"<small>MONTANARI, A.<\/small> and <small>ZHONG, Y.<\/small> (2022). The interpolation phase transition in neural networks: Memorization and generalization under lazy training. <i>Ann. Statist.<\/i> <b>50<\/b> 2816\u20132847.","DOI":"10.1214\/22-aos2211"},{"key":"70","unstructured":"<small>MONTANARI, A.<\/small> and <small>SAEED, B. N.<\/small> (2022). Universality of empirical risk minimization. In <i>Conference on Learning Theory<\/i> 4310\u20134312. PMLR."},{"key":"54","unstructured":"<small>KOEHLER, F.<\/small>, <small>ZHOU, L.<\/small>, <small>SUTHERLAND, D. J.<\/small> and <small>SREBRO, N.<\/small> (2021). Uniform convergence of interpolators: Gaussian width, norm bounds and benign overfitting. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>34<\/b> 20657\u201320668."},{"key":"84","unstructured":"<small>STOJNIC, M.<\/small> (2013). A framework to characterize performance of lasso algorithms. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1303.7291\">1303.7291<\/a>."},{"key":"3","unstructured":"<small>ARORA, S.<\/small>, <small>COHEN, N.<\/small>, <small>HU, W.<\/small> and <small>LUO, Y.<\/small> (2019). Implicit regularization in deep matrix factorization. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1905.13655\">1905.13655<\/a>."},{"key":"5","doi-asserted-by":"crossref","unstructured":"<small>BALCAN, M.-F.<\/small>, <small>BLUM, A.<\/small> and <small>VEMPALA, S.<\/small> (2004). Kernels as features: On kernels, margins, and low-dimensional mappings. <i>Mach. Learn.<\/i> <b>65<\/b> 79\u201394.","DOI":"10.1007\/s10994-006-7550-1"},{"key":"6","unstructured":"<small>BARBIER, J.<\/small>, <small>DIA, M.<\/small>, <small>MACRIS, N.<\/small>, <small>KRZAKALA, F.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> (2018). Rank-one matrix estimation: Analysis of algorithmic and information theoretic limits by the spatial coupling method. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1812.02537\">1812.02537<\/a>."},{"key":"18","unstructured":"<small>BELKIN, M.<\/small>, <small>HSU, D. J.<\/small> and <small>MITRA, P.<\/small> (2018). Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate. <i>Adv. Neural Inf. Process. Syst.<\/i> 2300\u20132311."},{"key":"19","unstructured":"<small>BELKIN, M.<\/small>, <small>MA, S.<\/small> and <small>MANDAL, S.<\/small> (2018). To understand deep learning we need to understand kernel learning. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1802.01396\">1802.01396<\/a>."},{"key":"23","doi-asserted-by":"publisher","unstructured":"<small>CHENG, C.<\/small> and <small>MONTANARI, A.<\/small> (2024). Dimension free ridge regression. <i>Ann. Statist.<\/i> <b>52<\/b> 2879\u20132912.","DOI":"10.1214\/24-aos2449"},{"key":"24","doi-asserted-by":"publisher","unstructured":"<small>CHENG, X.<\/small> and <small>SINGER, A.<\/small> (2013). The spectrum of random inner-product kernel matrices. <i>Random Matrices Theory Appl.<\/i> <b>2<\/b> 1350010, 47 pp.","DOI":"10.1142\/S201032631350010X"},{"key":"25","unstructured":"<small>CHIZAT, L.<\/small> and <small>BACH, F.<\/small> (2018). A note on lazy training in supervised differentiable programming. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1812.07956\">1812.07956<\/a>."},{"key":"31","unstructured":"<small>DU, S. S.<\/small>, <small>ZHAI, X.<\/small>, <small>POCZOS, B.<\/small> and <small>SINGH, A.<\/small> (2018). Gradient descent provably optimizes over-parameterized neural networks. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1810.02054\">1810.02054<\/a>."},{"key":"34","doi-asserted-by":"publisher","unstructured":"<small>ENGEL, A.<\/small> and <small>VAN DEN BROECK, C.<\/small> (2001). <i>Statistical Mechanics of Learning<\/i>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9781139164542"},{"key":"36","unstructured":"<small>FREI, S.<\/small>, <small>CHATTERJI, N. S.<\/small> and <small>BARTLETT, P.<\/small> (2022). Benign overfitting without linearity: Neural network classifiers trained by gradient descent for noisy linear data. In <i>Conference on Learning Theory<\/i> 2668\u20132703. PMLR."},{"key":"37","unstructured":"<small>FREI, S.<\/small>, <small>VARDI, G.<\/small>, <small>BARTLETT, P. L.<\/small> and <small>SREBRO, N.<\/small> (2023). Benign overfitting in linear classifiers and leaky relu networks from kkt conditions for margin maximization. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2303.01462\">2303.01462<\/a>."},{"key":"39","doi-asserted-by":"publisher","unstructured":"<small>GEIGER, M.<\/small>, <small>JACOT, A.<\/small>, <small>SPIGLER, S.<\/small>, <small>GABRIEL, F.<\/small>, <small>SAGUN, L.<\/small>, <small>D\u2019ASCOLI, S.<\/small>, <small>BIROLI, G.<\/small>, <small>HONGLER, C.<\/small> and <small>WYART, M.<\/small> (2020). Scaling description of generalization with number of parameters in deep learning. <i>J. Stat. Mech. Theory Exp.<\/i> <b>2020<\/b> 023401, 23 pp.","DOI":"10.1088\/1742-5468\/ab633c"},{"key":"40","doi-asserted-by":"publisher","unstructured":"<small>GERACE, F.<\/small>, <small>LOUREIRO, B.<\/small>, <small>KRZAKALA, F.<\/small>, <small>M\u00c9ZARD, M.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> (2021). Generalisation error in learning with random features and the hidden manifold model. <i>J. Stat. Mech. Theory Exp.<\/i> <b>2021<\/b> Paper No. 124013, 40 pp.","DOI":"10.1088\/1742-5468\/ac3ae6"},{"key":"41","unstructured":"<small>GOLDT, S.<\/small>, <small>MEZARD, M.<\/small>, <small>KRZAKALA, F.<\/small> and <small>ZDEBOROV\u00c1, L.<\/small> Modelling the influence of data structure on learning in neural networks: the hidden manifold model."},{"key":"42","doi-asserted-by":"publisher","unstructured":"<small>GORDON, Y.<\/small> (1988). On Milman\u2019s inequality and random subspaces which escape through a mesh in <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_11\"><msup><mrow><mi mathvariant=\"bold\">R<\/mi><\/mrow><mrow><mi mathvariant=\"italic\">n<\/mi><\/mrow><\/msup><\/math>. In <i>Geometric Aspects of Functional Analysis<\/i> (1986\/87). <i>Lecture Notes in Math.<\/i> <b>1317<\/b> 84\u2013106. Springer, Berlin.","DOI":"10.1007\/BFb0081737"},{"key":"43","unstructured":"<small>GUNASEKAR, S.<\/small>, <small>LEE, J.<\/small>, <small>SOUDRY, D.<\/small> and <small>SREBRO, N.<\/small> (2018). Characterizing implicit bias in terms of optimization geometry. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1802.08246\">1802.08246<\/a>."},{"key":"44","unstructured":"<small>GUNASEKAR, S.<\/small>, <small>LEE, J. D.<\/small>, <small>SOUDRY, D.<\/small> and <small>SREBRO, N.<\/small> (2018). Implicit bias of gradient descent on linear convolutional networks. <i>Adv. Neural Inf. Process. Syst.<\/i> 9461\u20139471."},{"key":"47","doi-asserted-by":"publisher","unstructured":"<small>HSU, D.<\/small>, <small>MUTHUKUMAR, V.<\/small> and <small>XU, J.<\/small> (2022). On the proliferation of support vectors in high dimensions. <i>J. Stat. Mech. Theory Exp.<\/i> <b>2022<\/b> Paper No. 114011, 27 pp.","DOI":"10.1088\/1742-5468\/ac98a9"},{"key":"49","unstructured":"<small>JACOT, A.<\/small>, <small>GABRIEL, F.<\/small> and <small>HONGLER, C.<\/small> (2018). Neural tangent kernel: Convergence and generalization in neural networks. <i>Adv. Neural Inf. Process. Syst.<\/i> 8571\u20138580."},{"key":"51","unstructured":"<small>KAKADE, S. M.<\/small>, <small>SRIDHARAN, K.<\/small> and <small>TEWARI, A.<\/small> (2009). On the complexity of linear prediction: Risk bounds, margin bounds, and regularization. <i>Adv. Neural Inf. Process. Syst.<\/i> 793\u2013800."},{"key":"52","unstructured":"<small>KINI, G. R.<\/small>, <small>PARASKEVAS, O.<\/small>, <small>OYMAK, S.<\/small> and <small>THRAMPOULIDIS, C.<\/small> (2021). Label-imbalanced and group-sensitive classification under overparameterization. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>34<\/b> 18970\u201318983."},{"key":"53","doi-asserted-by":"crossref","unstructured":"<small>KINI, G. R.<\/small> and <small>THRAMPOULIDIS, C.<\/small> (2020). Analytic study of double descent in binary classification: The impact of loss. In 2020 <i>IEEE International Symposium on Information Theory<\/i> (<i>ISIT<\/i>) 2527\u20132532. IEEE.","DOI":"10.1109\/ISIT44484.2020.9174344"},{"key":"57","unstructured":"<small>LECU\u00c9, G.<\/small>, <small>SHANG, Z.<\/small> and <small>SHANG, Z.<\/small> (2022). A geometrical viewpoint on the benign overfitting property of the minimum <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_12\"><msub><mrow><mi mathvariant=\"italic\">l<\/mi><\/mrow><mrow><mn>2<\/mn><\/mrow><\/msub><\/math>-norm interpolant estimator and its universality. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2203.05873\">2203.05873<\/a>."},{"key":"59","unstructured":"<small>LI, Y.<\/small>, <small>MA, T.<\/small> and <small>ZHANG, H.<\/small> (2017). Algorithmic regularization in over-parameterized matrix sensing and neural networks with quadratic activations. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1712.09203\">1712.09203<\/a>."},{"key":"68","unstructured":"<small>MONTANARI, A.<\/small>, <small>RUAN, F.<\/small>, <small>SAEED, B.<\/small> and <small>SOHN, Y.<\/small> (2023). Universality of max-margin classifiers. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2310.00176\">2310.00176<\/a>."},{"key":"69","doi-asserted-by":"crossref","unstructured":"<small>MONTANARI, A.<\/small>, <small>RUAN, F.<\/small>, <small>SOHN, Y.<\/small> and <small>YAN, J.<\/small> (2025). Supplement to \u201cThe generalization error of max-margin linear classifiers: Benign overfitting and high dimensional asymptotics in the overparametrized regime.\u201d <a href=\"https:\/\/doi.org\/10.1214\/25-AOS2489SUPP\">https:\/\/doi.org\/10.1214\/25-AOS2489SUPP<\/a>","DOI":"10.1214\/25-AOS2489"},{"key":"73","unstructured":"<small>MUTHUKUMAR, V.<\/small>, <small>VODRAHALLI, K.<\/small> and <small>SAHAI, A.<\/small> (2019). Harmless interpolation of noisy data in regression. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1903.09139\">1903.09139<\/a>."},{"key":"74","unstructured":"<small>NEYSHABUR, B.<\/small>, <small>BHOJANAPALLI, S.<\/small>, <small>MCALLESTER, D.<\/small> and <small>SREBRO, N.<\/small> (2017). Exploring generalization in deep learning. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>30<\/b>."},{"key":"75","doi-asserted-by":"crossref","unstructured":"<small>OYMAK, S.<\/small>, <small>THRAMPOULIDIS, C.<\/small> and <small>HASSIBI, B.<\/small> The squared-error of generalized lasso: A precise analysis. In 2013 51<i>st Annual Allerton Conference on Communication, Control, and Computing<\/i> (<i>Allerton<\/i>) 1002\u20131009.","DOI":"10.1109\/Allerton.2013.6736635"},{"key":"76","doi-asserted-by":"crossref","unstructured":"<small>RADFORD, M. N.<\/small> (1996). Priors for infinite networks. In <i>Bayesian Learning for Neural Networks<\/i> 29\u201353. Springer, Berlin.","DOI":"10.1007\/978-1-4612-0745-0_2"},{"key":"77","unstructured":"<small>RAHIMI, A.<\/small> and <small>RECHT, B.<\/small> (2008). Random features for large-scale kernel machines. <i>Adv. Neural Inf. Process. Syst.<\/i> 1177\u20131184."},{"key":"78","unstructured":"<small>RAKHLIN, A.<\/small> and <small>ZHAI, X.<\/small> (2018). Consistency of interpolation with Laplace kernels is a high-dimensional phenomenon. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1812.11167\">1812.11167<\/a>."},{"key":"79","unstructured":"<small>SALEHI, F.<\/small>, <small>ABBASI, E.<\/small> and <small>HASSIBI, B.<\/small> (2019). The impact of regularization on high-dimensional logistic regression. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1906.03761\">1906.03761<\/a>."},{"key":"83","unstructured":"<small>STOJNIC, M.<\/small> (2010). <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_13\"><msub><mrow><mi>\u2113<\/mi><\/mrow><mrow><mn>1<\/mn><\/mrow><\/msub><\/math> optimization and its various thresholds in compressed sensing. In 2010 <i>IEEE International Conference on Acoustics, Speech and Signal Processing<\/i> 3910\u20133913."},{"key":"86","unstructured":"<small>TAHERI, H.<\/small>, <small>PEDARSANI, R.<\/small> and <small>THRAMPOULIDIS, C.<\/small> (2020). Fundamental limits of ridge-regularized empirical risk minimization in high dimensions. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2006.08917\">2006.08917<\/a>."},{"key":"89","unstructured":"<small>TSIGLER, A.<\/small> and <small>BARTLETT, P. L.<\/small> (2023). Benign overfitting in ridge regression. <i>J. Mach. Learn. Res.<\/i> <b>24<\/b> Paper No. [123], 76 pp."},{"key":"93","unstructured":"<small>ZHANG, C.<\/small>, <small>BENGIO, S.<\/small>, <small>HARDT, M.<\/small>, <small>RECHT, B.<\/small> and <small>VINYALS, O.<\/small> (2016). Understanding deep learning requires rethinking generalization. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1611.03530\">1611.03530<\/a>."},{"key":"94","unstructured":"<small>ZHOU, L.<\/small>, <small>KOEHLER, F.<\/small>, <small>SUR, P.<\/small>, <small>SUTHERLAND, D. J.<\/small> and <small>SREBRO, N.<\/small> (2022). A non-asymptotic Moreau envelope theory for high-dimensional generalized linear models. Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2210.12082\">2210.12082<\/a>."}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T16:51:30Z","timestamp":1791391890000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-53\/issue-2\/The-generalization-error-of-max-margin-linear-classifiers--Benign\/10.1214\/25-AOS2489.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,1]]},"references-count":94,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,4,1]]}},"URL":"https:\/\/doi.org\/10.1214\/25-aos2489","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,1]]}}}