{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T17:23:19Z","timestamp":1791825799268,"version":"4.3.4"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T00:00:00Z","timestamp":1716249600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T00:00:00Z","timestamp":1716249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["200021_191960\/1"],"award-info":[{"award-number":["200021_191960\/1"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sankhya A"],"published-print":{"date-parts":[[2026,8]]},"abstract":"Abstract<\/jats:title>\n \n The logistic regression estimator is known to inflate the magnitude of its coefficients if the sample size\n n<\/jats:italic>\n is small, the dimension\n p<\/jats:italic>\n is (moderately) large or the signal-to-noise ratio\n \n \n $$1\/\\sigma $$<\/jats:tex-math>\n \n \n 1<\/mml:mn>\n \/<\/mml:mo>\n \u03c3<\/mml:mi>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n is large (probabilities of observing a label are close to 0 or 1). With this in mind, we study the logistic regression estimator with\n \n \n $$p\\ll n\/\\log n$$<\/jats:tex-math>\n \n \n p<\/mml:mi>\n \u226a<\/mml:mo>\n n<\/mml:mi>\n \/<\/mml:mo>\n log<\/mml:mo>\n n<\/mml:mi>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n , assuming Gaussian covariates and labels generated by the Gaussian link function, with a mild optimization constraint on the estimator\u2019s length to ensure existence. We provide finite sample guarantees for its direction, which serves as a classifier, and its Euclidean norm, which is an estimator for the signal-to-noise ratio. We distinguish between two regimes. In the low-noise\/small-sample regime (\n \n \n $$\\sigma \\lesssim (p\\log n)\/n$$<\/jats:tex-math>\n \n \n \u03c3<\/mml:mi>\n \u2272<\/mml:mo>\n (<\/mml:mo>\n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n )<\/mml:mo>\n \/<\/mml:mo>\n n<\/mml:mi>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n ), we show that the estimator\u2019s direction (and consequentially the classification error) achieve the rate\n \n \n $$(p\\log n)\/n$$<\/jats:tex-math>\n \n \n (<\/mml:mo>\n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n )<\/mml:mo>\n \/<\/mml:mo>\n n<\/mml:mi>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n - up to the log term as if the problem was noiseless. In this case, the norm of the estimator is at least of order\n \n \n $$n\/(p\\log n)$$<\/jats:tex-math>\n \n \n n<\/mml:mi>\n \/<\/mml:mo>\n (<\/mml:mo>\n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n . If instead\n \n \n $$(p\\log n)\/n\\lesssim \\sigma \\lesssim 1$$<\/jats:tex-math>\n \n \n (<\/mml:mo>\n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n )<\/mml:mo>\n \/<\/mml:mo>\n n<\/mml:mi>\n \u2272<\/mml:mo>\n \u03c3<\/mml:mi>\n \u2272<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n , the estimator\u2019s direction achieves the rate\n \n \n $$\\sqrt{\\sigma p\\log n\/n}$$<\/jats:tex-math>\n \n \n \n \u03c3<\/mml:mi>\n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n \/<\/mml:mo>\n n<\/mml:mi>\n <\/mml:mrow>\n <\/mml:msqrt>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n , whereas its norm converges to the true norm at the rate\n \n \n $$\\sqrt{p\\log n\/(n\\sigma ^3)}$$<\/jats:tex-math>\n \n \n \n p<\/mml:mi>\n log<\/mml:mo>\n n<\/mml:mi>\n \/<\/mml:mo>\n (<\/mml:mo>\n n<\/mml:mi>\n \n \u03c3<\/mml:mi>\n 3<\/mml:mn>\n <\/mml:msup>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:msqrt>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>\n . As a corollary, the data are not linearly separable with high probability in this regime. In either regime, logistic regression provides a competitive classifier.\n <\/jats:p>","DOI":"10.1007\/s13171-024-00358-6","type":"journal-article","created":{"date-parts":[[2024,5,21]],"date-time":"2024-05-21T01:02:00Z","timestamp":1716253320000},"page":"481-550","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Finite Sample Rates for Logistic Regression with Small Noise or Few Samples"],"prefix":"10.1007","volume":"88","author":[{"given":"Felix","family":"Kuchelmeister","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sara","family":"van de Geer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,5,21]]},"reference":[{"key":"358_CR1","doi-asserted-by":"crossref","unstructured":"Aad\u00a0W van\u00a0der Vaart and Jon\u00a0A Wellner. Weak Convergence and Empirical Processes With Applications to Statistics. Springer, 1996.","DOI":"10.1007\/978-1-4757-2545-2"},{"key":"358_CR2","unstructured":"Aad\u00a0W van\u00a0der Vaart. Asymptotic Statistics. Cambridge University Press, 2000."},{"issue":"5","key":"358_CR3","doi-asserted-by":"publisher","first-page":"3068","DOI":"10.1109\/TIT.2018.2884963","volume":"65","author":"Felix Abramovich","year":"2018","unstructured":"Felix Abramovich and Vadim Grinshtein. High-dimensional classification by sparse logistic regression. IEEE Transactions on Information Theory, 65(5): 3068\u20133079, 2018.","journal-title":"IEEE Transactions on Information Theory"},{"key":"358_CR4","unstructured":"Alan Agresti. Foundations of Linear and Generalized Linear Models. John Wiley & Sons, 2015."},{"key":"358_CR5","doi-asserted-by":"crossref","unstructured":"Alexandre Grothendieck. R\u00e9sum\u00e9 de la th\u00e9orie m\u00e9trique des produits tensoriels topologiques. Soc. de Matem\u00e1tica de S\u00e3o Paulo, 1956.","DOI":"10.1090\/memo\/0016"},{"key":"358_CR6","unstructured":"Andrea Montanari, Feng Ruan, Youngtak Sohn, and Jun Yan. The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime. arXiv:1911.01544, 2019."},{"key":"358_CR7","volume-title":"and Pascal Massart","author":"St\u00e9phane Boucheron","year":"2013","unstructured":"St\u00e9phane Boucheron, G\u00e1bor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013."},{"issue":"6","key":"358_CR8","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1016\/S1631-073X(02)02292-6","volume":"334","author":"Olivier Bousquet","year":"2002","unstructured":"Olivier Bousquet. A Bennett concentration inequality and its application to suprema of empirical processes. Comptes Rendus Mathematique, 334 (6): 495\u2013500, 2002.","journal-title":"Comptes Rendus Mathematique"},{"key":"358_CR9","unstructured":"Daniel Hsu and Arya Mazumdar. On the sample complexity of parameter estimation in logistic regression with normal design. arXiv:2307.04191v2, 2024."},{"key":"358_CR10","doi-asserted-by":"crossref","unstructured":"Dmitrii\u00a0M Ostrovskii and Francis Bach. Finite-sample analysis of M-estimators using self-concordance. Electronic Journal of Statistics, 15: 326\u2013391, 2021.","DOI":"10.1214\/20-EJS1780"},{"key":"358_CR11","doi-asserted-by":"crossref","unstructured":"Emmanuel\u00a0J Cand\u00e8s and Pragya Sur. The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. The Annals of Statistics, 48 (1): 27\u201342, 2020.","DOI":"10.1214\/18-AOS1789"},{"key":"358_CR12","unstructured":"Emmanuel\u00a0Jean Cand\u00e8s, Yingying Fan, Lucas Janson, and Jinchi Lv. Panning for gold: Model-free knockoffs for high-dimensional controlled variable selection, volume 1610. Department of Statistics, Stanford University Stanford, CA, USA, 2016."},{"key":"358_CR13","doi-asserted-by":"crossref","unstructured":"Enno Mammen and Alexandre\u00a0B Tsybakov. Smooth discrimination analysis. The Annals of Statistics, 27 (6): 1808\u20131829, 1999.","DOI":"10.1214\/aos\/1017939240"},{"issue":"2","key":"358_CR14","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/0097-3165(95)90052-7","volume":"69","author":"David Haussler","year":"1995","unstructured":"David Haussler. Sphere packing numbers for subsets of the boolean n-cube with bounded vapnik-chervonenkis dimension. Journal of Combinatorial Theory, Series A, 69 (2): 217\u2013232, 1995.","journal-title":"Journal of Combinatorial Theory, Series A"},{"key":"358_CR15","doi-asserted-by":"crossref","unstructured":"Lutz D\u00fcmbgen, Sara van\u00a0de Geer, Mark\u00a0C Veraar, and Jon\u00a0A Wellner. Nemirovski\u2019s inequalities revisited. The American Mathematical Monthly, 117 (2): 138\u2013160, 2010.","DOI":"10.4169\/000298910X476059"},{"key":"358_CR16","doi-asserted-by":"crossref","unstructured":"Peter B\u00fchlmann and Sara van\u00a0de Geer. Statistics for High-Dimensional Data: Methods, Theory and Applications. Springer Science & Business Media, 2011.","DOI":"10.1007\/978-3-642-20192-9"},{"key":"358_CR17","doi-asserted-by":"crossref","unstructured":"Peter McCullagh and John\u00a0A Nelder. Generalized Linear Models, Vol. 37 of Monographs on Statistics and Applied Probability London. 1989.","DOI":"10.1007\/978-1-4899-3242-6"},{"key":"358_CR18","doi-asserted-by":"crossref","unstructured":"Philip\u00a0M Long. An upper bound on the sample complexity of pac-learning halfspaces with respect to the uniform distribution. Information Processing Letters, 87 (5): 229\u2013234, 2003.","DOI":"10.1016\/S0020-0190(03)00311-9"},{"key":"358_CR19","doi-asserted-by":"crossref","unstructured":"Philip\u00a0M Long. On the sample complexity of pac learning half-spaces against the uniform distribution. IEEE Transactions on Neural Networks, 6 (6): 1556\u20131559, 1995.","DOI":"10.1109\/72.471352"},{"key":"358_CR20","doi-asserted-by":"crossref","unstructured":"Pragya Sur and Emmanuel\u00a0J Cand\u00e8s. A modern maximum-likelihood theory for high-dimensional logistic regression. Proceedings of the National Academy of Sciences, 116 (29): 14516\u201314525, 2019.","DOI":"10.1073\/pnas.1810420116"},{"key":"358_CR21","doi-asserted-by":"crossref","unstructured":"Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume\u00a047. Cambridge University Press, 2018.","DOI":"10.1017\/9781108231596"},{"key":"358_CR22","unstructured":"Sara\u00a0A van\u00a0de Geer. Empirical Processes in M-estimation, volume\u00a06. Cambridge University Press, 2000."},{"key":"358_CR23","doi-asserted-by":"crossref","unstructured":"Sara\u00a0A van\u00a0de Geer. High-dimensional generalized linear models and the lasso. The Annals of Statistics, pages 614\u2013645, 2008.","DOI":"10.1214\/009053607000000929"},{"key":"358_CR24","doi-asserted-by":"crossref","unstructured":"Szilard Nemes, Junmei\u00a0Miao Jonasson, Anna Genell, and Gunnar Steineck. Bias in odds ratios by logistic regression modelling and sample size. BMC Medical Research Methodology, 9: 1\u20135, 2009.","DOI":"10.1186\/1471-2288-9-56"},{"key":"358_CR25","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0M Cover. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Transactions on Electronic Computers, (3): 326\u2013334, 1965.","DOI":"10.1109\/PGEC.1965.264137"},{"key":"358_CR26","doi-asserted-by":"crossref","unstructured":"Walter\u00a0W Hauck\u00a0Jr and Allan Donner. Wald\u2019s test as applied to hypotheses in logit analysis. Journal of the American Statistical Association, 72 (360a): 851\u2013853, 1977.","DOI":"10.1080\/01621459.1977.10479969"}],"container-title":["Sankhya A"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13171-024-00358-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s13171-024-00358-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s13171-024-00358-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T08:02:41Z","timestamp":1790582561000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s13171-024-00358-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,21]]},"references-count":26,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,8]]}},"alternative-id":["358"],"URL":"https:\/\/doi.org\/10.1007\/s13171-024-00358-6","relation":{},"ISSN":["0976-836X","0976-8378"],"issn-type":[{"value":"0976-836X","type":"print"},{"value":"0976-8378","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,21]]},"assertion":[{"value":"30 May 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 May 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors have no conflicts of interest to declare. F.K. is funded by the Swiss National Science Foundation (SNF).","order":1,"name":"Ethics","label":"Conflict of interest and funding statement","group":{"name":"EthicsHeading","label":"Declarations"}}]}}