{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T21:27:56Z","timestamp":1778880476631,"version":"3.51.4"},"reference-count":37,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,11,2]],"date-time":"2023-11-02T00:00:00Z","timestamp":1698883200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Engineering and Physical Sciences Research Fellowship","award":["EP\/T01864X\/1"],"award-info":[{"award-number":["EP\/T01864X\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,9,1]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>Direct use of the likelihood function typically produces severely biased estimates when the dimension of the parameter vector is large relative to the effective sample size. With linearly separable data generated from a logistic regression model, the loglikelihood function asymptotes and the maximum likelihood estimator does not exist. We show that an exact analysis for each regression coefficient produces half-infinite confidence sets for some parameters when the data are separable. Such conclusions are not vacuous, but an honest portrayal of the limitations of the data. Finite confidence sets are only achievable when additional, perhaps implicit, assumptions are made. Under a notional double-asymptotic regime in which the dimension of the logistic coefficient vector increases with the sample size, the present paper considers the implications of enforcing a natural constraint on the vector of logistic transformed probabilities. We derive a relationship between the logistic coefficients and a notional parameter obtained as a probability limit of an ordinary least-squares estimator. The latter exists even when the data are separable. Consistency is ascertained under weak conditions on the design matrix.<\/jats:p>","DOI":"10.1093\/biomet\/asad065","type":"journal-article","created":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T19:02:50Z","timestamp":1699383770000},"page":"989-1011","source":"Crossref","is-referenced-by-count":6,"title":["On inference in high-dimensional logistic regression models with separated data"],"prefix":"10.1093","volume":"111","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9584-0333","authenticated-orcid":false,"given":"R M","family":"Lewis","sequence":"first","affiliation":[{"name":"Department of Mathematics, Imperial College London, South Kensington Campus , London SW7 2AZ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9387-4628","authenticated-orcid":false,"given":"H S","family":"Battey","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Imperial College London, South Kensington Campus , London SW7 2AZ, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,11,2]]},"reference":[{"key":"2024090418515776500_asad065-B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/biomet\/71.1.1","article-title":"On the existence of maximum likelihood estimates in logistic regression models","volume":"71","author":"Albert","year":"1984","journal-title":"Biometrika"},{"key":"2024090418515776500_asad065-B2","first-page":"611","article-title":"Methodologies in spectral analysis of large dimensional random matrices, a review","volume":"9","author":"Bai","year":"1999","journal-title":"Statist. Sinica"},{"key":"2024090418515776500_asad065-B3","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1017\/S0305004100019290","article-title":"The information available in small samples","volume":"32","author":"Bartlett","year":"1936","journal-title":"Proc. Camb. Phil. Soc"},{"key":"2024090418515776500_asad065-B4","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1098\/rspa.1937.0109","article-title":"Properties of sufficiency and statistical tests","volume":"160","author":"Bartlett","year":"1937","journal-title":"Proc. R. Soc. Lond. A"},{"key":"2024090418515776500_asad065-B5","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.190067.","article-title":"On the linear in probability model for binary data","volume":"6","author":"Battey","year":"2019","journal-title":"R. Soc. Open Sci."},{"key":"2024090418515776500_asad065-B6","volume-title":"Theory and Applications","author":"B\u00fchlmann","year":"2011"},{"key":"2024090418515776500_asad065-B7","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1080\/01621459.2021.1990769","article-title":"Statistical inference for high-dimensional generalized linear models with binary outcomes","volume":"118","author":"Cai","year":"2021","journal-title":"J. Am. Statist. Assoc"},{"key":"2024090418515776500_asad065-B8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1214\/18-AOS1789","article-title":"The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression","volume":"48","author":"Cand\u00e8s","year":"2020","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B9","doi-asserted-by":"crossref","first-page":"365001","DOI":"10.1088\/1751-8121\/aba028","article-title":"Replica analysis of overfitting in generalized linear regression models","volume":"53","author":"Coolen","year":"2020","journal-title":"J. Phys. A"},{"key":"2024090418515776500_asad065-B10","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1111\/j.2517-6161.1958.tb00292.x","article-title":"The regression analysis of binary sequences (with discussion)","volume":"20","author":"Cox","year":"1958","journal-title":"J. R. Statist. Soc. B"},{"key":"2024090418515776500_asad065-B11","volume-title":"Analysis of Binary Data","author":"Cox","year":"1970"},{"key":"2024090418515776500_asad065-B12","doi-asserted-by":"crossref","first-page":"8592","DOI":"10.1073\/pnas.1703764114","article-title":"Large numbers of explanatory variables, a semi-discriptive analysis","volume":"114","author":"Cox","year":"2017","journal-title":"Proc. Nat. Acad. Sci. USA"},{"key":"2024090418515776500_asad065-B13","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1093\/biomet\/79.3.441","article-title":"Response models for mixed binary and quantitative variables","volume":"79","author":"Cox","year":"1992","journal-title":"Biometrika"},{"key":"2024090418515776500_asad065-B14","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1080\/03610928908829944","article-title":"On the small sample properties of norm-restricted maximum likelihood estimators for logistic regression models","volume":"18","author":"Duffy","year":"1989","journal-title":"Commun. Statist. A"},{"key":"2024090418515776500_asad065-B15","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"J. Amer. Statist. Assoc"},{"key":"2024090418515776500_asad065-B16","doi-asserted-by":"crossref","first-page":"928","DOI":"10.1214\/009053604000000256","article-title":"Nonconcave penalized likelihood with a diverging number of parameters","volume":"32","author":"Fan","year":"2004","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B17","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/biomet\/80.1.27","article-title":"Bias reduction of maximum likelihood estimates","volume":"80","author":"Firth","year":"1993","journal-title":"Biometrika"},{"key":"2024090418515776500_asad065-B18","doi-asserted-by":"crossref","first-page":"2409","DOI":"10.1002\/sim.1047","article-title":"A solution to the problem of separation in logistic regression","volume":"21","author":"Heinze","year":"2002","journal-title":"Statist. Med"},{"key":"2024090418515776500_asad065-B19","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1214\/aos\/1176342503","article-title":"Robust regression: asymptotics, conjectures and Monte Carlo","volume":"1","author":"Huber","year":"1973","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B20","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1093\/biomet\/asaa052","article-title":"Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models","volume":"108","author":"Kosmidis","year":"2021","journal-title":"Biometrika"},{"key":"2024090418515776500_asad065-B21","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1007\/s00440-017-0824-7","article-title":"Asymptotics for high dimensional regression M-estimates: fixed design results","volume":"172","author":"Lei","year":"2018","journal-title":"Prob. Theory Rel. Fields"},{"key":"2024090418515776500_asad065-B22","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1080\/01621459.2019.1699421","article-title":"Global and simultaneous hypothesis testing for high-dimensional logistic regression models","volume":"116","author":"Ma","year":"2021","journal-title":"J. Am. Statist. Assoc"},{"key":"2024090418515776500_asad065-B23","doi-asserted-by":"crossref","first-page":"1225","DOI":"10.1214\/aos\/1035844977","article-title":"What is a statistical model?","volume":"30","author":"McCullagh","year":"2002","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B24","doi-asserted-by":"crossref","first-page":"2143","DOI":"10.1002\/sim.4780141908","article-title":"Exact logistic regression: theory and examples","volume":"14","author":"Mehta","year":"1995","journal-title":"Statist. Med."},{"key":"2024090418515776500_asad065-B25","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1111\/j.1467-9868.2007.00627.x","article-title":"The group lasso for logistic regression","volume":"70","author":"Meier","year":"2008","journal-title":"J. R. Statist. Soc. B"},{"key":"2024090418515776500_asad065-B26","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1214\/16-AOS1448","article-title":"A general theory of hypothesis tests and confidence regions for sparse high dimensional models","volume":"45","author":"Ning","year":"2017","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B27","author":"R Development Core Team","year":"2024"},{"key":"2024090418515776500_asad065-B28","doi-asserted-by":"crossref","first-page":"6976","DOI":"10.1109\/TIT.2011.2165799","article-title":"Minimax rates of estimation for high-dimensional linear regression over\u2002\u2113q-balls.","volume":"57","author":"Raskutti","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"2024090418515776500_asad065-B29","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1080\/01621459.2019.1710154","article-title":"Statistical inference for high-dimensional models via recursive online-score estimation","volume":"116","author":"Shi","year":"2021","journal-title":"J. Am. Statist. Assoc"},{"key":"2024090418515776500_asad065-B30","doi-asserted-by":"crossref","first-page":"14516","DOI":"10.1073\/pnas.1810420116","article-title":"A modern maximum-likelihood theory for high-dimensional logistic regression","volume":"116","author":"Sur","year":"2019","journal-title":"Proc. Nat. Acad. Sci. USA"},{"key":"2024090418515776500_asad065-B31","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1111\/rssb.12389","article-title":"Modified likelihood root in high dimensions","volume":"82","author":"Tang","year":"2020","journal-title":"J. R. Statist. Soc. B"},{"key":"2024090418515776500_asad065-B32","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Statist. Soc. B"},{"key":"2024090418515776500_asad065-B33","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1214\/009053607000000929","article-title":"High-dimensional generalized linear models and the lasso","volume":"36","author":"van de Geer","year":"2008","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B34","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1214\/14-AOS1221","article-title":"On asymptotically optimal confidence regions and tests for high-dimensional models","volume":"42","author":"van de Geer","year":"2014","journal-title":"Ann. Statist."},{"key":"2024090418515776500_asad065-B35","volume-title":"Statistical Decision Functions","author":"Wald","year":"1950"},{"key":"2024090418515776500_asad065-B36","first-page":"29517","volume-title":"Advances in Neural Information Processing Systems","author":"Yadlowsky","year":"2021"},{"key":"2024090418515776500_asad065-B37","author":"Zhao","year":"2023"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asad065\/56553449\/asad065.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/111\/3\/989\/59021334\/asad065.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/111\/3\/989\/59021334\/asad065.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,5]],"date-time":"2024-09-05T11:28:35Z","timestamp":1725535715000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/111\/3\/989\/7338235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,2]]},"references-count":37,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,11,2]]},"published-print":{"date-parts":[[2024,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asad065","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,9,1]]},"published":{"date-parts":[[2023,11,2]]}}}