{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,13]],"date-time":"2024-07-13T07:41:17Z","timestamp":1720856477371},"reference-count":23,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,6,21]],"date-time":"2024-06-21T00:00:00Z","timestamp":1718928000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,21]],"date-time":"2024-06-21T00:00:00Z","timestamp":1718928000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Stat Comput"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The widespread use of maximum Jeffreys\u2019-prior penalized likelihood in binomial-response generalized linear models, and in logistic regression, in particular, are supported by the results of Kosmidis and Firth (Biometrika 108:71\u201382, 2021. <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"doi\" xlink:href=\"10.1093\/biomet\/asaa052\">https:\/\/doi.org\/10.1093\/biomet\/asaa052<\/jats:ext-link>), who show that the resulting estimates are always finite-valued, even in cases where the maximum likelihood estimates are not, which is a practical issue regardless of the size of the data set. In logistic regression, the implied adjusted score equations are formally bias-reducing in asymptotic frameworks with a fixed number of parameters and appear to deliver a substantial reduction in the persistent bias of the maximum likelihood estimator in high-dimensional settings where the number of parameters grows asymptotically as a proportion of the number of observations. In this work, we develop and present two new variants of iteratively reweighted least squares for estimating generalized linear models with adjusted score equations for mean bias reduction and maximization of the likelihood penalized by a positive power of the Jeffreys-prior penalty, which eliminate the requirement of storing <jats:italic>O<\/jats:italic>(<jats:italic>n<\/jats:italic>) quantities in memory, and can operate with data sets that exceed computer memory or even hard drive capacity. We achieve that through incremental QR decompositions, which enable IWLS iterations to have access only to data chunks of predetermined size. Both procedures can also be readily adapted to fit generalized linear models when distinct parts of the data is stored across different sites and, due to privacy concerns, cannot be fully transferred across sites. We assess the procedures through a real-data application with millions of observations.<\/jats:p>","DOI":"10.1007\/s11222-024-10447-z","type":"journal-article","created":{"date-parts":[[2024,6,21]],"date-time":"2024-06-21T07:02:01Z","timestamp":1718953321000},"update-policy":"http:\/\/dx.doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Bounded-memory adjusted scores estimation in generalized linear models with large data sets"],"prefix":"10.1007","volume":"34","author":[{"given":"Patrick","family":"Zietkiewicz","sequence":"first","affiliation":[]},{"given":"Ioannis","family":"Kosmidis","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,6,21]]},"reference":[{"issue":"1","key":"10447_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2307\/2336390","volume":"71","author":"A Albert","year":"1984","unstructured":"Albert, A., Anderson, J.: On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71(1), 1\u201310 (1984). https:\/\/doi.org\/10.2307\/2336390","journal-title":"Biometrika"},{"issue":"1","key":"10447_CR2","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1214\/18-AOS1789","volume":"48","author":"EJ Cand\u00e8s","year":"2020","unstructured":"Cand\u00e8s, E.J., Sur, P.: The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. Ann. Stat. 48(1), 27\u201342 (2020). https:\/\/doi.org\/10.1214\/18-AOS1789","journal-title":"Ann. Stat."},{"key":"10447_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/j.2517-6161.1987.tb01422.x","volume":"49","author":"DR Cox","year":"1987","unstructured":"Cox, D.R., Reid, N.: Parameter orthogonality and approximate conditional inference. J. R. Stat. Soc. Ser. B: Methodol. 49, 1\u201318 (1987). https:\/\/doi.org\/10.1111\/j.2517-6161.1987.tb01422.x","journal-title":"J. R. Stat. Soc. Ser. B: Methodol."},{"issue":"111","key":"10447_CR4","first-page":"3475","volume":"13","author":"P Drineas","year":"2012","unstructured":"Drineas, P., Magdon-Ismail, M., Mahoney, M.W., Woodruff, D.P.: Fast approximation of matrix coherence and statistical leverage. J. Mach. Learn. Res. 13(111), 3475\u20133506 (2012)","journal-title":"J. Mach. Learn. Res."},{"issue":"1","key":"10447_CR5","doi-asserted-by":"publisher","first-page":"27","DOI":"10.2307\/2336755","volume":"80","author":"D Firth","year":"1993","unstructured":"Firth, D.: Bias reduction of maximum likelihood estimates. Biometrika 80(1), 27\u201338 (1993). https:\/\/doi.org\/10.2307\/2336755","journal-title":"Biometrika"},{"key":"10447_CR6","doi-asserted-by":"crossref","unstructured":"Golub, G.H., Van\u00a0Loan, C.F.: Matrix Computations, 4th edition, The Johns Hopkins University Press, Baltimore (2013). ISBN 978-1-4214-0794-4","DOI":"10.56021\/9781421407944"},{"issue":"2","key":"10447_CR7","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1111\/j.2517-6161.1984.tb01288.x","volume":"46","author":"PJ Green","year":"1984","unstructured":"Green, P.J.: Iteratively reweighted least squares for maximum likelihood estimation, and some robust and resistant alternatives. J. R. Stat. Soc. Ser. B (Methodol.) 46(2), 149\u2013192 (1984). https:\/\/doi.org\/10.1111\/j.2517-6161.1984.tb01288.x","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"10447_CR8","unstructured":"Heinze, G., Ploner, M., Jiricka, L., Steiner, G.: logistf: Firth\u2019s bias-reduced logistic regression. R package version 1.26.0, (2023)https:\/\/cran.r-project.org\/web\/packages\/logistf\/index.html"},{"key":"10447_CR9","doi-asserted-by":"publisher","unstructured":"In Data Expo 2009: Airline on time data,(2008).2000.csv.bz2. Harvard Dataverse. https:\/\/doi.org\/10.7910\/DVN\/HG7NV7\/YGU3TD","DOI":"10.7910\/DVN\/HG7NV7\/YGU3TD"},{"key":"10447_CR10","unstructured":"Konis, K.: Linear Programming Algorithms for Detecting Separated Data in Binary Logistic Regression Models. Ph.D. thesis, University of Oxford. (2007).https:\/\/ora.ox.ac.uk\/objects\/uuid:8f9ee0d0-d78e-4101-9ab4-f9cbceed2a2a"},{"key":"10447_CR11","unstructured":"Kosmidis, I.: brglm2: Bias reduction in generalized linear models. R package version 0.9.2, (2023).https:\/\/CRAN.R-project.org\/package=brglm2"},{"key":"10447_CR12","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1093\/biomet\/asaa052","volume":"108","author":"I Kosmidis","year":"2021","unstructured":"Kosmidis, I., Firth, D.: Jeffreys-prior penalty, finiteness and shrinkage in binomial-response generalized linear models. Biometrika 108, 71\u201382 (2021). https:\/\/doi.org\/10.1093\/biomet\/asaa052","journal-title":"Biometrika"},{"key":"10447_CR13","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1007\/s11222-019-09860-6","volume":"30","author":"I Kosmidis","year":"2020","unstructured":"Kosmidis, I., Kenne Pagui, E.C., Sartori, N.: Mean and median bias reduction in generalized linear models. Stat. Comput. 30, 43\u201359 (2020). https:\/\/doi.org\/10.1007\/s11222-019-09860-6","journal-title":"Stat. Comput."},{"key":"10447_CR14","unstructured":"Kosmidis, I., Schumacher, D., Schwendinger, F.: detectseparation: Detect and check for separation and infinite maximum likelihood estimates. R package version 0.3,(2022). https:\/\/CRAN.R-project.org\/package=detectseparation"},{"issue":"1","key":"10447_CR15","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1111\/j.2517-6161.1989.tb01752.x","volume":"51","author":"E Lesaffre","year":"1989","unstructured":"Lesaffre, E., Albert, A.: Partial separation in logistic discrimination. J. R. Stat. Soc. Ser. B (Methodol.) 51(1), 109\u2013116 (1989). https:\/\/doi.org\/10.1111\/j.2517-6161.1989.tb01752.x","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"10447_CR16","unstructured":"Lumley, T.: biglm: bounded memory linear and generalized linear models. R package version 0.9-2.1, (2020).https:\/\/CRAN.R-project.org\/package=biglm"},{"issue":"4","key":"10447_CR17","doi-asserted-by":"publisher","first-page":"864","DOI":"10.1093\/aje\/kwx299","volume":"187","author":"MA Mansournia","year":"2018","unstructured":"Mansournia, M.A., Geroldinger, A., Greenland, S., Heinze, G.: Separation in logistic regression: causes, consequences, and control. Am. J. Epidemiol. 187(4), 864\u2013870 (2018). https:\/\/doi.org\/10.1093\/aje\/kwx299","journal-title":"Am. J. Epidemiol."},{"key":"10447_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3242-6","volume-title":"Generalized Linear Models","author":"P McCullagh","year":"1989","unstructured":"McCullagh, P., Nelder, J.A.: Generalized Linear Models, 2nd edn. Chapman and Hall, London (1989)","edition":"2"},{"issue":"2","key":"10447_CR19","doi-asserted-by":"publisher","first-page":"458","DOI":"10.2307\/2347583","volume":"41","author":"AJ Miller","year":"1992","unstructured":"Miller, A.J.: Algorithm AS 274: least squares routines to supplement those of Gentleman. Appl. Stat. 41(2), 458 (1992). https:\/\/doi.org\/10.2307\/2347583","journal-title":"Appl. Stat."},{"key":"10447_CR20","unstructured":"R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. (2024).https:\/\/www.R-project.org\/"},{"key":"10447_CR21","doi-asserted-by":"publisher","first-page":"1613","DOI":"10.1007\/s00180-021-01084-5","volume":"0943\u20134062","author":"F Schwendinger","year":"2021","unstructured":"Schwendinger, F., Gr\u00fcn, B., Hornik, K.: A comparison of optimization solvers for log binomial regression including conic programming. Comput. Stat. 0943\u20134062, 1613\u20139658 (2021). https:\/\/doi.org\/10.1007\/s00180-021-01084-5","journal-title":"Comput. Stat."},{"key":"10447_CR22","doi-asserted-by":"publisher","unstructured":"Sur, P., Cand\u00e8s, E.J.: A modern maximum-likelihood theory for high-dimensional logistic regression. Proc. Natl. Acad. Sci. 116(29), 14516\u201314525 (2019). https:\/\/doi.org\/10.1073\/pnas.1810420116","DOI":"10.1073\/pnas.1810420116"},{"key":"10447_CR23","doi-asserted-by":"publisher","unstructured":"Wedderburn, R.W.M.: On the existence and uniqueness of the maximum likelihood estimates for certain generalized linear models. Biometrika 63(1), 27\u201332 (1976). https:\/\/doi.org\/10.1093\/biomet\/63.1.27","DOI":"10.1093\/biomet\/63.1.27"}],"container-title":["Statistics and Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-024-10447-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11222-024-10447-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11222-024-10447-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,13]],"date-time":"2024-07-13T07:15:40Z","timestamp":1720854940000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11222-024-10447-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,21]]},"references-count":23,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10447"],"URL":"https:\/\/doi.org\/10.1007\/s11222-024-10447-z","relation":{},"ISSN":["0960-3174","1573-1375"],"issn-type":[{"value":"0960-3174","type":"print"},{"value":"1573-1375","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,21]]},"assertion":[{"value":"1 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 June 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 June 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"138"}}