{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T07:16:05Z","timestamp":1793862965030,"version":"4.6.1"},"reference-count":40,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000001","name":"U.S. National Science Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Union","doi-asserted-by":"publisher","award":["737\/2021"],"award-info":[{"award-number":["737\/2021"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,7,16]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>Estimation of crossed random effects models commonly incurs computational costs that grow faster than linearly in the sample size $ N $, often as fast as $ \\Omega(N^{3\/2}) $, making them unsuitable for large datasets. For non-Gaussian responses, integrating out the random effects to obtain a marginal likelihood poses significant challenges, especially for high-dimensional integrals for which the Laplace approximation may not be accurate. In this article we develop a composite likelihood approach to probit models that replaces the crossed random effects model with some hierarchical models that require only one-dimensional integrals. We show how to consistently estimate the crossed effects model parameters from the hierarchical model fits. We find that the computation scales linearly in the sample size. The method is illustrated by applying it to approximately five million observations from Stitch Fix, where the crossed effects formulation would require an integral of dimension larger than $ 700\\,000 $.<\/jats:p>","DOI":"10.1093\/biomet\/asaf037","type":"journal-article","created":{"date-parts":[[2025,5,26]],"date-time":"2025-05-26T10:44:48Z","timestamp":1748256288000},"source":"Crossref","is-referenced-by-count":3,"title":["Consistent and scalable composite likelihood estimation of probit models with crossed random effects"],"prefix":"10.1093","volume":"112","author":[{"given":"R","family":"Bellio","sequence":"first","affiliation":[{"name":"Department of Economics and Statistics, University of Udine , Via Tomadini, 30\/A , 33100 Udine,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"S","family":"Ghosh","sequence":"additional","affiliation":[{"name":"Department of Statistics, Stanford University , Sequoia Hall , Stanford California 94305,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"A B","family":"Owen","sequence":"additional","affiliation":[{"name":"Department of Statistics, Stanford University , Sequoia Hall , Stanford California 94305,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C","family":"Varin","sequence":"additional","affiliation":[{"name":"Department of Environmental Sciences, Informatics and Statistics, Ca\u2019 Foscari University , Via Torino 155 , 30172 Venice,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,5,26]]},"reference":[{"key":"2025101607432799000_asaf037-B1","doi-asserted-by":"crossref","DOI":"10.1002\/0471249688","volume-title":"Categorical Data Analysis","author":"Agresti","year":"2002"},{"key":"2025101607432799000_asaf037-B2","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1080\/10618600.2016.1172018","article-title":"Composite likelihood inference in a discrete latent variable model for two-way \u2032clustering-by-segmentation' problems","volume":"26","author":"Bartolucci","year":"2017","journal-title":"J. Comp. Graph. Statist"},{"key":"2025101607432799000_asaf037-B3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting linear mixed-effects models using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Statist. Software"},{"key":"2025101607432799000_asaf037-B4","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1191\/1471082X05st095oa","article-title":"A pairwise likelihood approach to generalized linear models with crossed random effects","volume":"5","author":"Bellio","year":"2005","journal-title":"Statist. Mod"},{"key":"2025101607432799000_asaf037-B5","author":"Bennett","year":"2007"},{"key":"2025101607432799000_asaf037-B6","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1080\/01621459.2022.2141635","article-title":"Stochastic convergence rates and applications of adaptive quadrature in Bayesian inference","volume":"119","author":"Bilodeau","year":"2024","journal-title":"J. Am. Statist. Assoc"},{"key":"2025101607432799000_asaf037-B7","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1080\/01621459.1993.10594284","article-title":"Approximate inference in generalized linear mixed models","volume":"88","author":"Breslow","year":"1993","journal-title":"J. Am. Statist. Assoc"},{"key":"2025101607432799000_asaf037-B8","doi-asserted-by":"crossref","first-page":"238","DOI":"10.1198\/jbes.2010.07136","article-title":"Robust inference with multiway clustering","volume":"29","author":"Cameron","year":"2011","journal-title":"J. Bus. Econ. Statist"},{"key":"2025101607432799000_asaf037-B9","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-6868-4","volume-title":"Seamless R and C++ Integration with Rcpp","author":"Eddelbuettel","year":"2013"},{"key":"2025101607432799000_asaf037-B10","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1017\/S0962492904000236","article-title":"Random matrix theory","volume":"14","author":"Edelman","year":"2005","journal-title":"Acta Numer"},{"key":"2025101607432799000_asaf037-B11","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1214\/17-EJS1236","article-title":"Efficient moment calculations for variance components in large unbalanced crossed random effects models","volume":"11","author":"Gao","year":"2017","journal-title":"Electron. J. Statist."},{"key":"2025101607432799000_asaf037-B12","first-page":"1741","article-title":"Estimation and inference for very large linear mixed effects models","volume":"30","author":"Gao","year":"2020","journal-title":"Statist. Sinica"},{"key":"2025101607432799000_asaf037-B13","author":"Ghandwani","year":"2023"},{"key":"2025101607432799000_asaf037-B14","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1214\/21-AOS2121","article-title":"Backfitting for large scale crossed random effects regressions","volume":"50","author":"Ghosh","year":"2022","journal-title":"Ann. Statist."},{"key":"2025101607432799000_asaf037-B15","doi-asserted-by":"crossref","first-page":"4604","DOI":"10.1214\/22-EJS2047","article-title":"Scalable logistic regression with crossed random effects","volume":"16","author":"Ghosh","year":"2022","journal-title":"Electron. J. Statist."},{"key":"2025101607432799000_asaf037-B16","author":"Ghosh","year":"2021"},{"key":"2025101607432799000_asaf037-B17","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.2307\/2533520","article-title":"Random effects probit and logistic regression models for three-level data","volume":"53","author":"Gibbons","year":"1997","journal-title":"Biometrics"},{"key":"2025101607432799000_asaf037-B18","doi-asserted-by":"crossref","first-page":"asae067","DOI":"10.1093\/biomet\/asae067","article-title":"Partially factorized variational inference for high-dimensional mixed models","volume":"112","author":"Goplerud","year":"2025","journal-title":"Biometrika"},{"key":"2025101607432799000_asaf037-B19","doi-asserted-by":"crossref","first-page":"1902","DOI":"10.1080\/01621459.2019.1665529","article-title":"Fast and accurate binary response mixed model analysis via expectation propagation","volume":"115","author":"Hall","year":"2020","journal-title":"J. Am. Statist. Assoc"},{"key":"2025101607432799000_asaf037-B20","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1214\/13-AOS1084","article-title":"The subset argument and consistency of MLE in GLMM: Answer to an open problem and beyond","volume":"41","author":"Jiang","year":"2013","journal-title":"Ann. Statist."},{"key":"2025101607432799000_asaf037-B21","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1111\/rssb.12473","article-title":"Usable and precise asymptotics for generalized linear mixed model analysis and design","volume":"84","author":"Jiang","year":"2022","journal-title":"J. R. Statist. Soc. B"},{"key":"2025101607432799000_asaf037-B22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v070.i05","article-title":"TMB: Automatic differentiation and Laplace approximation","volume":"70","author":"Kristensen","year":"2016","journal-title":"J. Statist. Software"},{"key":"2025101607432799000_asaf037-B23","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1090\/conm\/080\/999014","article-title":"Composite likelihood methods","volume":"80","author":"Lindsay","year":"1988","journal-title":"Contemp. Math"},{"key":"2025101607432799000_asaf037-B24","first-page":"624","article-title":"A note on Gauss\u2013Hermite quadrature","volume":"81","author":"Liu","year":"1994","journal-title":"Biometrika"},{"key":"2025101607432799000_asaf037-B25","author":"Lumley","year":"2003"},{"key":"2025101607432799000_asaf037-B26","doi-asserted-by":"crossref","first-page":"2956","DOI":"10.1214\/24-AOS2469","article-title":"Increasing dimension asymptotics for two-way crossed mixed effect models","volume":"52","author":"Lyu","year":"2024","journal-title":"Ann. Statist."},{"key":"2025101607432799000_asaf037-B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4899-3242-6","volume-title":"Generalized Linear Models","author":"McCullagh","year":"1989"},{"key":"2025101607432799000_asaf037-B28","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1093\/biostatistics\/kxg042","article-title":"Marginal modeling of multilevel binary data with time-varying covariates","volume":"5","author":"Miglioretti","year":"2004","journal-title":"Biostatistics"},{"key":"2025101607432799000_asaf037-B29","doi-asserted-by":"crossref","first-page":"e380","DOI":"10.1002\/sta4.380","article-title":"On the error in Laplace approximations of high-dimensional integrals","volume":"10","author":"Ogden","year":"2021","journal-title":"Stat"},{"key":"2025101607432799000_asaf037-B30","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1214\/07-AOAS122","article-title":"The pigeonhole bootstrap","volume":"1","author":"Owen","year":"2007","journal-title":"Ann. Appl. Statist"},{"key":"2025101607432799000_asaf037-B31","first-page":"25","article-title":"Scalable inference for crossed random effects models","volume":"107","author":"Papaspiliopoulos","year":"2020","journal-title":"Biometrika"},{"key":"2025101607432799000_asaf037-B32","doi-asserted-by":"crossref","first-page":"3575","DOI":"10.1214\/23-EJS2172","article-title":"Scalable Bayesian computation for crossed and nested hierarchical models","volume":"17","author":"Papaspiliopoulos","year":"2023","journal-title":"Electron. J. Statist."},{"key":"2025101607432799000_asaf037-B33","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Development Core Team","year":"2025"},{"key":"2025101607432799000_asaf037-B34","doi-asserted-by":"crossref","first-page":"3986","DOI":"10.1214\/16-EJS1218","article-title":"Improved Laplace approximation for marginal likelihoods","volume":"10","author":"Ruli","year":"2016","journal-title":"Electron. J. Statist."},{"key":"2025101607432799000_asaf037-B35","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1093\/biomet\/78.4.719","article-title":"Estimation in generalized linear models with random effects","volume":"78","author":"Schall","year":"1991","journal-title":"Biometrika"},{"key":"2025101607432799000_asaf037-B36","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1111\/j.2517-6161.1995.tb02060.x","article-title":"Laplace approximation of high dimensional integrals","volume":"57","author":"Shun","year":"1995","journal-title":"J. R. Statist. Soc. B"},{"key":"2025101607432799000_asaf037-B37","doi-asserted-by":"crossref","first-page":"961","DOI":"10.2307\/2531147","article-title":"Random-effects models for serial observations with binary response","volume":"40","author":"Stiratelli","year":"1984","journal-title":"Biometrics"},{"key":"2025101607432799000_asaf037-B38","doi-asserted-by":"crossref","first-page":"1759","DOI":"10.3150\/24-BEJ1758","article-title":"Laplace and saddlepoint approximations in high dimensions","volume":"31","author":"Tang","year":"2025","journal-title":"Bernoulli"},{"key":"2025101607432799000_asaf037-B39","first-page":"5","article-title":"An overview of composite likelihood methods","volume":"21","author":"Varin","year":"2011","journal-title":"Statist. Sinica"},{"key":"2025101607432799000_asaf037-B40","author":"Xu","year":"2023"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asaf037\/63355918\/asaf037.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/112\/3\/asaf037\/63355918\/asaf037.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/112\/3\/asaf037\/63355918\/asaf037.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T11:44:02Z","timestamp":1760615042000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/doi\/10.1093\/biomet\/asaf037\/8149185"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":40,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,7,16]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asaf037","relation":{},"ISSN":["1464-3510"],"issn-type":[{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025]]},"published":{"date-parts":[[2025]]},"article-number":"asaf037"}}