{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T17:00:30Z","timestamp":1791392430430,"version":"4.3.3"},"reference-count":54,"publisher":"Institute of Mathematical Statistics","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2025,6,1]]},"abstract":"<jats:p>Two linearly uncorrelated binary variables must be also independent because nonlinear dependence cannot manifest with only two possible states. This inherent linearity is the atom of dependency constituting any complex form of relationship. Inspired by this observation, we develop a framework called binary expansion linear effect (BELIEF) for understanding arbitrary relationships with a binary outcome. Models from the BELIEF framework are easily interpretable because they describe the association of binary variables in the language of linear models, yielding convenient theoretical insight and striking Gaussian parallels. With BELIEF, one may study generalized linear models (GLM) through transparent linear models, providing insight into how the choice of link affects modeling. For example, setting a GLM interaction coefficient to zero does not necessarily lead to the kind of no-interaction model assumption as understood under their linear model counterparts. Furthermore, for a binary response, maximum likelihood estimation for GLMs paradoxically fails under complete separation, when the data are most discriminative, whereas BELIEF estimation automatically reveals the perfect predictor in the data that is responsible for complete separation. We explore these phenomena and provide related theoretical results. We also provide preliminary empirical demonstration of some theoretical results.<\/jats:p>","DOI":"10.1214\/25-aos2493","type":"journal-article","created":{"date-parts":[[2025,6,27]],"date-time":"2025-06-27T02:32:35Z","timestamp":1750991555000},"page":"1068-1094","source":"Crossref","is-referenced-by-count":0,"title":["BELIEF in dependence: Leveraging atomic linearity in data bits for rethinking generalized linear models"],"prefix":"10.1214","volume":"53","author":[{"given":"Benjamin","family":"Brown","sequence":"first","affiliation":[{"name":"Department of Statistics and Operations Research, University of North Carolina at Chapel Hill"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Statistics and Operations Research, University of North Carolina at Chapel Hill"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiao-Li","family":"Meng","sequence":"additional","affiliation":[{"name":"Department of Statistics, Harvard University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"27","doi-asserted-by":"publisher","unstructured":"<small>Lei, J.<\/small>, <small>G\u2019Sell, M.<\/small>, <small>Rinaldo, A.<\/small>, <small>Tibshirani, R. J.<\/small> and <small>Wasserman, L.<\/small> (2018). Distribution-free predictive inference for regression. <i>J. Amer. Statist. Assoc.<\/i> <b>113<\/b> 1094\u20131111.","DOI":"10.1080\/01621459.2017.1307116"},{"key":"11","doi-asserted-by":"publisher","unstructured":"<small>Buja, A.<\/small>, <small>Brown, L.<\/small>, <small>Berk, R.<\/small>, <small>George, E.<\/small>, <small>Pitkin, E.<\/small>, <small>Traskin, M.<\/small>, <small>Zhang, K.<\/small> and <small>Zhao, L.<\/small> (2019). Models as approximations I: Consequences illustrated with linear regression. <i>Statist. Sci.<\/i> <b>34<\/b> 523\u2013544.","DOI":"10.1214\/18-STS693"},{"key":"6","doi-asserted-by":"publisher","unstructured":"<small>BARBER, R. F.<\/small> (2020). Is distribution-free inference possible for binary regression? <i>Electron. J. Stat.<\/i> <b>14<\/b> 3487\u20133524.","DOI":"10.1214\/20-EJS1749"},{"key":"26","unstructured":"<small>LAURITZEN, S. L.<\/small> (1996). <i>Graphical Models<\/i>. <i>Oxford Statistical Science Series<\/i> <b>17<\/b>. The Clarendon Press, Oxford University Press, New York. Oxford Science Publications."},{"key":"14","doi-asserted-by":"publisher","unstructured":"<small>CHERNOZHUKOV, V.<\/small>, <small>CHETVERIKOV, D.<\/small>, <small>KATO, K.<\/small> and <small>KOIKE, Y.<\/small> (2023). High-dimensional data bootstrap. <i>Annu. Rev. Stat. Appl.<\/i> <b>10<\/b> 427\u2013449.","DOI":"10.1146\/annurev-statistics-040120-022239"},{"key":"34","doi-asserted-by":"publisher","unstructured":"<small>O\u2019Donnell, R.<\/small> (2014). <i>Analysis of Boolean Functions<\/i>. Cambridge Univ. Press, New York.","DOI":"10.1017\/CBO9781139814782"},{"key":"42","doi-asserted-by":"publisher","unstructured":"<small>Sur, P.<\/small> and <small>Cand\u00e8s, E. J.<\/small> (2019). A modern maximum-likelihood theory for high-dimensional logistic regression. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>116<\/b> 14516\u201314525.","DOI":"10.1073\/pnas.1810420116"},{"key":"16","doi-asserted-by":"publisher","unstructured":"<small>Dai, B.<\/small>, <small>Ding, S.<\/small> and <small>Wahba, G.<\/small> (2013). Multivariate Bernoulli distribution. <i>Bernoulli<\/i> <b>19<\/b> 1465\u20131483.","DOI":"10.3150\/12-BEJSP10"},{"key":"52","doi-asserted-by":"publisher","unstructured":"<small>ZHANG, K.<\/small> (2019). BET on independence. <i>J. Amer. Statist. Assoc.<\/i> <b>114<\/b> 1620\u20131637.","DOI":"10.1080\/01621459.2018.1537921"},{"key":"39","doi-asserted-by":"publisher","unstructured":"<small>RUDIN, C.<\/small> (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. <i>Nat. Mach. Intell.<\/i> <b>1<\/b> 206\u2013215.","DOI":"10.1038\/s42256-019-0048-x"},{"key":"33","doi-asserted-by":"publisher","unstructured":"<small>MURDOCH, W. J.<\/small>, <small>SINGH, C.<\/small>, <small>KUMBIER, K.<\/small>, <small>ABBASI-ASL, R.<\/small> and <small>YU, B.<\/small> (2019). Definitions, methods, and applications in interpretable machine learning. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>116<\/b> 22071\u201322080.","DOI":"10.1073\/pnas.1900654116"},{"key":"7","doi-asserted-by":"publisher","unstructured":"<small>BARBER, R. F.<\/small>, <small>CAND\u00c8S, E. J.<\/small>, <small>RAMDAS, A.<\/small> and <small>TIBSHIRANI, R. J.<\/small> (2021). The limits of distribution-free conditional predictive inference. <i>Inf. Inference<\/i> <b>10<\/b> 455\u2013482.","DOI":"10.1093\/imaiai\/iaaa017"},{"key":"20","unstructured":"<small>GELMAN, A.<\/small>, <small>CARLIN, J. B.<\/small>, <small>STERN, H. S.<\/small> and <small>RUBIN, D. B.<\/small> (1995). <i>Bayesian Data Analysis<\/i>. <i>Texts in Statistical Science Series<\/i>. CRC Press, London."},{"key":"25","doi-asserted-by":"publisher","unstructured":"<small>HE, X.<\/small>, <small>WANG, L.<\/small> and <small>HONG, H. G.<\/small> (2013). Quantile-adaptive model-free variable screening for high-dimensional heterogeneous data. <i>Ann. Statist.<\/i> <b>41<\/b> 342\u2013369.","DOI":"10.1214\/13-AOS1087"},{"key":"32","doi-asserted-by":"publisher","unstructured":"<small>MCCULLAGH, P.<\/small> and <small>NELDER, J. A.<\/small> (2019). <i>Generalized Linear Models<\/i>. Routledge.","DOI":"10.1007\/978-1-4899-3242-6"},{"key":"38","doi-asserted-by":"publisher","unstructured":"<small>Rosenbaum, P. R.<\/small> and <small>Rubin, D. B.<\/small> (1983). The central role of the propensity score in observational studies for causal effects. <i>Biometrika<\/i> <b>70<\/b> 41\u201355.","DOI":"10.1093\/biomet\/70.1.41"},{"key":"2","doi-asserted-by":"publisher","unstructured":"<small>Albert, A.<\/small> and <small>Anderson, J. A.<\/small> (1984). On the existence of maximum likelihood estimates in logistic regression models. <i>Biometrika<\/i> <b>71<\/b> 1\u201310.","DOI":"10.1093\/biomet\/71.1.1"},{"key":"10","doi-asserted-by":"publisher","unstructured":"<small>B\u00fchlmann, P.<\/small> and <small>van de Geer, S.<\/small> (2011). <i>Statistics for High-Dimensional Data<\/i>: <i>Methods, Theory and Applications<\/i>. <i>Springer Series in Statistics<\/i>. Springer, Heidelberg.","DOI":"10.1007\/978-3-642-20192-9"},{"key":"22","unstructured":"<small>Goodfellow, I.<\/small>, <small>Bengio, Y.<\/small> and <small>Courville, A.<\/small> (2016). <i>Deep Learning<\/i>. <i>Adaptive Computation and Machine Learning<\/i>. MIT Press, Cambridge, MA."},{"key":"30","doi-asserted-by":"publisher","unstructured":"<small>Lin, Y.<\/small> (2000). Tensor product space ANOVA models. <i>Ann. Statist.<\/i> <b>28<\/b> 734\u2013755.","DOI":"10.1214\/aos\/1015951996"},{"key":"48","doi-asserted-by":"publisher","unstructured":"<small>Wahba, G.<\/small>, <small>Wang, Y.<\/small>, <small>Gu, C.<\/small>, <small>Klein, R.<\/small> and <small>Klein, B.<\/small> (1995). Smoothing spline ANOVA for exponential families, with application to the Wisconsin Epidemiological Study of Diabetic Retinopathy. <i>Ann. Statist.<\/i> <b>23<\/b> 1865\u20131895.","DOI":"10.1214\/aos\/1034713638"},{"key":"44","doi-asserted-by":"crossref","unstructured":"<small>Tibshirani, R.<\/small> (1996). Regression shrinkage and selection via the lasso. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>58<\/b> 267\u2013288.","DOI":"10.1111\/j.2517-6161.1996.tb02080.x"},{"key":"49","unstructured":"<small>Wooldridge, J. M.<\/small> (2010). <i>Econometric Analysis of Cross Section and Panel Data<\/i>, 2nd ed. MIT Press, Cambridge, MA."},{"key":"18","doi-asserted-by":"publisher","unstructured":"<small>Fan, J.<\/small> and <small>Lv, J.<\/small> (2008). Sure independence screening for ultrahigh dimensional feature space. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>70<\/b> 849\u2013911.","DOI":"10.1111\/j.1467-9868.2008.00674.x"},{"key":"24","doi-asserted-by":"publisher","unstructured":"<small>Hastie, T.<\/small>, <small>Tibshirani, R.<\/small> and <small>Friedman, J.<\/small> (2009). <i>The Elements of Statistical Learning<\/i>: <i>Data Mining, Inference, and Prediction<\/i>, 2nd ed. <i>Springer Series in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"37","doi-asserted-by":"publisher","unstructured":"<small>RAVIKUMAR, P.<\/small>, <small>WAINWRIGHT, M. J.<\/small> and <small>LAFFERTY, J. D.<\/small> (2010). High-dimensional Ising model selection using <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_2\"><msub><mrow><mi>\u2113<\/mi><\/mrow><mrow><mn>1<\/mn><\/mrow><\/msub><\/math>-regularized logistic regression. <i>Ann. Statist.<\/i> <b>38<\/b> 1287\u20131319.","DOI":"10.1214\/09-AOS691"},{"key":"47","doi-asserted-by":"crossref","unstructured":"<small>VANSTEELANDT, S.<\/small> and <small>DUKES, O.<\/small> (2022). Assumption-lean inference for generalised linear model parameters. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>84<\/b> 657\u2013685.","DOI":"10.1111\/rssb.12504"},{"key":"1","doi-asserted-by":"crossref","unstructured":"<small>AGRESTI, A.<\/small> and <small>KATERI, M.<\/small> (2011). <i>Categorical Data Analysis<\/i>. Springer.","DOI":"10.1007\/978-3-642-04898-2_161"},{"key":"3","doi-asserted-by":"crossref","unstructured":"<small>ANGRIST, J. D.<\/small> and <small>PISCHKE, J.-S.<\/small> (2009). <i>Mostly Harmless Econometrics<\/i>: <i>An Empiricist\u2019s Companion<\/i>. Princeton Univ. Press.","DOI":"10.1515\/9781400829828"},{"key":"4","unstructured":"<small>ANTONIADIS, A.<\/small> and <small>OPPENHEIM, G.<\/small> (2012). <i>Wavelets and Statistics<\/i>. Springer."},{"key":"5","unstructured":"<small>BAHADUR, R. R.<\/small> (1961). A representation of the joint distribution of responses to <i>n<\/i> dichotomous items. In <i>Studies in Item Analysis and Prediction<\/i>. <i>Stanford Mathematical Studies in the Social Sciences, VI<\/i> 158\u2013168. Stanford Univ. Press, Stanford, CA."},{"key":"8","doi-asserted-by":"crossref","unstructured":"<small>BERRY, W. D.<\/small>, <small>DEMERITT, J. H.<\/small> and <small>ESAREY, J.<\/small> (2010). Testing for interaction in binary logit and probit models: Is a product term essential? <i>Amer. J. Polit. Sci.<\/i> <b>54<\/b> 248\u2013266.","DOI":"10.1111\/j.1540-5907.2009.00429.x"},{"key":"9","doi-asserted-by":"crossref","unstructured":"<small>BROWN, B.<\/small>, <small>ZHANG, K.<\/small> and <small>MENG, X.-L.<\/small> (2025). Supplement to \u201cBELIEF in Dependence: Leveraging Atomic Linearity in Data Bits for Rethinking Generalized Linear Models.\u201d <a href=\"https:\/\/doi.org\/10.1214\/25-AOS2493SUPPA\">https:\/\/doi.org\/10.1214\/25-AOS2493SUPPA<\/a>, <a href=\"https:\/\/doi.org\/10.1214\/25-AOS2493SUPPB\">https:\/\/doi.org\/10.1214\/25-AOS2493SUPPB<\/a>","DOI":"10.1214\/25-AOS2493SUPPB"},{"key":"12","unstructured":"<small>CHEN, Y.<\/small>, <small>LI, X.<\/small>, <small>LIU, J.<\/small> and <small>YING, Z.<\/small> (2016). A fused latent and graphical model for multivariate binary data. arXiv preprint arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1606.08925\">1606.08925<\/a>."},{"key":"13","doi-asserted-by":"publisher","unstructured":"<small>CHERNOFF, H.<\/small> (2004). Information, for testing the equality of two probabilities, from the margins of the <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_1\"><mn>2<\/mn><mo>\u00d7<\/mo><mn>2<\/mn><\/math> table. <i>J. Statist. Plann. Inference<\/i> <b>121<\/b> 209\u2013214.","DOI":"10.1016\/S0378-3758(03)00135-6"},{"key":"15","doi-asserted-by":"crossref","unstructured":"<small>COX, D. R.<\/small> (1972). The analysis of multivariate binary data. <i>Appl. Stat.<\/i> 113\u2013120.","DOI":"10.2307\/2346482"},{"key":"17","doi-asserted-by":"publisher","unstructured":"<small>ESARY, J. D.<\/small>, <small>PROSCHAN, F.<\/small> and <small>WALKUP, D. W.<\/small> (1967). Association of random variables, with applications. <i>Ann. Math. Statist.<\/i> <b>38<\/b> 1466\u20131474.","DOI":"10.1214\/aoms\/1177698701"},{"key":"19","doi-asserted-by":"publisher","unstructured":"<small>FAN, Y.<\/small>, <small>KONG, Y.<\/small>, <small>LI, D.<\/small> and <small>ZHENG, Z.<\/small> (2015). Innovated interaction screening for high-dimensional nonlinear classification. <i>Ann. Statist.<\/i> <b>43<\/b> 1243\u20131272.","DOI":"10.1214\/14-AOS1308"},{"key":"21","doi-asserted-by":"publisher","unstructured":"<small>GELMAN, A.<\/small> and <small>HENNIG, C.<\/small> (2017). Beyond subjective and objective in statistics. <i>J. R. Stat. Soc. A, General<\/i> <b>180<\/b> 967\u20131033.","DOI":"10.1111\/rssa.12276"},{"key":"23","unstructured":"<small>GUPTA, C.<\/small>, <small>PODKOPAEV, A.<\/small> and <small>RAMDAS, A.<\/small> (2020). Distribution-free binary classification: Prediction sets, confidence intervals and calibration. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>33<\/b> 3711\u20133723."},{"key":"28","doi-asserted-by":"publisher","unstructured":"<small>LI, J.<\/small>, <small>ZHANG, W.<\/small>, <small>WANG, P.<\/small>, <small>LI, Q.<\/small>, <small>ZHANG, K.<\/small> and <small>LIU, Y.<\/small> (2023). Nonparametric prediction distribution from resolution-wise regression with heterogeneous data. <i>J. Bus. Econom. Statist.<\/i> <b>41<\/b> 1157\u20131172.","DOI":"10.1080\/07350015.2022.2115498"},{"key":"29","doi-asserted-by":"publisher","unstructured":"<small>LI, X.<\/small> and <small>MENG, X.-L.<\/small> (2021). A multi-resolution theory for approximating infinite-<i>p<\/i>-zero-<i>n<\/i>: Transitional inference, individualized predictions, and a world without bias-variance tradeoff. <i>J. Amer. Statist. Assoc.<\/i> <b>116<\/b> 353\u2013367.","DOI":"10.1080\/01621459.2020.1844210"},{"key":"31","doi-asserted-by":"publisher","unstructured":"<small>MCCULLAGH, P.<\/small> (2000). Invariance and factorial models. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>62<\/b> 209\u2013256. With discussion and a reply by the author.","DOI":"10.1111\/1467-9868.00229"},{"key":"35","doi-asserted-by":"crossref","unstructured":"<small>RAINEY, C.<\/small> (2016). Compression and conditional effects: A product term is essential when using logistic regression to test for interaction. <i>Political Science Research and Methods<\/i> <b>4<\/b> 621\u2013639.","DOI":"10.1017\/psrm.2015.59"},{"key":"36","doi-asserted-by":"crossref","unstructured":"<small>RAINEY, C.<\/small> (2016). Dealing with separation in logistic regression models. <i>Polit. Anal.<\/i> <b>24<\/b> 339\u2013355.","DOI":"10.1093\/pan\/mpw014"},{"key":"40","unstructured":"<small>RUSSETT, B. M.<\/small> and <small>ONEAL, J. R.<\/small> (2001). <i>Triangulating Peace<\/i>: <i>Democracy, Interdependence, and International Organizations<\/i>. W. W. Norton, New York."},{"key":"41","unstructured":"<small>SPECKMAN, P. L.<\/small>, <small>LEE, J.<\/small> and <small>SUN, D.<\/small> (2009). Existence of the MLE and propriety of posteriors for a general multinomial choice model. <i>Statist. Sinica<\/i> <b>19<\/b> 731\u2013748."},{"key":"43","unstructured":"<small>THANEI, G.-A.<\/small>, <small>MEINSHAUSEN, N.<\/small> and <small>SHAH, R. D.<\/small> (2018). The <math xmlns=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" overflow=\"scroll\" id=\"eq_3\"><mi mathvariant=\"italic\">x<\/mi><mi mathvariant=\"italic\">y<\/mi><mi mathvariant=\"italic\">z<\/mi><\/math> algorithm for fast interaction search in high-dimensional data. <i>J. Mach. Learn. Res.<\/i> <b>19<\/b> Paper No. 37, 42."},{"key":"45","doi-asserted-by":"crossref","unstructured":"<small>VALIANT, L. G.<\/small> (1984). A theory of the learnable. <i>Commun. ACM<\/i> <b>27<\/b> 1134\u20131142.","DOI":"10.1145\/1968.1972"},{"key":"46","doi-asserted-by":"publisher","unstructured":"<small>VANDERWEELE, T. J.<\/small> and <small>ROBINS, J. M.<\/small> (2008). Empirical and counterfactual conditions for sufficient cause interactions. <i>Biometrika<\/i> <b>95<\/b> 49\u201361.","DOI":"10.1093\/biomet\/asm090"},{"key":"50","doi-asserted-by":"publisher","unstructured":"<small>XIANG, S.<\/small>, <small>ZHANG, W.<\/small>, <small>LIU, S.<\/small>, <small>HOADLEY, K. A.<\/small>, <small>PEROU, C. M.<\/small>, <small>ZHANG, K.<\/small> and <small>MARRON, J. S.<\/small> (2023). Pairwise nonlinear dependence analysis of genomic data. <i>Ann. Appl. Stat.<\/i> <b>17<\/b> 2924\u20132943.","DOI":"10.1214\/23-aoas1745"},{"key":"51","doi-asserted-by":"publisher","unstructured":"<small>ZHANG, K.<\/small> (2017). Spherical cap packing asymptotics and rank-extreme detection. <i>IEEE Trans. Inf. Theory<\/i> <b>63<\/b> 4572\u20134584.","DOI":"10.1109\/TIT.2017.2700202"},{"key":"53","unstructured":"<small>ZHANG, K.<\/small>, <small>ZHANG, W.<\/small>, <small>ZHAO, Z.<\/small> and <small>ZHOU, W.<\/small> (2021). BEAUTY powered BEAST. arXiv preprint arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2103.00674\">2103.00674<\/a>."},{"key":"54","unstructured":"<small>ZHANG, W.<\/small>, <small>ZHAO, Z.<\/small>, <small>BAIOCCHI, M.<\/small>, <small>LI, Y.<\/small> and <small>ZHANG, K.<\/small> (2021). SorBET: A Fast and Powerful Algorithm to Test Dependence of Variables. Submitted."}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T16:43:38Z","timestamp":1791391418000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-53\/issue-3\/BELIEF-in-dependence--Leveraging-atomic-linearity-in-data-bits\/10.1214\/25-AOS2493.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,1]]},"references-count":54,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,6,1]]}},"URL":"https:\/\/doi.org\/10.1214\/25-aos2493","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,1]]}}}