{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T06:22:33Z","timestamp":1792736553044,"version":"4.5.0"},"reference-count":48,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,22]],"date-time":"2022-01-22T00:00:00Z","timestamp":1642809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008982","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DMS 1914739"],"award-info":[{"award-number":["DMS 1914739"]}],"id":[{"id":"10.13039\/501100008982","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R01 CA129102"],"award-info":[{"award-number":["R01 CA129102"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>There has been a lot of interest in sufficient dimension reduction (SDR) methodologies, as well as nonlinear extensions in the statistics literature. The SDR methodology has previously been motivated by several considerations: (a) finding data-driven subspaces that capture the essential facets of regression relationships; (b) analyzing data in a \u2018model-free\u2019 manner. In this article, we develop an approach to interpreting SDR techniques using information theory. Such a framework leads to a more assumption-lean understanding of what SDR methods do and also allows for some connections to results in the information theory literature.<\/jats:p>","DOI":"10.3390\/e24020167","type":"journal-article","created":{"date-parts":[[2022,1,23]],"date-time":"2022-01-23T20:32:52Z","timestamp":1642969972000},"page":"167","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Sufficient Dimension Reduction: An Information-Theoretic Viewpoint"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6618-1316","authenticated-orcid":false,"given":"Debashis","family":"Ghosh","sequence":"first","affiliation":[{"name":"Department of Biostatistics and Informatics, Colorado School of Public Health, Aurora, CO 80045, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, B. (2018). Sufficient Dimension Reduction: Methods and Applications with R, CRC Press.","DOI":"10.1201\/9781315119427"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Brillinger, D.R. (2012). A generalized linear model with \u201cGaussian\u201d regressor variables. Selected Works of David Brillinger, Springer.","DOI":"10.1007\/978-1-4614-1344-8_34"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1214\/aos\/1176347254","article-title":"Regression analysis under link violation","volume":"17","author":"Li","year":"1989","journal-title":"Ann. Stat."},{"key":"ref_4","unstructured":"Tishby, N., Pereira, F., Bialek, W., Hajek, B., and Sreenivas, R. (1999, January 30). The informational bottleneck method. Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing, Florham Park, NJ, USA. Available online: https:\/\/www.bibsonomy.org\/bibtex\/15bd5efbf394791da00b09839b9a5757."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1109\/TIT.1972.1054855","article-title":"Computation of channel capacity and rate-distortion functions","volume":"18","author":"Blahut","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/TIT.1972.1054753","article-title":"An algorithm for computing the capacity of arbitrary discrete memoryless channels","volume":"18","author":"Arimoto","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_7","unstructured":"Slonim, N., and Tishby, N. (1999). Agglomerative Information Bottleneck, ACM."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Slonim, N., and Tishby, N. (2000, January 24\u201328). Document clustering using word clusters via the information bottleneck method. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece.","DOI":"10.1145\/345508.345578"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1739","DOI":"10.1162\/neco.2006.18.8.1739","article-title":"Multivariate information bottleneck","volume":"18","author":"Slonim","year":"2006","journal-title":"Neural Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tishby, N., and Zaslavsky, N. (May, January 26). Deep learning and the information bottleneck principle. Proceedings of the 2015 IEEE Information Theory Workshop (ITW), Jerusalem, Israel.","DOI":"10.1109\/ITW.2015.7133169"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"124020","DOI":"10.1088\/1742-5468\/ab3985","article-title":"On the information bottleneck theory of deep learning","volume":"2019","author":"Saxe","year":"2019","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1111\/1467-9868.03411","article-title":"An adaptive estimation of dimension reduction space","volume":"64","author":"Xia","year":"2002","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_13","first-page":"73","article-title":"Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces","volume":"5","author":"Fukumizu","year":"2004","journal-title":"J. Mach. Learn. Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1871","DOI":"10.1214\/08-AOS637","article-title":"Kernel dimension reduction in regression","volume":"37","author":"Fukumizu","year":"2009","journal-title":"Ann. Stat."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3182","DOI":"10.1214\/11-AOS932","article-title":"Principal support vector machines for linear and nonlinear sufficient dimension reduction","volume":"39","author":"Li","year":"2011","journal-title":"Ann. Stat."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1214\/12-AOS1071","article-title":"A general theory for nonlinear sufficient dimension reduction: Formulation and estimation","volume":"41","author":"Lee","year":"2013","journal-title":"Ann. Stat."},{"key":"ref_17","first-page":"165","article-title":"Information Bottleneck for Gaussian Variables","volume":"6","author":"Chechik","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1093\/biomet\/asu062","article-title":"Dimension reduction based on the Hellinger integral","volume":"102","author":"Wang","year":"2015","journal-title":"Biometrika"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/S0047-259X(03)00129-5","article-title":"Canonical correlation analysis based on information theory","volume":"91","author":"Yin","year":"2004","journal-title":"J. Multivar. Anal."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1198\/016214508000000472","article-title":"An informational measure of association and dimension reduction for multiple sets and groups with applications in morphometric analysis","volume":"103","author":"Iaci","year":"2008","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_22","first-page":"335","article-title":"Common canonical variates for independent groups using information theory","volume":"18","author":"Yin","year":"2008","journal-title":"Stat. Sin."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.jspi.2018.02.001","article-title":"A unified approach to sufficient dimension reduction","volume":"197","author":"Xue","year":"2018","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1198\/016214504000001501","article-title":"Sufficient dimension reduction via inverse regression: A minimum discrepancy approach","volume":"100","author":"Cook","year":"2005","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1752","DOI":"10.1080\/01621459.2018.1515080","article-title":"Covariate information matrix for sufficient dimension reduction","volume":"114","author":"Yao","year":"2019","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lauritzen, S.L. (1996). Graphical Models, Clarendon Press.","DOI":"10.1093\/oso\/9780198522195.001.0001"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/0304-4076(93)90114-K","article-title":"Semiparametric least squares (SLS) and weighted SLS estimation of single-index models","volume":"58","author":"Ichimura","year":"1993","journal-title":"J. Econom."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1080\/01621459.1991.10475035","article-title":"Sliced inverse regression for dimension reduction","volume":"86","author":"Li","year":"1991","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_29","unstructured":"Cook, R.D. (2009). Regression Graphics: Ideas for Studying Regressions through Graphics, John Wiley & Sons."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1016\/j.jmva.2008.01.006","article-title":"Successive direction extraction for estimating the central subspace in a multiple-index regression","volume":"99","author":"Yin","year":"2008","journal-title":"J. Multivar. Anal."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1080\/01621459.1992.10476258","article-title":"On principal Hessian directions for data visualization and dimension reduction: Another application of Stein\u2019s lemma","volume":"87","author":"Li","year":"1992","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wu, Q., Liang, F., and Mukherjee, S. (2013). Kernel sliced inverse regression: Regularization and consistency. Abstract and Applied Analysis, Hindawi.","DOI":"10.1155\/2013\/540725"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1214\/aos\/1176349155","article-title":"On almost linearity of low dimensional projections from high dimensional data","volume":"21","author":"Hall","year":"1993","journal-title":"Ann. Stat."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1214\/aos\/1021379862","article-title":"Sufficient dimension reduction in regressions with categorical predictors","volume":"30","author":"Chiaromonte","year":"2002","journal-title":"Ann. Stat."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pearl, J. (2009). Causality, Cambridge University Press.","DOI":"10.1017\/CBO9780511803161"},{"key":"ref_36","unstructured":"Berge, C. (1984). Hypergraphs: Combinatorics of Finite Sets, Elsevier."},{"key":"ref_37","unstructured":"Cover, T.M., and Thomas, J. (2006). Elements of Information Theory, John Wiley & Sons. [2nd ed.]."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1080\/00031305.2019.1592781","article-title":"Assumption lean regression","volume":"75","author":"Berk","year":"2019","journal-title":"Am. Stat."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"968","DOI":"10.1198\/016214503000000927","article-title":"Using the bootstrap to select one of a new class of dimension reduction methods","volume":"98","author":"Ye","year":"2003","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"875","DOI":"10.1093\/biomet\/asw051","article-title":"Combining eigenvalues and variation of eigenvectors for order determination","volume":"103","author":"Luo","year":"2016","journal-title":"Biometrika"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1093\/biomet\/asaa077","article-title":"On order determination by predictor augmentation","volume":"108","author":"Luo","year":"2021","journal-title":"Biometrika"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2055","DOI":"10.1214\/15-AOS1337","article-title":"Controlling the false discovery rate via knockoffs","volume":"43","author":"Barber","year":"2015","journal-title":"Ann. Stat."},{"key":"ref_43","unstructured":"Substance Abuse and Mental Health Services Administration (2020). Opioid Treatment Program (OTP) Guidance."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.drugalcdep.2012.08.002","article-title":"Buprenorphine\/naloxone and methadone effects on laboratory indices of liver health: A randomized trial","volume":"128","author":"Saxon","year":"2013","journal-title":"Drug Alcohol Depend."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1111\/1467-9868.00262","article-title":"Partial least squares estimator for single-index models","volume":"62","author":"Naik","year":"2000","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1198\/016214502388618573","article-title":"Identification of shared components in large ensembles of time series using dimension reduction","volume":"97","author":"Li","year":"2002","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_47","first-page":"1","article-title":"Online Sufficient Dimension Reduction Through Sliced Inverse Regression","volume":"21","author":"Cai","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"107768","DOI":"10.1016\/j.patcog.2020.107768","article-title":"Real-time sufficient dimension reduction through principal least squares support vector machines","volume":"112","author":"Artemiou","year":"2021","journal-title":"Pattern Recognit."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/2\/167\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:05:50Z","timestamp":1760133950000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/2\/167"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,22]]},"references-count":48,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["e24020167"],"URL":"https:\/\/doi.org\/10.3390\/e24020167","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,22]]}}}