{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T14:38:09Z","timestamp":1775054289576,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2017,11,2]],"date-time":"2017-11-02T00:00:00Z","timestamp":1509580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Discovering a correlation from one variable to another variable is of fundamental scientific and practical interest. While existing correlation measures are suitable for discovering average correlation, they fail to discover hidden or potential correlations. To bridge this gap, (i) we postulate a set of natural axioms that we expect a measure of potential correlation to satisfy; (ii) we show that the rate of information bottleneck, i.e., the hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we provide a novel estimator to estimate the hypercontractivity coefficient from samples; and (iv) we provide numerical experiments demonstrating that this proposed estimator discovers potential correlations among various indicators of WHO datasets, is robust in discovering gene interactions from gene expression time series data, and is statistically more powerful than the estimators for other correlation measures in binary hypothesis testing of canonical examples of potential correlations.<\/jats:p>","DOI":"10.3390\/e19110586","type":"journal-article","created":{"date-parts":[[2017,11,3]],"date-time":"2017-11-03T04:43:13Z","timestamp":1509684193000},"page":"586","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Discovering Potential Correlations via Hypercontractivity"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2256-5729","authenticated-orcid":false,"given":"Hyeji","family":"Kim","sequence":"first","affiliation":[{"name":"Coordinated Science Lab, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"},{"name":"Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"}]},{"given":"Weihao","family":"Gao","sequence":"additional","affiliation":[{"name":"Coordinated Science Lab, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"},{"name":"Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"}]},{"given":"Sreeram","family":"Kannan","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, University of Washington, Seattle, WA 98195, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8975-8306","authenticated-orcid":false,"given":"Sewoong","family":"Oh","sequence":"additional","affiliation":[{"name":"Coordinated Science Lab, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"},{"name":"Department of Industrial and Enterprise Systems Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"}]},{"given":"Pramod","family":"Viswanath","sequence":"additional","affiliation":[{"name":"Coordinated Science Lab, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"},{"name":"Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA"}]}],"member":"1968","published-online":{"date-parts":[[2017,11,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1098\/rspl.1895.0041","article-title":"Note on Regression and Inheritance in the Case of Two Parents","volume":"58","author":"Pearson","year":"1895","journal-title":"Proc. R. Soc. Lond."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1017\/S0305004100013517","article-title":"A Connection between Correlation and Contingency","volume":"31","author":"Hirschfeld","year":"1935","journal-title":"Math. Proc. Camb. Philos. Soc."},{"key":"ref_3","first-page":"364","article-title":"Das statistische Problem der Korrelation als Variations-und Eigenwertproblem und sein Zusammenhang mit der Ausgleichsrechnung","volume":"21","author":"Gebelein","year":"1941","journal-title":"J. Appl. Math. Mech."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1007\/BF02024507","article-title":"On measures of dependence","volume":"10","year":"1959","journal-title":"Acta Math. Hung."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1126\/science.1205438","article-title":"Detecting Novel Associations in Large Data Sets","volume":"334","author":"Reshef","year":"2011","journal-title":"Science"},{"key":"ref_6","first-page":"2769","article-title":"Measuring and testing dependence by correlation of distances","volume":"35","author":"Rizzo","year":"2007","journal-title":"Ann. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1250689","DOI":"10.1126\/science.1250689","article-title":"Conditional density-based analysis of T cell signaling in single-cell data","volume":"346","author":"Krishnaswamy","year":"2014","journal-title":"Science"},{"key":"ref_8","unstructured":"Tishby, N., Pereira, F.C., and Bialek, W. (1999, January 22\u201324). The Information Bottleneck Method. Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing, Champaign, IL, USA."},{"key":"ref_9","first-page":"1265","article-title":"A Divisive Information-Theoretic Feature Clustering Algorithm for Text Classification","volume":"3","author":"Dhillon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_10","first-page":"1183","article-title":"Distributional Word Clusters vs. Words for Text Categorization","volume":"3","author":"Bekkerman","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Anantharam, V., Gohari, A.A., Kamath, S., and Nair, C. (arXiv, 2013). On Maximal Correlation, Hypercontractivity, and the Data Processing Inequality studied by Erkip and Cover, arXiv.","DOI":"10.1109\/ISIT.2014.6875389"},{"key":"ref_12","unstructured":"Davies, E.B., Gross, L., and Simone, B. (1992). Hypercontractivity: A Bibliographic Review. Ideas and Methods in Quantum and Statistical Physics (Oslo, 1988), Cambridge University Press."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/0022-1236(73)90091-8","article-title":"Construction of quantum fields from Markoff fields","volume":"12","author":"Nelson","year":"1973","journal-title":"J. Funct. Anal."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kahn, J., Kalai, G., and Linial, N. (1988, January 24\u201326). The Influence of Variables on Boolean Functions. Proceedings of the IEEE Computer Society SFCS \u201988 29th Annual Symposium on Foundations of Computer Science, White Plains, NY, USA.","DOI":"10.1109\/SFCS.1988.21923"},{"key":"ref_15","unstructured":"O\u2019Donnell, R. (2014). Analysis of Boolean Functions, Cambridge University Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"335","DOI":"10.5802\/aif.357","article-title":"\u00c9tude des coefficients de Fourier des fonctions de Lp(G)","volume":"20","author":"Bonami","year":"1970","journal-title":"Annales de l\u2019institut Fourier"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/1970980","article-title":"Inequalities in Fourier analysis","volume":"102","author":"Beckner","year":"1975","journal-title":"Ann. Math."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1215\/S0012-7094-75-04237-4","article-title":"Hypercontractivity and logarithmic Sobolev inequalities for the Clifford-Dirichlet form","volume":"42","author":"Gross","year":"1975","journal-title":"Duke Math. J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1214\/aop\/1176995937","article-title":"Spreading of sets in product spaces and hypercontraction of the Markov operator","volume":"4","author":"Ahlswede","year":"1976","journal-title":"Ann. Probab."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1062","DOI":"10.1007\/s00039-013-0229-4","article-title":"On reverse hypercontractivity","volume":"23","author":"Mossel","year":"2013","journal-title":"Geom. Funct. Anal."},{"key":"ref_21","unstructured":"Nair, C., and Kamath, S. Personal communication."},{"key":"ref_22","unstructured":"Alemi, A.A., Fischer, I., Dillion, J.V., and Murphy, K. (arXiv, 2017). Deep variational information bottleneck, arXiv."},{"key":"ref_23","unstructured":"Achille, A., and Soatto, S. (arXiv, 2016). Information Dropout: Learning Optimal Representations Through Noisy Computation, arXiv."},{"key":"ref_24","unstructured":"Nair, C. (2014, January 9\u201314). An extremal inequality related to hypercontractivity of Gaussian random variables. Proceedings of the Information Theory and Applications Workshop, San Diego, CA, USA."},{"key":"ref_25","unstructured":"Gao, W., Kannan, S., Oh, S., and Viswanath, P. (2016, January 19\u201324). Conditional Dependence via Shannon Capacity: Axioms, Estimators and Applications. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_26","unstructured":"Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., and Ng, A.Y. (July, January 28). Multimodal deep learning. Proceedings of the 28th International Conference on Machine Learning (ICML-11), Washington, DC, USA."},{"key":"ref_27","unstructured":"Srivastava, N., and Salakhutdinov, R.R. (2012, January 3\u20136). Multimodal learning with deep boltzmann machines. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_28","unstructured":"Andrew, G., Arora, R., Bilmes, J., and Livescu, K. (2013, January 16\u201321). Deep canonical correlation analysis. Proceedings of the International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_29","unstructured":"Makur, A., and Zheng, L. (arXiv, 2017). Linear Bounds between Contraction Coefficients for f-Divergences, arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1137\/0128010","article-title":"On Sequences of Pairs of Dependent Random Variables","volume":"28","author":"Witsenhausen","year":"1975","journal-title":"SIAM J. Appl. Math."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1214\/aoms\/1177704583","article-title":"Mutual information and maximal correlation as measures of dependence","volume":"33","author":"Bell","year":"1962","journal-title":"Ann. Math. Stat."},{"key":"ref_32","unstructured":"Nair, C. (2014, January 26\u201328). Equivalent Formulations of Hypercontractivity Using Information Measures. Proceedings of the 2014 International Zurich Seminar on Communications, Zurich, Switzerland."},{"key":"ref_33","first-page":"165","article-title":"Information Bottleneck for Gaussian Variables","volume":"6","author":"Chechik","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","unstructured":"Michaeli, T., Wang, W., and Livescu, K. (2016, January 19\u201324). Nonparametric Canonical Correlation Analysis. Proceedings of the ICML\u201916 33rd International Conference on International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_35","unstructured":"Simon, N., and Tibshirani, R. (arXiv, 2014). Comment on \u201cDetecting Novel Associations In Large Data Sets\u201d by Reshef Et Al, Science 16 December 2011, arXiv."},{"key":"ref_36","unstructured":"Gorfine, M., Heller, R., and Heller, Y. (2017, October 30). Comment on Detecting Novel Associations in Large Data Sets. Available online: http:\/\/emotion.technion.ac.il\/~gorfinm\/filesscience6.pdf."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1214\/aoms\/1177730388","article-title":"Low Moments for Small Samples: A Comparative Study of Order Statistics","volume":"18","author":"Hastings","year":"1947","journal-title":"Ann. Math. Stat."},{"key":"ref_38","unstructured":"McBean, E.A., and Rovers, F. (1998). Statistical Procedures for Analysis of Environmental Monitoring Data and Assessment, Prentice-Hall."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1061\/(ASCE)0733-9372(2007)133:9(909)","article-title":"Replacing outliers and missing values from activated sludge data using Kohonen self-organizing map","volume":"133","author":"Rustum","year":"2007","journal-title":"J. Environ. Eng."},{"key":"ref_40","unstructured":"Kumar, G. (2017, October 30). Binary R\u00e9nyi Correlation: A Simpler Proof of Witsenhausen\u2019s Result and a Tight Lower Bound. Available online: http:\/\/www.gowthamiitm.com\/research\/Witsenhausen_simpleproof.pdf."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/11\/586\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:47:51Z","timestamp":1760208471000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/11\/586"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,2]]},"references-count":40,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2017,11]]}},"alternative-id":["e19110586"],"URL":"https:\/\/doi.org\/10.3390\/e19110586","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,2]]}}}