{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,8]],"date-time":"2026-05-08T15:40:30Z","timestamp":1778254830527,"version":"3.51.4"},"reference-count":49,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,9,2]],"date-time":"2022-09-02T00:00:00Z","timestamp":1662076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001665","name":"MIAI@Grenoble Alpes","doi-asserted-by":"publisher","award":["ANR-19-P3IA-0003"],"award-info":[{"award-number":["ANR-19-P3IA-0003"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this study, we focus on mixed data which are either observations of univariate random variables which can be quantitative or qualitative, or observations of multivariate random variables such that each variable can include both quantitative and qualitative components. We first propose a novel method, called CMIh, to estimate conditional mutual information taking advantages of the previously proposed approaches for qualitative and quantitative data. We then introduce a new local permutation test, called LocAT for local adaptive test, which is well adapted to mixed data. Our experiments illustrate the good behaviour of CMIh and LocAT, and show their respective abilities to accurately estimate conditional mutual information and to detect conditional (in)dependence for mixed data.<\/jats:p>","DOI":"10.3390\/e24091234","type":"journal-article","created":{"date-parts":[[2022,9,5]],"date-time":"2022-09-05T23:35:57Z","timestamp":1662420957000},"page":"1234","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Conditional Mutual Information Estimator for Mixed Data and an Associated Conditional Independence Test"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4695-5059","authenticated-orcid":false,"given":"Lei","family":"Zan","sequence":"first","affiliation":[{"name":"Department of Mathematics, Information and Communication Sciences, Universit\u00e9 Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France"},{"name":"R&D Department, EasyVista, 38000 Grenoble, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8579-0010","authenticated-orcid":false,"given":"Anouar","family":"Meynaoui","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Information and Communication Sciences, Universit\u00e9 Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3571-3636","authenticated-orcid":false,"given":"Charles K.","family":"Assaad","sequence":"additional","affiliation":[{"name":"R&D Department, EasyVista, 38000 Grenoble, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8360-1834","authenticated-orcid":false,"given":"Emilie","family":"Devijver","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Information and Communication Sciences, Universit\u00e9 Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8858-3233","authenticated-orcid":false,"given":"Eric","family":"Gaussier","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Information and Communication Sciences, Universit\u00e9 Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Spirtes, P., Glymour, C.N., Scheines, R., and Heckerman, D. (2000). Causation, Prediction, and Search, MIT Press.","DOI":"10.7551\/mitpress\/1754.001.0001"},{"key":"ref_2","unstructured":"Whittaker, J. (2009). Graphical Models in Applied Multivariate Statistics, Wiley Publishing."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Vinh, N., Chan, J., and Bailey, J. (2014, January 27\u201331). Reconsidering mutual information based feature selection: A statistical significance view. Proceedings of the AAAI Conference on Artificial Intelligence, Quebec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.8953"},{"key":"ref_4","unstructured":"Thomas, M., and Joy, A.T. (2006). Elements of Information Theory, Wiley-Interscience."},{"key":"ref_5","first-page":"2769","article-title":"Measuring and testing dependence by correlation of distances","volume":"35","author":"Rizzo","year":"2007","journal-title":"Ann. Stat."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gretton, A., Bousquet, O., Smola, A., and Sch\u00f6lkopf, B. (2005, January 8\u201311). Measuring statistical dependence with Hilbert-Schmidt norms. Proceedings of the International Conference on Algorithmic Learning Theory, Singapore.","DOI":"10.1007\/11564089_7"},{"key":"ref_7","unstructured":"Gretton, A., Smola, A., Bousquet, O., Herbrich, R., Belitski, A., Augath, M., Murayama, Y., Pauls, J., Sch\u00f6lkopf, B., and Logothetis, N. (2005, January 6\u20138). Kernel constrained covariance for dependence measurement. Proceedings of the International Workshop on Artificial Intelligence and Statistics, Hastings, Barbados."},{"key":"ref_8","unstructured":"P\u00f3czos, B., Ghahramani, Z., and Schneider, J. (2012). Copula-based kernel dependency measures. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1093\/biomet\/asz024","article-title":"Nonparametric independence testing via mutual information","volume":"106","author":"Berrett","year":"2019","journal-title":"Biometrika"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/S0019-9958(78)90026-8","article-title":"A definition of conditional mutual information for arbitrary ensembles","volume":"38","author":"Wyner","year":"1978","journal-title":"Inf. Control."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1002\/j.1538-7305.1948.tb00917.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"204101","DOI":"10.1103\/PhysRevLett.99.204101","article-title":"Partial Mutual Information for Coupling Analysis of Multivariate Time Series","volume":"99","author":"Frenzel","year":"2007","journal-title":"Phys. Rev. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"026214","DOI":"10.1103\/PhysRevE.77.026214","article-title":"Inferring the directionality of coupling with conditional mutual information","volume":"77","author":"Vejmelka","year":"2008","journal-title":"Phys. Rev. E"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Scott, D.W. (2015). Multivariate Density Estimation: Theory, Practice, and Visualization, John Wiley & Sons.","DOI":"10.1002\/9781118575574"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Cabeli, V., Verny, L., Sella, N., Uguzzoni, G., Verny, M., and Isambert, H. (2020). Learning clinical networks from medical records based on information estimates in mixed-type data. PLoS Comput. Biol., 16.","DOI":"10.1371\/journal.pcbi.1007866"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Marx, A., Yang, L., and van Leeuwen, M. (May, January 29). Estimating conditional mutual information for discrete-continuous mixtures using multi-dimensional adaptive histograms. Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), SIAM, Virtual Event.","DOI":"10.1137\/1.9781611976700.44"},{"key":"ref_17","first-page":"17","article-title":"Nonparametric entropy estimation: An overview","volume":"6","author":"Beirlant","year":"1997","journal-title":"Int. J. Math. Stat. Sci."},{"key":"ref_18","first-page":"9","article-title":"Sample estimate of the entropy of a random vector","volume":"23","author":"Kozachenko","year":"1987","journal-title":"Probl. Peredachi Informatsii"},{"key":"ref_19","first-page":"301","article-title":"Nearest neighbor estimates of entropy","volume":"23","author":"Singh","year":"2003","journal-title":"Am. J. Math. Manag. Sci."},{"key":"ref_20","unstructured":"Singh, S., and P\u00f3czos, B. (2016, January 5\u201310). Finite-sample analysis of fixed-k nearest neighbor density functional estimators. Proceedings of the Advances in Neural Information Processing Systems 29 (NIPS 2016), Barcelona, Spain."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ross, B.C. (2014). Mutual Information between Discrete and Continuous Data Sets. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0087357"},{"key":"ref_23","unstructured":"Gao, W., Kannan, S., Oh, S., and Viswanath, P. (2017, January 4\u20139). Estimating mutual information for discrete-continuous mixtures. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_24","unstructured":"Rahimzamani, A., Asnani, H., Viswanath, P., and Kannan, S. (2018, January 3\u20138). Estimators for multivariate information measures in general probability spaces. Proceedings of the Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Montreal, QC, Canada."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1109\/TIT.2020.3024886","article-title":"Conditional Mutual Information Estimation for Mixed, Discrete and Continuous Data","volume":"67","author":"Mesner","year":"2020","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"31883","DOI":"10.1109\/ACCESS.2019.2903568","article-title":"Survey of state-of-the-art mixed data clustering algorithms","volume":"7","author":"Ahmad","year":"2019","journal-title":"IEEE Access"},{"key":"ref_27","unstructured":"Mukherjee, S., Asnani, H., and Kannan, S. (2020, January 22\u201325). CCMI: Classifier based conditional mutual information estimation. Proceedings of the 35th Uncertainty in Artificial Intelligence Conference, Tel Aviv, Israel."},{"key":"ref_28","unstructured":"Mondal, A., Bhattacharjee, A., Mukherjee, S., Asnani, H., Kannan, S., and Prathosh, A. (2020, January 3\u20136). C-MI-GAN: Estimation of conditional mutual information using minmax formulation. Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), Virtual."},{"key":"ref_29","unstructured":"Meynaoui, A. (2019). New Developments around Dependence Measures for Sensitivity Analysis: Application to Severe Accident Studies for Generation IV Reactors. [Ph.D. Thesis, INSA de Toulouse]."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1514","DOI":"10.1214\/19-AOS1857","article-title":"The hardness of conditional independence testing and the generalised covariance measure","volume":"48","author":"Shah","year":"2020","journal-title":"Ann. Stat."},{"key":"ref_31","unstructured":"Fukumizu, K., Gretton, A., Sun, X., and Sch\u00f6lkopf, B. (2007, January 3\u20136). Kernel measures of conditional dependence. Proceedings of the Advances in Neural Information Processing Systems 20 (NIPS 2007), Vancouver, BC, Canada."},{"key":"ref_32","unstructured":"Zhang, K., Peters, J., Janzing, D., and Sch\u00f6lkopf, B. (2011, January 14\u201317). Kernel-Based Conditional Independence Test and Application in Causal Discovery. Proceedings of the Twenty-Seventh Conference on Uncertainty in Artificial Intelligence, UAI\u201911, Barcelona, Spain."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Strobl, E.V., Zhang, K., and Visweswaran, S. (2019). Approximate kernel-based conditional independence tests for fast non-parametric causal discovery. J. Causal Inference, 7.","DOI":"10.1515\/jci-2018-0017"},{"key":"ref_34","unstructured":"Zhang, Q., Filippi, S., Flaxman, S., and Sejdinovic, D. (2017, January 11\u201315). Feature-to-Feature Regression for a Two-Step Conditional Independence Test. Proceedings of the Association for Uncertainty in Artificial Intelligence, UAI 2017, Sydney, Australia."},{"key":"ref_35","unstructured":"Doran, G., Muandet, K., Zhang, K., and Sch\u00f6lkopf, B. (2014, January 23\u201327). A Permutation-Based Kernel Conditional Independence Test. Proceedings of the Association for Uncertainty in Artificial Intelligence UAI, Quebec City, QC, Canada."},{"key":"ref_36","first-page":"723","article-title":"A kernel two-sample test","volume":"13","author":"Gretton","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s41060-018-0097-y","article-title":"Constraint-based causal discovery with mixed data","volume":"6","author":"Tsagris","year":"2018","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Berry, K.J., Johnston, J.E., and Mielke, P.W. (2018). Permutation statistical methods. The Measurement of Association, Springer.","DOI":"10.1007\/978-3-319-98926-6"},{"key":"ref_39","unstructured":"Runge, J. (2018, January 9\u201311). Conditional independence testing based on a nearest-neighbor estimator of conditional mutual information. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics 2018, Lanzarote, Spain."},{"key":"ref_40","unstructured":"Manoukian, E.B. (2022). Mathematical Nonparametric Statistics, Taylor & Francis."},{"key":"ref_41","unstructured":"Antos, A., and Kontoyiannis, I. (2001, January 24\u201329). Estimating the entropy of discrete distributions. Proceedings of the IEEE International Symposium on Information Theory 2001, Washington, DC, USA."},{"key":"ref_42","unstructured":"Vollmer, M., Rutter, I., and B\u00f6hm, K. (2018, January 26\u201329). On Complexity and Efficiency of Mutual Information Estimation on Static and Dynamic Data. Proceedings of the EDBT, Vienna, Austria."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1145\/361002.361007","article-title":"Multidimensional binary search trees used for associative searching","volume":"18","author":"Bentley","year":"1975","journal-title":"Commun. ACM"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1198\/016214504000000539","article-title":"Exact and approximate stepdown methods for multiple hypothesis testing","volume":"100","author":"Romano","year":"2005","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_45","first-page":"1103","article-title":"Distinguishing Cause from Effect Using Observational Data: Methods and Benchmarks","volume":"17","author":"Mooij","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1097\/01.wnr.0000224769.92454.5d","article-title":"Abnormal neural activity in children with attention deficit hyperactivity disorder: A resting-state functional magnetic resonance imaging study","volume":"17","author":"Cao","year":"2006","journal-title":"Neuroreport"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1111\/j.1469-7610.2007.01750.x","article-title":"ADHD and gender: Are risks and sequela of ADHD the same for boys and girls?","volume":"48","author":"Bauermeister","year":"2007","journal-title":"J. Child Psychol. Psychiatry"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1023\/A:1005170730653","article-title":"Etiology of inattention and hyperactivity\/impulsivity in a community sample of twins with learning difficulties","volume":"28","author":"Willcutt","year":"2000","journal-title":"J. Abnorm. Child Psychol."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Cui, R., Groot, P., and Heskes, T. (2016, January 19\u201323). Copula PC algorithm for causal discovery from mixed data. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Riva del Garda, Italy.","DOI":"10.1007\/978-3-319-46227-1_24"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1234\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:22:28Z","timestamp":1760142148000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/9\/1234"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,2]]},"references-count":49,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["e24091234"],"URL":"https:\/\/doi.org\/10.3390\/e24091234","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,2]]}}}