{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T18:40:15Z","timestamp":1783622415145,"version":"3.55.0"},"reference-count":30,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2016,5,20]],"date-time":"2016-05-20T00:00:00Z","timestamp":1463702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"973 Program","doi-asserted-by":"publisher","award":["2015CB351703"],"award-info":[{"award-number":["2015CB351703"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61372152"],"award-info":[{"award-number":["61372152"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Entropy has been widely employed as a measure of variability for problems, such as machine learning and signal processing. In this paper, we provide some new insights into the behaviors of entropy as a measure of multivariate variability. The relationships between multivariate entropy (joint or total marginal) and traditional measures of multivariate variability, such as total dispersion and generalized variance, are investigated. It is shown that for the jointly Gaussian case, the joint entropy (or entropy power) is equivalent to the generalized variance, while total marginal entropy is equivalent to the geometric mean of the marginal variances and total marginal entropy power is equivalent to the total dispersion. The smoothed multivariate entropy (joint or total marginal) and the kernel density estimation (KDE)-based entropy estimator (with finite samples) are also studied, which, under certain conditions, will be approximately equivalent to the total dispersion (or a total dispersion estimator), regardless of the data distribution.<\/jats:p>","DOI":"10.3390\/e18050196","type":"journal-article","created":{"date-parts":[[2016,5,21]],"date-time":"2016-05-21T19:24:37Z","timestamp":1463858677000},"page":"196","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Insights into Entropy as a Measure of Multivariate Variability"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1710-3818","authenticated-orcid":false,"given":"Badong","family":"Chen","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianji","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haiquan","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Southwest Jiaotong University, Chengdu 610031, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jose","family":"Principe","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"},{"name":"Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2016,5,20]]},"reference":[{"key":"ref_1","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, Wiley."},{"key":"ref_2","unstructured":"Renyi, A. (July, January 20). On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"Tsallis","year":"1988","journal-title":"J. Stat. Phys."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Principe, J.C. (2010). Information Theoretic Learning: R\u00e9nyi\u2019s Entropy and Kernel Perspectives, Springer.","DOI":"10.1007\/978-1-4419-1570-2"},{"key":"ref_5","first-page":"217","article-title":"Exponential entropy as a measure of extent of a distribution","volume":"53","author":"Campbell","year":"1966","journal-title":"Probab. Theory Relat. Fields"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, B., Zhu, Y., Hu, J., and Principe, J.C. (2013). System Parameter Identification: Information Criteria and Algorithms, Elsevier.","DOI":"10.1016\/B978-0-12-404574-3.00005-1"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/34.982897","article-title":"Information theoretic clustering","volume":"24","author":"Gokcay","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/SP-M.2006.248709","article-title":"From linear adaptive filtering to nonlinear information processing","volume":"23","author":"Erdogmus","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_9","first-page":"27","article-title":"Conditional likelihood maximization: A unifying framework for information theoretic feature selection","volume":"13","author":"Brown","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1184","DOI":"10.1109\/TSP.2011.2178406","article-title":"Survival information potential: A new criterion for adaptive system training","volume":"60","author":"Chen","year":"2012","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/TSP.2002.1011217","article-title":"An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems","volume":"50","author":"Erdogmus","year":"2002","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1109\/TNN.2002.1031936","article-title":"Generalized information potential criterion for adaptive system training","volume":"13","author":"Erdogmus","year":"2002","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1007\/s00034-007-9004-9","article-title":"Stochastic gradient algorithm under (h, \u03c6)-entropy criterion","volume":"26","author":"Chen","year":"2007","journal-title":"Circuits Syst. Signal Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.1109\/TNN.2010.2050212","article-title":"Mean-square convergence analysis of ADALINE training with minimum error entropy criterion","volume":"21","author":"Chen","year":"2010","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"966","DOI":"10.3390\/e14050966","article-title":"Some further results on the minimum error entropy estimation","volume":"14","author":"Chen","year":"2012","journal-title":"Entropy"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2311","DOI":"10.3390\/e14112311","article-title":"On the Smoothed Minimum Error Entropy Criterion","volume":"14","author":"Chen","year":"2012","journal-title":"Entropy"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.neucom.2013.04.037","article-title":"Kernel minimum error entropy algorithm","volume":"121","author":"Chen","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"213","DOI":"10.2307\/2528939","article-title":"Note: Disadvantages of the Generalized Variance as a Measure of Variability","volume":"27","author":"Kowal","year":"1971","journal-title":"Biometrics"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/S0167-9473(96)00042-4","article-title":"A measure for total variability in multivariate normal distribution","volume":"23","author":"Mustonen","year":"1997","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/S0047-259X(02)00061-1","article-title":"Descriptive measures of multivariate scatter and linear dependence","volume":"85","year":"2003","journal-title":"J. Multivar. Anal."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1093\/biomet\/24.3-4.471","article-title":"Certain generalizations in the analysis of variance","volume":"24","author":"Wilks","year":"1932","journal-title":"Biometrika"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1111\/j.1469-185X.1965.tb00806.x","article-title":"Statistical methods in systematics","volume":"40","author":"Sokal","year":"1965","journal-title":"Biol. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"189","DOI":"10.2307\/2528472","article-title":"Note: A Measure of \u201cOverall Variability\u201d in Populations","volume":"24","author":"Goodman","year":"1968","journal-title":"Biometrics"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"238701","DOI":"10.1103\/PhysRevLett.103.238701","article-title":"Granger causality and transfer entropy are equivalent for Gaussian variables","volume":"103","author":"Barnett","year":"2009","journal-title":"Phys. Rev. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"041907","DOI":"10.1103\/PhysRevE.81.041907","article-title":"Multivariate Granger causality and generalized variance","volume":"81","author":"Barrett","year":"2010","journal-title":"Phys. Rev. E"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/TIT.1966.1053843","article-title":"The information generating function of a probability distribution","volume":"12","author":"Golomb","year":"1966","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"708","DOI":"10.1109\/TIT.2014.2383379","article-title":"Entropy power inequality for the Renyi entropy","volume":"61","author":"Bobkov","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating Mutual Information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_29","unstructured":"Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, Chapman & Hall."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/TIT.2010.2090193","article-title":"Information theoretic proofs of entropy power inequalities","volume":"57","author":"Rioul","year":"2011","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/5\/196\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:24:13Z","timestamp":1760210653000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/5\/196"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,20]]},"references-count":30,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2016,5]]}},"alternative-id":["e18050196"],"URL":"https:\/\/doi.org\/10.3390\/e18050196","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,5,20]]}}}