{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T10:16:50Z","timestamp":1768731410265,"version":"3.49.0"},"reference-count":23,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,8,1]],"date-time":"2024-08-01T00:00:00Z","timestamp":1722470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003246","name":"NWO","doi-asserted-by":"publisher","award":["CS.001"],"award-info":[{"award-number":["CS.001"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We calculate the average differential entropy of a q-component Gaussian mixture in Rn. For simplicity, all components have covariance matrix \u03c321, while the means {Wi}i=1q are i.i.d. Gaussian vectors with zero mean and covariance s21. We obtain a series expansion in \u03bc=s2\/\u03c32 for the average differential entropy up to order O(\u03bc2), and we provide a recipe to calculate higher-order terms. Our result provides an analytic approximation with a quantifiable order of magnitude for the error, which is not achieved in previous literature.<\/jats:p>","DOI":"10.3390\/e26080659","type":"journal-article","created":{"date-parts":[[2024,8,1]],"date-time":"2024-08-01T15:26:53Z","timestamp":1722526013000},"page":"659","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Average Entropy of Gaussian Mixtures"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8311-6161","authenticated-orcid":false,"given":"Basheer","family":"Joudeh","sequence":"first","affiliation":[{"name":"Department of Computer Science and Mathematics, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1409-4127","authenticated-orcid":false,"given":"Boris","family":"\u0160kori\u0107","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Mathematics, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,1]]},"reference":[{"key":"ref_1","unstructured":"Zhu, H., Guo, R., Shen, J., Liu, J., Liu, C., Xue, X.X., Zhang, L., and Mao, S. (2024). The Local Dark Matter Kinematic Substructure Based on LAMOST K Giants. arXiv."},{"key":"ref_2","unstructured":"Turner, W., Martini, P., Kara\u00e7ayl\u0131, N.G., Aguilar, J., Ahlen, S., Brooks, D., Claybaugh, T., de la Macorra, A., Dey, A., and Doel, P. (2024). New measurements of the Lyman-\u03b1 forest continuum and effective optical depth with LyCAN and DESI Y1 data. arXiv."},{"key":"ref_3","unstructured":"Wu, Y., Chen, M., Li, Z., Wang, M., and Wei, Y. (2024). Theoretical insights for diffusion guidance: A case study for gaussian mixture models. arXiv."},{"key":"ref_4","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sulam, J., Romano, Y., and Elad, M. (2016, January 16\u201318). Gaussian mixture diffusion. Proceedings of the 2016 IEEE International Conference on the Science of Electrical Engineering (ICSEE), Eilat, Israel.","DOI":"10.1109\/ICSEE.2016.7806173"},{"key":"ref_6","unstructured":"Guo, H., Lu, C., Bao, F., Pang, T., Yan, S., Du, C., and Li, C. (2024). Gaussian Mixture Solvers for Diffusion Models. Adv. Neural Inf. Process. Syst., 36."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Turan, N., B\u00f6ck, B., Chan, K.J., Fesl, B., Burmeister, F., Joham, M., Fettweis, G., and Utschick, W. (2024). Wireless Channel Prediction via Gaussian Mixture Models. arXiv.","DOI":"10.1109\/WSA61681.2024.10512246"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1109\/TCCN.2023.3342409","article-title":"Gaussian Mixture Model Based Anomaly Detection for Defense Against Byzantine Attack in Cooperative Spectrum Sensing","volume":"10","author":"Parmar","year":"2023","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"53583","DOI":"10.1109\/ACCESS.2018.2871514","article-title":"Physical layer authentication enhancement using a Gaussian mixture model","volume":"6","author":"Qiu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2705","DOI":"10.1093\/bioinformatics\/btq498","article-title":"Model-based clustering of microarray expression data via latent Gaussian mixture models","volume":"26","author":"McNicholas","year":"2010","journal-title":"Bioinformatics"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1093\/bioinformatics\/18.2.287","article-title":"Inference of a genetic network by a combined approach of cluster analysis and graphical Gaussian modeling","volume":"18","author":"Toh","year":"2002","journal-title":"Bioinformatics"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"194110","DOI":"10.1063\/1.5025058","article-title":"A path integral methodology for obtaining thermodynamic properties of nonadiabatic systems using Gaussian mixture distributions","volume":"148","author":"Raymond","year":"2018","journal-title":"J. Chem. Phys."},{"key":"ref_13","unstructured":"Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. (2015, January 7\u20139). Deep unsupervised learning using nonequilibrium thermodynamics. Proceedings of the International Conference on Machine Learning, PMLR, Lille, France."},{"key":"ref_14","unstructured":"Cover, T., and Thomas, J. (1999). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Michalowicz, J.V., Nichols, J.M., and Bucholtz, F. (2008). Calculation of differential entropy for a mixed Gaussian distribution. Entropy, 10.","DOI":"10.3390\/entropy-e10030200"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1109\/LSP.2016.2606661","article-title":"Guaranteed bounds on the Kullback\u2013Leibler divergence of univariate mixtures","volume":"23","author":"Nielsen","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_17","unstructured":"Nielsen, F., and Nock, R. (2016). A series of maximum entropy upper bounds of the differential entropy. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Hershey, J.R., and Olsen, P.A. (2007, January 15\u201320). Approximating the Kullback Leibler divergence between Gaussian mixture models. Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing-ICASSP\u201907, IEEE, Honolulu, HI, USA.","DOI":"10.1109\/ICASSP.2007.366913"},{"key":"ref_19","unstructured":"(2003, January 13\u201316). An efficient image similarity measure based on approximations of KL-divergence between two Gaussian mixtures. Proceedings of the Ninth IEEE International Conference on Computer Vision, Nice, France."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huber, M.F., Bailey, T., Durrant-Whyte, H., and Hanebeck, U.D. (2008, January 20\u201322). On entropy approximation for Gaussian mixture random vectors. Proceedings of the 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, Seoul, Republic of Korea.","DOI":"10.1109\/MFI.2008.4648062"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kolchinsky, A., and Tracey, B.D. (2017). Estimating mixture entropy with pairwise distances. Entropy, 19.","DOI":"10.3390\/e19070361"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1673","DOI":"10.1109\/83.650120","article-title":"Secure spread spectrum watermarking for multimedia","volume":"6","author":"Cox","year":"1997","journal-title":"IEEE Trans. Image Process."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wu, S., Huang, Y., Guan, H., Zhang, S., and Liu, J. (2022). ECSS: High-Embedding-Capacity Audio Watermarking with Diversity Reception. Entropy, 22.","DOI":"10.3390\/e24121843"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/659\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:28:24Z","timestamp":1760110104000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/8\/659"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,1]]},"references-count":23,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["e26080659"],"URL":"https:\/\/doi.org\/10.3390\/e26080659","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,1]]}}}