{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T12:36:57Z","timestamp":1752669417200,"version":"3.37.3"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T00:00:00Z","timestamp":1704153600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T00:00:00Z","timestamp":1704153600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002543","name":"Pusan National University","doi-asserted-by":"publisher","award":["2 Year Grant"],"award-info":[{"award-number":["2 Year Grant"]}],"id":[{"id":"10.13039\/501100002543","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Mutual information (MI) quantifies the statistical dependency between a pair of random variables and plays a central role in signal processing and data analysis. Recent advances in machine learning have enabled the estimation of MI from a dataset using the expressive power of neural networks. In this study, we conducted a comparative experimental analysis of several existing neural estimators of MI between random vectors that model power spectrum features. We explored alternative models of power spectrum features by leveraging information-theoretic data processing inequality and bijective transformations. Empirical results demonstrated that each neural estimator of MI covered in this study has its limitations. In practical applications, we recommend the collective use of existing neural estimators in a complementary manner for the problem of estimating MI between power spectrum features.<\/jats:p>","DOI":"10.1186\/s13634-023-01092-1","type":"journal-article","created":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T11:02:24Z","timestamp":1704193344000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["An experimental study of neural estimators of the mutual information between random vectors modeling power spectrum features"],"prefix":"10.1186","volume":"2024","author":[{"given":"Donghoon","family":"Shin","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5070-897X","authenticated-orcid":false,"given":"Hyung Soon","family":"Kim","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,1,2]]},"reference":[{"key":"1092_CR1","volume-title":"Elements of Information Theory","author":"TM Cover","year":"1991","unstructured":"T.M. Cover, J.A. Thomas, Elements of Information Theory (Wiley, New Jersey, 1991)"},{"issue":"3","key":"1092_CR2","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1109\/2.36","volume":"21","author":"R Linsker","year":"1988","unstructured":"R. Linsker, Self-organization in a perceptual network. Computer 21(3), 105\u2013117 (1988). https:\/\/doi.org\/10.1109\/2.36","journal-title":"Computer"},{"issue":"5","key":"1092_CR3","doi-asserted-by":"publisher","first-page":"691","DOI":"10.1162\/neco.1992.4.5.691","volume":"4","author":"R Linsker","year":"1992","unstructured":"R. Linsker, Local synaptic learning rules suffice to maximize mutual information in a linear network. Neural Comput. 4(5), 691\u2013702 (1992). https:\/\/doi.org\/10.1162\/neco.1992.4.5.691","journal-title":"Neural Comput."},{"issue":"1","key":"1092_CR4","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1088\/0954-898X\/7\/1\/003","volume":"7","author":"S Becker","year":"1996","unstructured":"S. Becker, Mutual information maximization: models of cortical self-organization. Netw. Comput. Neural Syst. 7(1), 7 (1996). https:\/\/doi.org\/10.1088\/0954-898X\/7\/1\/003","journal-title":"Netw. Comput. Neural Syst."},{"issue":"6","key":"1092_CR5","doi-asserted-by":"publisher","first-page":"1129","DOI":"10.1162\/neco.1995.7.6.1129","volume":"7","author":"AJ Bell","year":"1995","unstructured":"A.J. Bell, T.J. Sejknowksi, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7(6), 1129\u20131159 (1995). https:\/\/doi.org\/10.1162\/neco.1995.7.6.1129","journal-title":"Neural Comput."},{"key":"1092_CR6","doi-asserted-by":"publisher","unstructured":"L.\u00a0Bahl, P.\u00a0Brown, P.\u00a0De\u00a0Souza, R.\u00a0Mercer, Maximum mutual information estimation of hidden Markov model parameters for speech recognition, in Proc. IEEE ICASSP\u201986, pp. 49\u201352 (1986). https:\/\/doi.org\/10.1109\/ICASSP.1986.1169179","DOI":"10.1109\/ICASSP.1986.1169179"},{"key":"1092_CR7","unstructured":"R.\u00a0D.\u00a0Hjelm, A.\u00a0Fedorov, S.\u00a0Lavoie-Marchildon, K.\u00a0Grewal, P.\u00a0Bachman, A.\u00a0Trischler, Y.\u00a0Bengio, Learning deep representations by mutual information estimation and maximization, in International Conference on Learning Representations (2018)"},{"key":"1092_CR8","unstructured":"M.\u00a0Tschannen, J.\u00a0Djolonga, P.\u00a0K.\u00a0Rubenstein, S.\u00a0Gelly, M.\u00a0Lucic, On Mutual Information Maximization for Representation Learning, in International Conference on Learning Representations (2019)"},{"key":"1092_CR9","unstructured":"A.\u00a0Van\u00a0Den\u00a0Oord, Y.\u00a0Li, O.\u00a0Vinyals, Representation learning with contrastive predictive coding (2018), arXiv:1807.03748"},{"key":"1092_CR10","unstructured":"D.\u00a0McAllester, K.\u00a0Stratos, Formal limitations on the measurement of mutual information, in International Conference on Artificial Intelligence and Statistics, PMLR, pp. 875\u2013884 (2020)"},{"key":"1092_CR11","unstructured":"S.\u00a0Gao, G.\u00a0Ver\u00a0Steeg, A.\u00a0Galstyan, Efficient estimation of mutual information for strongly dependent variables, in International Conference on Artificial Intelligence and Statistics, PMLR, pp. 277\u2013286 (2015)"},{"issue":"6","key":"1092_CR12","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.066138","volume":"69","author":"A Kraskov","year":"2004","unstructured":"A. Kraskov, H. St\u00f6gbauer, P. Grassberger, Estimating mutual information. Phys. Rev. E 69(6), 066138 (2004). https:\/\/doi.org\/10.1103\/PhysRevE.69.066138","journal-title":"Phys. Rev. E"},{"key":"1092_CR13","unstructured":"B.\u00a0Poole, S.\u00a0Ozair, A.\u00a0Van\u00a0Den\u00a0Oord, A.\u00a0Alemi, G.\u00a0Tucker, On variational bounds of mutual information, in International Conference on Machine Learning, PMLR, p 5171\u20135180 (2019)"},{"key":"1092_CR14","unstructured":"J.\u00a0Song, S.\u00a0Ermon, Understanding the Limitations of Variational Mutual Information Estimators, in International Conference on Learning Representations (2019)"},{"key":"1092_CR15","unstructured":"L.\u00a0Dinh, J.\u00a0Sohl-Dickstein, S.\u00a0Bengio, Density estimation using real NVP, in International Conference on Learning Representations (2017)"},{"issue":"12","key":"1092_CR16","first-page":"199","volume":"2","author":"IM Gelfand","year":"1959","unstructured":"I.M. Gelfand, Calculation of the amount of information about a random function contained in another such function. Ann. Math. Soc. Transl. Ser. 2(12), 199\u2013246 (1959)","journal-title":"Ann. Math. Soc. Transl. Ser."},{"issue":"1","key":"1092_CR17","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1109\/TASL.2013.2281574","volume":"22","author":"J Taghia","year":"2014","unstructured":"J. Taghia, R. Martin, Objective intelligibility measures based on mutual information for speech subjected to speech enhancement processing. IEEE\/ACM Trans. Audio Speech Lang. Process. 22(1), 6\u201316 (2014). https:\/\/doi.org\/10.1109\/TASL.2013.2281574","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"issue":"2","key":"1092_CR18","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1109\/TASLP.2013.2295914","volume":"22","author":"J Jensen","year":"2014","unstructured":"J. Jensen, C.H. Taal, Speech intelligibility prediction based on mutual information. IEEE\/ACM Trans. Audio Speech Lang. Process. 22(2), 430\u2013440 (2014). https:\/\/doi.org\/10.1109\/TASLP.2013.2295914","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"issue":"8","key":"1092_CR19","doi-asserted-by":"publisher","first-page":"1694","DOI":"10.1109\/TASLP.2017.2714424","volume":"25","author":"S Khademi","year":"2017","unstructured":"S. Khademi, R.C. Hendriks, W.B. Kleijn, Intelligibility Enhancement Based on Mutual Information. IEEE\/ACM Trans. Audio Speech Lang. Process. 25(8), 1694\u20131708 (2017). https:\/\/doi.org\/10.1109\/TASLP.2017.2714424","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"issue":"320","key":"1092_CR20","first-page":"201","volume":"16","author":"D Barber","year":"2004","unstructured":"D. Barber, F. Agakov, The IM algorithm: a variational approach to information maximization. Adv. Neural. Inf. Process. Syst. 16(320), 201 (2004)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"issue":"11","key":"1092_CR21","doi-asserted-by":"publisher","first-page":"5847","DOI":"10.1109\/TIT.2010.2068870","volume":"56","author":"X Nguyen","year":"2010","unstructured":"X. Nguyen, M.J. Wainwright, M.I. Jordan, Estimating divergence functionals and the likelihood ratio by convex risk minimization. IEEE Trans. Inf. Theory 56(11), 5847\u20135861 (2010). https:\/\/doi.org\/10.1109\/TIT.2010.2068870","journal-title":"IEEE Trans. Inf. Theory"},{"key":"1092_CR22","unstructured":"M.\u00a0I.\u00a0Belghazi, A.\u00a0Baratin, S.\u00a0Rajeshwar, S.\u00a0Ozair, Y.\u00a0Bengio, A.\u00a0Courville, D.\u00a0Hjelm Mutual Information Neural Estimation, in International Conference on Machine Learning, PMLR, pp. 531\u2013540 (2018)"},{"issue":"7","key":"1092_CR23","doi-asserted-by":"publisher","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","volume":"18","author":"GE Hinton","year":"2006","unstructured":"G.E. Hinton, S. Osindero, Y.W. Teh, A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527\u20131554 (2006). https:\/\/doi.org\/10.1162\/neco.2006.18.7.1527","journal-title":"Neural Comput."},{"key":"1092_CR24","doi-asserted-by":"publisher","unstructured":"R.\u00a0Salakhutdinov, I.\u00a0Murray, On the quantitative analysis of deep belief networks, in International Conference on Machine Learning. PMLR, pp. 872\u2013879 (2008). https:\/\/doi.org\/10.1145\/1390156.1390266","DOI":"10.1145\/1390156.1390266"},{"key":"1092_CR25","unstructured":"S.\u00a0Gershman, N.\u00a0Goodman, Amortized inference in probabilistic reasoning, in Proceedings of the Annual Meeting of the Cognitive Science Society (2014)"},{"key":"1092_CR26","doi-asserted-by":"crossref","unstructured":"D.P.\u00a0Kingma, M.\u00a0Welling, An introduction to variational autoencoders (2019), arXiv:1906.02691","DOI":"10.1561\/9781680836233"},{"key":"1092_CR27","unstructured":"D.\u00a0Rezende, S.\u00a0Mohamed, Variational inference with normalizing flows, in International Conference on Machine Learning. PMLR, pp. 1530\u20131538 (2015)"},{"issue":"3","key":"1092_CR28","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/TIM.1986.6499210","volume":"IM\u201335","author":"J Schoukens","year":"1986","unstructured":"J. Schoukens, J. Renneboog, Modeling the noise influence on the Fourier coefficients after a discrete Fourier transform. IEEE Trans. Instrum. Meas. IM\u201335(3), 278\u2013286 (1986). https:\/\/doi.org\/10.1109\/TIM.1986.6499210","journal-title":"IEEE Trans. Instrum. Meas."},{"issue":"6","key":"1092_CR29","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.1109\/TIT.2003.811910","volume":"49","author":"RK Mallik","year":"2003","unstructured":"R.K. Mallik, On multivariate Rayleigh and exponential distributions. IEEE Trans. Inf. Theory 49(6), 1499\u20131515 (2003). https:\/\/doi.org\/10.1109\/TIT.2003.811910","journal-title":"IEEE Trans. Inf. Theory"},{"issue":"7","key":"1092_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/ASP.2005.1110","volume":"2005","author":"T Lotter","year":"2005","unstructured":"T. Lotter, P. Vary, Speech enhancement by MAP spectral amplitude estimation using a super-Gaussian speech model. EURASIP J. Adv. Signal Process. 2005(7), 1\u201317 (2005). https:\/\/doi.org\/10.1155\/ASP.2005.1110","journal-title":"EURASIP J. Adv. Signal Process."},{"issue":"5","key":"1092_CR31","doi-asserted-by":"publisher","first-page":"845","DOI":"10.1109\/TSA.2005.851927","volume":"13","author":"R Martin","year":"2005","unstructured":"R. Martin, Speech enhancement based on minimum mean-square error estimation and super Gaussian priors. IEEE Trans. Speech Audio Process. 13(5), 845\u2013856 (2005). https:\/\/doi.org\/10.1109\/TSA.2005.851927","journal-title":"IEEE Trans. Speech Audio Process."},{"issue":"3","key":"1092_CR32","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1016\/j.specom.2005.02.011","volume":"47","author":"I Cohen","year":"2005","unstructured":"I. Cohen, Speech enhancement using super-Gaussian speech models and noncausal a priori SNR estimation. Speech Commun. 47(3), 336\u2013350 (2005). https:\/\/doi.org\/10.1016\/j.specom.2005.02.011","journal-title":"Speech Commun."},{"issue":"10","key":"1092_CR33","doi-asserted-by":"publisher","first-page":"3608","DOI":"10.1109\/TIT.2005.855598","volume":"51","author":"NC Sagias","year":"2005","unstructured":"N.C. Sagias, G.K. Karagiannidis, Gaussian class multivariate Weibull distributions: theory and applications in fading channels. IEEE Trans. Inf. Theory 51(10), 3608\u20133619 (2005). https:\/\/doi.org\/10.1109\/TIT.2005.855598","journal-title":"IEEE Trans. Inf. Theory"},{"key":"1092_CR34","unstructured":"D.\u00a0P.\u00a0Kingma, J.\u00a0Ba, Adam: a method for stochastic optimization (2014), arXiv:1412.6980"},{"key":"1092_CR35","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1093\/oso\/9780198538493.001.0001","volume-title":"Neural Networks for Pattern Recognition","author":"CM Bishop","year":"1995","unstructured":"C.M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, London, 1995), pp.298\u2013300"},{"issue":"9","key":"1092_CR36","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1145\/359146.359153","volume":"22","author":"DHD West","year":"1979","unstructured":"D.H.D. West, Updating mean and variance estimates: an improved method. Commun. ACM 22(9), 532\u2013535 (1979). https:\/\/doi.org\/10.1145\/359146.359153","journal-title":"Commun. ACM"}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-023-01092-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-023-01092-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-023-01092-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T11:08:03Z","timestamp":1704193683000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-023-01092-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,2]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["1092"],"URL":"https:\/\/doi.org\/10.1186\/s13634-023-01092-1","relation":{},"ISSN":["1687-6180"],"issn-type":[{"type":"electronic","value":"1687-6180"}],"subject":[],"published":{"date-parts":[[2024,1,2]]},"assertion":[{"value":"20 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 December 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 January 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declaration"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2"}}