{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T01:44:51Z","timestamp":1770515091092,"version":"3.49.0"},"reference-count":54,"publisher":"MIT Press","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2018,4]]},"abstract":"<jats:p> While Shannon's mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality. This article focuses on effective approximation methods for evaluating mutual information in the context of neural population coding. For large but finite neural populations, we derive several information-theoretic asymptotic bounds and approximation formulas that remain valid in high-dimensional spaces. We prove that optimizing the population density distribution based on these approximation formulas is a convex optimization problem that allows efficient numerical solutions. Numerical simulation results confirmed that our asymptotic formulas were highly accurate for approximating mutual information for large neural populations. In special cases, the approximation formulas are exactly equal to the true mutual information. We also discuss techniques of variable transformation and dimensionality reduction to facilitate computation of the approximations. <\/jats:p>","DOI":"10.1162\/neco_a_01056","type":"journal-article","created":{"date-parts":[[2018,1,18]],"date-time":"2018-01-18T00:39:05Z","timestamp":1516235945000},"page":"885-944","source":"Crossref","is-referenced-by-count":9,"title":["Information-Theoretic Bounds and Approximations in Neural Population Coding"],"prefix":"10.1162","volume":"30","author":[{"given":"Wentao","family":"Huang","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD 21205, U.S.A., and Cognitive and Intelligent Lab and Information Science Academy of China Electronics Technology Group Corporation, Beijing 100846, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kechen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, MD 21205, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1162\/089976699300016827"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1162\/0899766053429426"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1992.4.4.559"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1990.2.3.308"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1038\/355161a0"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(97)00121-1"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1162\/08997660260293247"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1038\/14731"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511804441"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1038\/nn1228"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300017115"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1037\/h0026857"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0915-05.2005"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2006.06.030"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1109\/18.54897"},{"key":"B16","unstructured":"Cover, T. M. & Thomas, J. A. (2006). Elements of information (2nd ed.) New York: Wiley-Interscience."},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1007\/BF00326705"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00638"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.13-07-02758.1993"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1152\/jn.01106.2006"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2005.844072"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1038\/nature02768"},{"key":"B23","first-page":"1","author":"Huang W.","year":"2017","journal-title":"Proceedings of the 51st Annual Conference on Information Sciences and Systems"},{"key":"B24","author":"Huang W.","year":"2017","journal-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR)"},{"key":"B25","volume-title":"Theory of probability","author":"Jeffreys H.","year":"1961","edition":"3"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.86.4958"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.76.026209"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.066138"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1126\/science.1089662"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1167\/6.3.8"},{"key":"B31","author":"MacKay D. J. C.","year":"2003","journal-title":"Information theory, inference and learning algorithms"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1991.66.3.794"},{"key":"B33","first-page":"95","author":"Miller G. A.","year":"1955","journal-title":"Information theory in psychology: Problems and methods II-B"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1038\/381607a0"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1987.57.1.162"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1162\/089976603321780272"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1038\/35039062"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2578"},{"issue":"3","key":"B39","first-page":"81","volume":"37","author":"Rao C. R.","year":"1945","journal-title":"Bulletin of the Calcutta Mathematical Society"},{"key":"B40","author":"Rieke F.","year":"1997","journal-title":"Spikes: Exploring the neural code"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1109\/18.481776"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb00917.x"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.64.051904"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1993.70.2.640"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.97.098102"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1995.7.2.399"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1038\/360068a0"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1109\/9780470544198"},{"key":"B49","first-page":"1025","volume-title":"Proc. 24th Allerton Conf. Communication, Control and Computing","author":"Verdu S.","year":"1986"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300014728"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00804"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00292"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1998.79.2.1017"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.1162\/089976699300016809"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01056","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:42:21Z","timestamp":1615585341000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/30\/4\/885-944\/8375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,4]]},"references-count":54,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,4]]}},"alternative-id":["10.1162\/neco_a_01056"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01056","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4]]}}}