{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:23:51Z","timestamp":1760243031889,"version":"build-2065373602"},"reference-count":51,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2015,6,9]],"date-time":"2015-06-09T00:00:00Z","timestamp":1433808000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The Fisher information constitutes a natural measure for the sensitivity of a probability distribution with respect to a set of parameters. An implementation of the stationarity principle for synaptic learning in terms of the Fisher information results in a Hebbian self-limiting learning rule for synaptic plasticity. In the present work, we study the dependence of the solutions to this rule in terms of the moments of the input probability distribution and find a preference for non-Gaussian directions, making it a suitable candidate for independent component analysis (ICA). We confirm in a numerical experiment that a neuron trained under these rules is able to find the independent components in the non-linear bars problem. The specific form of the plasticity rule depends on the transfer function used, becoming a simple cubic polynomial of the membrane potential for the case of the rescaled error function. The cubic learning rule is also an excellent approximation for other transfer functions, as the standard sigmoidal, and can be used to show analytically that the proposed plasticity rules are selective for directions in the space of presynaptic neural activities characterized by a negative excess kurtosis.<\/jats:p>","DOI":"10.3390\/e17063838","type":"journal-article","created":{"date-parts":[[2015,6,10]],"date-time":"2015-06-10T14:06:12Z","timestamp":1433945172000},"page":"3838-3856","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["The Fisher Information as a Neural Guiding Principle for Independent Component Analysis"],"prefix":"10.3390","volume":"17","author":[{"given":"Rodrigo","family":"Echeveste","sequence":"first","affiliation":[{"name":"Institute for Theoretical Physics, Goethe University Frankfurt, Frankfurt, 60438, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Samuel","family":"Eckmann","sequence":"additional","affiliation":[{"name":"Institute for Theoretical Physics, Goethe University Frankfurt, Frankfurt, 60438, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claudius","family":"Gros","sequence":"additional","affiliation":[{"name":"Institute for Theoretical Physics, Goethe University Frankfurt, Frankfurt, 60438, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,6,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1097\/00004647-200110000-00001","article-title":"An energy budget for signaling in the grey matter of the brain","volume":"21","author":"Attwell","year":"2001","journal-title":"J. Cereb. Blood Flow Metab."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"R203","DOI":"10.1152\/ajpregu.1981.241.3.R203","article-title":"Ratio of central nervous system to body metabolism in vertebrates: its constancy and functional basis","volume":"241","author":"Mink","year":"1981","journal-title":"Am. J. Physiol.-Regul. Integr. Comp. Physiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1792","DOI":"10.1242\/jeb.017574","article-title":"Energy limitation as a selective pressure on the evolution of sensory systems","volume":"211","author":"Niven","year":"2008","journal-title":"J. Exp. Biol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1038\/nrn3214","article-title":"The economy of brain network organization","volume":"13","author":"Bullmore","year":"2012","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_5","unstructured":"Jordan, M.I., LeCun, Y., and Solla, S.A. (2001). Advances in Neural Information Processing Systems: Proceedings of The First 12 Conferences, The MIT Press."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1038\/9173","article-title":"How voltage-dependent conductances can adapt to maximize the information encoded by neuronal firing rate","volume":"2","author":"Stemmler","year":"1999","journal-title":"Nat. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Prokopenko, M. (2014). Guided Self-Organization: Inception, Springer.","DOI":"10.1007\/978-3-642-53734-9"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1162\/neco.1992.4.4.590","article-title":"Information-based objective functions for active data selection","volume":"4","author":"MacKay","year":"1992","journal-title":"Neural Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s00158-003-0368-6","article-title":"Survey of multi-objective optimization methods for engineering","volume":"26","author":"Marler","year":"2004","journal-title":"Struct. Multidiscip. Optim."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/S0893-6080(05)80003-6","article-title":"Objective function formulation of the BCM theory of visual cortical plasticity: Statistical connections, stability conditions","volume":"5","author":"Intrator","year":"1992","journal-title":"Neural Netw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1007\/s11538-010-9564-x","article-title":"Coherent infomax as a computational goal for neural systems","volume":"73","author":"Kay","year":"2011","journal-title":"Bull. Math. Biol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"307","DOI":"10.2976\/1.3171566","article-title":"Information: currency of life","volume":"3","author":"Polani","year":"2009","journal-title":"HFSP J"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1177\/1059712310375314","article-title":"Higher coordination with less control\u2014A result of information maximization in the sensorimotor loop","volume":"18","author":"Zahedi","year":"2010","journal-title":"Adapt. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1303001","DOI":"10.1142\/S021952591303001X","article-title":"Information and self-organization of behavior","volume":"16","author":"Polani","year":"2013","journal-title":"Adv. Complex Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5232","DOI":"10.3390\/e16105232","article-title":"Entropy Methods in Guided Self-Organisation","volume":"16","author":"Prokopenko","year":"2014","journal-title":"Entropy"},{"key":"ref_16","unstructured":"Der, R., and Martius, G. (2012). The Playful Machine: Theoretical Foundation and Practical Realization of Self-Organizing Robots, Springer."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"068702","DOI":"10.1103\/PhysRevLett.105.068702","article-title":"Self-organized chaos through polyhomeostatic optimization","volume":"105","author":"Markovic","year":"2010","journal-title":"Phys. Rev. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1162\/NECO_a_00232","article-title":"Intrinsic adaptation in autonomous recurrent neural networks","volume":"24","author":"Gros","year":"2012","journal-title":"Neural Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1162\/neco.2007.19.4.885","article-title":"Synergies between intrinsic and synaptic plasticity mechanisms","volume":"19","author":"Triesch","year":"2007","journal-title":"Neural Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1162\/neco.1992.4.5.691","article-title":"Local synaptic learning rules suffice to maximize mutual information in a linear network","volume":"4","author":"Linsker","year":"1992","journal-title":"Neural Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1162\/089976603321891774","article-title":"Spike-timing-dependent plasticity and relevant mutual information maximization","volume":"15","author":"Chechik","year":"2003","journal-title":"Neural Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5239","DOI":"10.1073\/pnas.0500495102","article-title":"Generalized Bienenstock\u2013Cooper\u2013Munro rule for spiking neurons that maximizes information transmission","volume":"102","author":"Toyoizumi","year":"2005","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.tins.2009.10.001","article-title":"More than synaptic plasticity: Role of nonsynaptic plasticity in learning and memory","volume":"33","author":"Mozzachiodi","year":"2010","journal-title":"Trends Neurosci."},{"key":"ref_25","unstructured":"Strogatz, S.H. (2001). Nonlinear Dynamics and Chaos: With Applications to Physics, Biology and Chemistry, Perseus Publishing."},{"key":"ref_26","unstructured":"Hebb, D.O. (2002). The Organization of Behavior: A Neuropsychological Theory, Psychology Press."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/S0925-2312(97)00045-3","article-title":"The nonlinear PCA learning rule in independent component analysis","volume":"17","author":"Oja","year":"1997","journal-title":"Neurocomputing"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"10464","DOI":"10.1523\/JNEUROSCI.18-24-10464.1998","article-title":"Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type","volume":"18","author":"Bi","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1038\/416433a","article-title":"Spike-timing-dependent synaptic modification induced by natural spike trains","volume":"416","author":"Froemke","year":"2002","journal-title":"Nature"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1511","DOI":"10.1162\/089976603321891783","article-title":"Relating stdp to bcm","volume":"15","author":"Izhikevich","year":"2003","journal-title":"Neural Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1162\/NECO_a_00707","article-title":"Two-trace model for spike-timing-dependent synaptic plasticity","volume":"27","author":"Echeveste","year":"2015","journal-title":"Neural Comput."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Echeveste, R., and Gros, C. (2014). Generating functionals for computational intelligence: The Fisher information as an objective function for self-limiting Hebbian learning rules. Front. Robot. AI, 1.","DOI":"10.3389\/frobt.2014.00001"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1162\/neco.1995.7.6.1129","article-title":"An information-maximization approach to blind separation and blind deconvolution","volume":"7","author":"Bell","year":"1995","journal-title":"Neural Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e63400","DOI":"10.1371\/journal.pone.0063400","article-title":"Information driven self-organization of complex robotic behaviors","volume":"8","author":"Martius","year":"2013","journal-title":"PloS ONE"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1007\/BF02331346","article-title":"Forming sparse representations by local anti-Hebbian learning","volume":"64","year":"1990","journal-title":"Biol. Cybern."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1731","DOI":"10.1162\/089976698300017115","article-title":"Mutual information, Fisher information, and population coding","volume":"10","author":"Brunel","year":"1998","journal-title":"Neural Comput."},{"key":"ref_37","unstructured":"Echeveste, R., and Gros, C. (2015, January 22\u201324). An objective function for self-limiting neural plasticity rules. Bruges, Belgium."},{"key":"ref_38","unstructured":"Hyv\u00e4rinen, A., Karhunen, J., and Oja, E. (2004). Independent Component Analysis, Wiley."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"3327","DOI":"10.1016\/S0042-6989(97)00121-1","article-title":"The \u201cindependent components\u201d of natural scenes are edge filters","volume":"37","author":"Bell","year":"1997","journal-title":"Vis. Res."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1007\/BF00363954","article-title":"A theory for the use of visual orientation information which exploits the columnar structure of striate cortex","volume":"58","author":"Paradiso","year":"1988","journal-title":"Biol. Cybern."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"10749","DOI":"10.1073\/pnas.90.22.10749","article-title":"Simple models for reading neuronal population codes","volume":"90","author":"Seung","year":"1993","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1038\/nature06563","article-title":"Adaptive coding of visual information in neural populations","volume":"452","author":"Gutnisky","year":"2008","journal-title":"Nature"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1088\/0954-898X_14_2_307","article-title":"Optimal neural rate coding leads to bimodal firing rate distributions","volume":"14","author":"Bethge","year":"2003","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.biosystems.2006.04.010","article-title":"Optimal signal in sensory neurons under an extended rate coding concept","volume":"89","author":"Lansky","year":"2007","journal-title":"BioSystems"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"14272","DOI":"10.1523\/JNEUROSCI.2539-11.2011","article-title":"The effect of noise correlations in populations of diversely tuned neurons","volume":"31","author":"Ecker","year":"2011","journal-title":"J. Neurosci."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1775","DOI":"10.1103\/PhysRevA.58.1775","article-title":"Derivation of the equations of nonrelativistic quantum mechanics using the principle of minimum Fisher information","volume":"58","author":"Reginatto","year":"1998","journal-title":"Phys. Rev. A"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1037\/1082-989X.2.3.292","article-title":"On the meaning and use of kurtosis","volume":"2","author":"DeCarlo","year":"1997","journal-title":"Psychol. Methods."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/0165-1684(94)90029-9","article-title":"Independent component analysis, a new concept","volume":"36","author":"Comon","year":"1994","journal-title":"Signal Process"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1016\/S0893-6080(00)00026-5","article-title":"Independent component analysis: Algorithms and applications","volume":"13","author":"Oja","year":"2000","journal-title":"Neural Netw."},{"key":"ref_50","unstructured":"Girolami, M., and Fyfe, C. (1996, January 7). Negentropy and Kurtosis as Projection Pursuit Indices Provide Generalised ICA Algorithms. Snowmaas, Aspen, CO, USA."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1109\/TNN.2007.908636","article-title":"A class of complex ICA algorithms based on the kurtosis cost function","volume":"19","author":"Li","year":"2008","journal-title":"IEEE Trans. Neural Netw."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/3838\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:47:38Z","timestamp":1760215658000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/3838"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,6,9]]},"references-count":51,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2015,6]]}},"alternative-id":["e17063838"],"URL":"https:\/\/doi.org\/10.3390\/e17063838","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2015,6,9]]}}}