{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:41:24Z","timestamp":1760240484017,"version":"build-2065373602"},"reference-count":69,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,6,26]],"date-time":"2019-06-26T00:00:00Z","timestamp":1561507200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000893","name":"Simons Foundation","doi-asserted-by":"publisher","award":["SFARI, Explorer Grant No. 602849"],"award-info":[{"award-number":["SFARI, Explorer Grant No. 602849"]}],"id":[{"id":"10.13039\/100000893","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Flag-Era JTC Human Brain Project","award":["SLOW-DYN"],"award-info":[{"award-number":["SLOW-DYN"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we study the statistical properties of the stationary firing-rate states of a neural network model with quenched disorder. The model has arbitrary size, discrete-time evolution equations and binary firing rates, while the topology and the strength of the synaptic connections are randomly generated from known, generally arbitrary, probability distributions. We derived semi-analytical expressions of the occurrence probability of the stationary states and the mean multistability diagram of the model, in terms of the distribution of the synaptic connections and of the external stimuli to the network. Our calculations rely on the probability distribution of the bifurcation points of the stationary states with respect to the external stimuli, calculated in terms of the permanent of special matrices using extreme value theory. While our semi-analytical expressions are exact for any size of the network and for any distribution of the synaptic connections, we focus our study on networks made of several populations, that we term \u201cstatistically homogeneous\u201d to indicate that the probability distribution of their connections depends only on the pre- and post-synaptic population indexes, and not on the individual synaptic pair indexes. In this specific case, we calculated analytically the permanent, obtaining a compact formula that outperforms of several orders of magnitude the Balasubramanian-Bax-Franklin-Glynn algorithm. To conclude, by applying the Fisher-Tippett-Gnedenko theorem, we derived asymptotic expressions of the stationary-state statistics of multi-population networks in the large-network-size limit, in terms of the Gumbel (double exponential) distribution. We also provide a Python implementation of our formulas and some examples of the results generated by the code.<\/jats:p>","DOI":"10.3390\/e21070630","type":"journal-article","created":{"date-parts":[[2019,6,27]],"date-time":"2019-06-27T02:50:15Z","timestamp":1561603815000},"page":"630","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Stationary-State Statistics of a Binary Neural Network Model with Quenched Disorder"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4278-6630","authenticated-orcid":false,"given":"Diego","family":"Fasoli","sequence":"first","affiliation":[{"name":"Laboratory of Neural Computation, Center for Neuroscience and Cognitive Systems@UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1700-8909","authenticated-orcid":false,"given":"Stefano","family":"Panzeri","sequence":"additional","affiliation":[{"name":"Laboratory of Neural Computation, Center for Neuroscience and Cognitive Systems@UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto, Italy"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1038\/nrn1949","article-title":"Variability, compensation and homeostasis in neuron and network function","volume":"7","author":"Marder","year":"2006","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1016\/S0896-6273(00)80338-4","article-title":"Heterogeneity of release probability, facilitation, and depletion at central synapses","volume":"18","author":"Dobrunz","year":"1997","journal-title":"Neuron"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1113\/jphysiol.2001.013485","article-title":"Vesicle pool partitioning influences presynaptic diversity and weighting in rat hippocampal synapses","volume":"541","author":"Waters","year":"2002","journal-title":"J. Physiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3154","DOI":"10.1523\/JNEUROSCI.23-08-03154.2003","article-title":"Variable properties in a single class of excitatory spinal synapse","volume":"23","author":"Parker","year":"2003","journal-title":"J. Neurosci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1038\/nrn2634","article-title":"The probability of neurotransmitter release: Variability and feedback control at single synapses","volume":"10","author":"Branco","year":"2009","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1016\/j.tins.2004.06.002","article-title":"Plasticity of interneuronal species diversity and parameter variance in neurological diseases","volume":"27","author":"Santhakumar","year":"2004","journal-title":"Trends Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1792","DOI":"10.1103\/PhysRevLett.35.1792","article-title":"Solvable model of a spin-glass","volume":"35","author":"Sherrington","year":"1976","journal-title":"Phys. Rev. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4384","DOI":"10.1103\/PhysRevB.17.4384","article-title":"Infinite-ranged models of spin-glasses","volume":"17","author":"Kirkpatrick","year":"1978","journal-title":"Phys. Rev. B"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"983","DOI":"10.1088\/0305-4470\/11\/5\/028","article-title":"Stability of the Sherrington-Kirkpatrick solution of a spin glass model","volume":"11","author":"Thouless","year":"1978","journal-title":"J. Phys. A Math. Gen."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1051\/jphys:01984004505084300","article-title":"Replica symmetry breaking and the nature of the spin glass phase","volume":"45","author":"Sourlas","year":"1984","journal-title":"J. Phys."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"M\u00e9zard, M., Parisi, G., and Virasoro, M. (1986). Spin Glass Theory and Beyond: An Introduction to the Replica Method and Its Applications, World Scientific Singapore.","DOI":"10.1142\/0271"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Das, M.P. (1999). Spin glasses. Physics of Novel Materials, World Scientific Singapore. Chapter 4.","DOI":"10.1142\/3864"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1498","DOI":"10.1038\/nn.3220","article-title":"Slow dynamics and high variability in balanced cortical networks with clustered connections","volume":"15","author":"Doiron","year":"2012","journal-title":"Nat. Neurosci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"56","DOI":"10.3389\/fncom.2014.00056","article-title":"Balanced neural architecture and the idling brain","volume":"8","author":"Doiron","year":"2014","journal-title":"Front. Comput. Neurosci."},{"key":"ref_15","first-page":"021039","article-title":"Balanced networks of spiking neurons with spatially dependent recurrent connections","volume":"4","author":"Rosenbaum","year":"2014","journal-title":"Phys. Rev. X"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1103\/PhysRevLett.61.259","article-title":"Chaos in random neural networks","volume":"61","author":"Sompolinsky","year":"1988","journal-title":"Phys. Rev. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1051\/jp1:1995135","article-title":"Increase in complexity in random neural networks","volume":"5","author":"Cessac","year":"1995","journal-title":"J. Phys. I France"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/neuro.10.001.2009","article-title":"A constructive mean-field analysis of multi-population neural networks with random synaptic weights and stochastic inputs","volume":"3","author":"Faugeras","year":"2009","journal-title":"Front. Comput. Neurosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"018702","DOI":"10.1103\/PhysRevLett.109.018702","article-title":"Heterogeneous connections induce oscillations in large-scale networks","volume":"109","author":"Hermann","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10955-013-0818-5","article-title":"Large deviations, dynamics and phase transitions in large stochastic and disordered neural networks","volume":"153","author":"Cabana","year":"2013","journal-title":"J. Stat. Phys."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3886","DOI":"10.1103\/PhysRevLett.71.3886","article-title":"Dynamics of fully connected attractor neural networks near saturation","volume":"71","author":"Coolen","year":"1993","journal-title":"Phys. Rev. Lett."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1146\/annurev.ne.11.030188.002231","article-title":"The control of neuron number","volume":"11","author":"Williams","year":"1988","journal-title":"Ann. Rev. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1093\/brain\/120.4.701","article-title":"The columnar organization of the neocortex","volume":"120","author":"Mountcastle","year":"1997","journal-title":"Brain"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.brainresrev.2007.07.011","article-title":"Reconstruction of an average cortical column in silico","volume":"55","author":"Helmstaedter","year":"2007","journal-title":"Brain Res. Rev."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2277","DOI":"10.1093\/cercor\/bhq067","article-title":"Number and laminar distribution of neurons in a thalamocortical projection column of rat vibrissal cortex","volume":"20","author":"Meyer","year":"2010","journal-title":"Cereb. Cortex"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1016\/j.neuroimage.2005.08.035","article-title":"fMRI resting state networks define distinct modes of long-distance interactions in the human brain","volume":"29","author":"Beckmann","year":"2006","journal-title":"NeuroImage"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"13170","DOI":"10.1073\/pnas.0700668104","article-title":"Electrophysiological signatures of resting state networks in the human brain","volume":"104","author":"Mantini","year":"2007","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"012316","DOI":"10.1103\/PhysRevE.99.012316","article-title":"Optimized brute-force algorithms for the bifurcation analysis of a binary neural network model","volume":"99","author":"Fasoli","year":"2019","journal-title":"Phys. Rev. E"},{"key":"ref_29","unstructured":"Kuznetsov, Y.A. (1998). Elements of Applied Bifurcation Theory, Springer."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"Hopfield","year":"1982","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1016\/S1383-8121(01)80017-8","article-title":"Statistical mechanics of recurrent neural networks I\u2014Statics","volume":"Volume 4","author":"Coolen","year":"2001","journal-title":"Handbook of Biological Physics"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1063\/1.1703954","article-title":"Time dependent statistics of the Ising model","volume":"4","author":"Glauber","year":"1963","journal-title":"J. Math. Phys."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"De Haan, L., and Ferreira, A. (2006). Extreme Value Theory: An Introduction, Springer.","DOI":"10.1007\/0-387-34471-3"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1111\/j.2517-6161.1972.tb00909.x","article-title":"Permanent expressions for order statistic densities","volume":"34","author":"Vaughan","year":"1972","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_35","first-page":"79","article-title":"Order statistics for nonidentically distributed variables and permanents","volume":"51","author":"Bapat","year":"1989","journal-title":"Sankhy\u0101 Ser. A"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/0024-3795(90)90332-7","article-title":"Permanents in probability and statistics","volume":"127","author":"Bapat","year":"1990","journal-title":"Linear Algebra Appl."},{"key":"ref_37","first-page":"365","article-title":"A note on order statistics for nondentically distributed variables","volume":"56","author":"Hande","year":"1994","journal-title":"Sankhy\u0101 Ser. A"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0304-3975(79)90044-6","article-title":"The complexity of computing the permanent","volume":"8","author":"Valiant","year":"1979","journal-title":"Theor. Comput. Sci."},{"key":"ref_39","unstructured":"Balasubramanian, K. (1980). Combinatorics and Diagonals of Matrices. [Ph.D. Thesis, Indian Statistical Institute]."},{"key":"ref_40","unstructured":"Bax, E., and Franklin, J. (1996). A Finite-Difference Sieve to Compute the Permanent, CalTech. Technical Report CalTech-CS-TR-96-04."},{"key":"ref_41","unstructured":"Bax, E. (1998). Finite-Difference Algorithms for Counting Problems. [Ph.D. Thesis, California Institute of Technology]."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1887","DOI":"10.1016\/j.ejc.2010.01.010","article-title":"The permanent of a square matrix","volume":"31","author":"Glynn","year":"2010","journal-title":"Eur. J. Combin."},{"key":"ref_43","unstructured":"Billingsley, P. (1995). Probability and Measure, John Wiley & Sons."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1103\/PhysRevLett.47.359","article-title":"Dynamic theory of the spin-glass phase","volume":"47","author":"Sompolinsky","year":"1981","journal-title":"Phys. Rev. Lett."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/PL00008774","article-title":"Aging of spherical spin glasses","volume":"120","author":"Dembo","year":"2001","journal-title":"Probab. Theory Relat. Fields"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Yilmaz, E., Ozer, M., Baysal, V., and Perc, M. (2016). Autapse-induced multiple coherence resonance in single neurons and neuronal networks. Sci. Rep., 6.","DOI":"10.1038\/srep30914"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1017\/S0305004100015681","article-title":"Limiting forms of the frequency distribution of the largest or smallest member of a sample","volume":"24","author":"Fisher","year":"1928","journal-title":"Math. Proc. Camb. Philos. Soc."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"423","DOI":"10.2307\/1968974","article-title":"Sur la distribution limite du terme maximum d\u2019une s\u00e9rie al\u00e9atoire","volume":"44","author":"Gnedenko","year":"1943","journal-title":"Ann. Math."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Gumbel, E.J. (1958). Statistics of Extremes, Columbia University Press.","DOI":"10.7312\/gumb92958"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"055037","DOI":"10.1088\/0143-0807\/36\/5\/055037","article-title":"Large deviations of the maximum of independent and identically distributed random variables","volume":"36","author":"Vivo","year":"2015","journal-title":"Eur. J. Phys."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/S0361-9230(99)00100-8","article-title":"Dale\u2019s principle","volume":"50","author":"Strata","year":"1999","journal-title":"Brain Res. Bull."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1258","DOI":"10.1162\/neco_a_01069","article-title":"Pattern storage, bifurcations and groupwise correlation structure of an exactly solvable asymmetric neural network model","volume":"30","author":"Fasoli","year":"2018","journal-title":"Neural Comput."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/BF00203668","article-title":"Bifurcation analysis of a neural network model","volume":"66","author":"Borisyuk","year":"1992","journal-title":"Biol. Cybern."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1177\/105971239500300405","article-title":"On the dynamics of small continuous-time recurrent neural networks","volume":"3","author":"Beer","year":"1995","journal-title":"Adapt. Behav."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1080\/net.13.2.195.216","article-title":"Complex dynamics and the structure of small neural networks","volume":"13","author":"Pasemann","year":"2002","journal-title":"Netw.-Comp. Neural"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.neucom.2004.11.030","article-title":"Input space bifurcation manifolds of recurrent neural networks","volume":"64","author":"Haschke","year":"2005","journal-title":"Neurocomputing"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1023\/A:1008384928636","article-title":"Bifurcations in nonlinear discontinuous systems","volume":"23","author":"Leine","year":"2000","journal-title":"Nonlinear Dyn."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Awrejcewicz, J., and Lamarque, C.H. (2003). Bifurcation and Chaos in Nonsmooth Mechanical Systems, World Scientific.","DOI":"10.1142\/9789812564801"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1016\/j.euromechsol.2006.04.004","article-title":"Bifurcation phenomena in non-smooth dynamical systems","volume":"25","author":"Leine","year":"2006","journal-title":"Eur. J. Mech. A-Solid"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1016\/j.physd.2012.08.002","article-title":"Dynamics and bifurcations of nonsmooth systems: A survey","volume":"241","author":"Makarenkov","year":"2012","journal-title":"Phys. D"},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1137\/140977953","article-title":"Bifurcations in the Wilson\u2013Cowan equations with nonsmooth firing rate","volume":"14","author":"Harris","year":"2015","journal-title":"SIAM J. Appl. Dyn. Syst."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1063\/1.1521390","article-title":"Border collision bifurcations at the change of state-space dimension","volume":"12","author":"Parui","year":"2002","journal-title":"Chaos"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1088\/0951-7715\/19\/8\/007","article-title":"Multi-parametric bifurcations in a piecewise-linear discontinuous map","volume":"19","author":"Avrutin","year":"2006","journal-title":"Nonlinearity"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1023\/A:1022221005097","article-title":"The cavity method at zero temperature","volume":"111","author":"Parisi","year":"2003","journal-title":"J. Stat. Phys."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Kreinovich, V. (2017). Modeling extremal events is not easy: Why the extreme value theorem cannot be as general as the central limit theorem. Uncertainty Modeling, Springer International Publishing. Chapter 8.","DOI":"10.1007\/978-3-319-51052-1_8"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"4701","DOI":"10.3390\/e17074701","article-title":"Asymptotic description of neural networks with correlated synaptic weights","volume":"17","author":"Faugeras","year":"2015","journal-title":"Entropy"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.neucom.2011.08.026","article-title":"Analysis and design of associative memories based on recurrent neural network with discontinuous activation functions","volume":"77","author":"Bao","year":"2012","journal-title":"Neurocomputing"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1088\/0954-898X_2_4_004","article-title":"What determines the capacity of autoassociative memories in the brain?","volume":"2","author":"Treves","year":"1991","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1152\/jn.01095.2002","article-title":"What determines the frequency of fast network oscillations with irregular neural discharges? I. Synaptic dynamics and excitation-inhibition balance","volume":"90","author":"Brunel","year":"2003","journal-title":"J. Neurophysiol."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/7\/630\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:01:29Z","timestamp":1760187689000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/7\/630"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,26]]},"references-count":69,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,7]]}},"alternative-id":["e21070630"],"URL":"https:\/\/doi.org\/10.3390\/e21070630","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2019,6,26]]}}}