{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T05:53:30Z","timestamp":1777614810427,"version":"3.51.4"},"reference-count":52,"publisher":"MIT Press","issue":"6","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,5,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We propose and analyze a continuous-time firing-rate neural network, the positive firing-rate competitive network (PFCN), to tackle sparse reconstruction problems with non-negativity constraints. These problems, which involve approximating a given input stimulus from a dictionary using a set of sparse (active) neurons, play a key role in a wide range of domains, including, for example, neuroscience, signal processing, and machine learning. First, by leveraging the theory of proximal operators, we relate the equilibria of a family of continuous-time firing-rate neural networks to the optimal solutions of sparse reconstruction problems. Then we prove that the PFCN is a positive system and give rigorous conditions for the convergence to the equilibrium. Specifically, we show that the convergence depends only on a property of the dictionary and is linear-exponential in the sense that initially, the convergence rate is at worst linear and then, after a transient, becomes exponential. We also prove a number of technical results to assess the contractivity properties of the neural dynamics of interest. Our analysis leverages contraction theory to characterize the behavior of a family of firing-rate competitive networks for sparse reconstruction with and without non-negativity constraints. Finally, we validate the effectiveness of our approach via a numerical example.<\/jats:p>","DOI":"10.1162\/neco_a_01657","type":"journal-article","created":{"date-parts":[[2024,4,25]],"date-time":"2024-04-25T00:08:48Z","timestamp":1714003728000},"page":"1163-1197","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["Positive Competitive Networks for Sparse Reconstruction"],"prefix":"10.1162","volume":"36","author":[{"given":"Veronica","family":"Centorrino","sequence":"first","affiliation":[{"name":"Scuola Superiore Meridionale, Naples 80138, Italy veronica.centorrino@unina.it"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anand","family":"Gokhale","sequence":"additional","affiliation":[{"name":"Center for Control, Dynamical Systems, and Computation, University of California, Santa Barbara, Santa Barbara, CA 93106 U.S.A. anand_gokhale@ucsb.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander","family":"Davydov","sequence":"additional","affiliation":[{"name":"Center for Control, Dynamical Systems, and Computation, University of California, Santa Barbara, Santa Barbara, CA 93106 U.S.A. davydov@ucsb.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Giovanni","family":"Russo","sequence":"additional","affiliation":[{"name":"Department of Information and Electric Engineering and Applied Mathematics, University of Salerno, Fisciano 84084, Italy giovarusso@unisa.it"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesco","family":"Bullo","sequence":"additional","affiliation":[{"name":"Center for Control, Dynamical Systems, and Computation, University of California, Santa Barbara, Santa Barbara, CA 93106 U.S.A. bullo@ucsb.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,5,10]]},"reference":[{"key":"2024082215382307900_bib1","volume-title":"Studies in linear and non-linear programming","author":"Arrow","year":"1958"},{"issue":"9","key":"2024082215382307900_bib2","doi-asserted-by":"publisher","first-page":"1377","DOI":"10.1109\/TNNLS.2012.2202400","article-title":"Convergence and rate analysis of neural networks for sparse approximation","volume":"23","author":"Balavoine","year":"2012","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2024082215382307900_bib3","first-page":"431","article-title":"Global convergence of the locally competitive algorithm","author":"Balavoine","year":"2011"},{"key":"2024082215382307900_bib4","first-page":"1","article-title":"Convergence of a neural network for sparse approximation using the nonsmooth \u0141ojasiewicz inequality","volume-title":"Proceedings of the International Joint Conference on Neural Networks","author":"Balavoine","year":"2013"},{"issue":"17","key":"2024082215382307900_bib5","doi-asserted-by":"publisher","first-page":"4259","DOI":"10.1109\/TSP.2013.2271482","article-title":"Convergence speed of a dynamical system for sparse recovery","volume":"61","author":"Balavoine","year":"2013","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"12","key":"2024082215382307900_bib6","doi-asserted-by":"publisher","first-page":"3165","DOI":"10.1109\/TSP.2015.2420535","article-title":"Discrete and continuous-time soft-thresholding for dynamic signal recovery","volume":"63","author":"Balavoine","year":"2015","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"4","key":"2024082215382307900_bib7","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1068\/p010371","article-title":"Single units and sensation: A neuron doctrine for perceptual psychology?","volume":"1","author":"Barlow","year":"1972","journal-title":"Perception"},{"key":"2024082215382307900_bib8","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611974997","volume-title":"First-order methods in optimization","author":"Beck","year":"2017"},{"issue":"2","key":"2024082215382307900_bib9","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1109\/72.207617","article-title":"Neural network for quadratic optimization with bound constraints","volume":"4","author":"Bouzerdoum","year":"1993","journal-title":"IEEE Transactions on Neural Networks"},{"issue":"9","key":"2024082215382307900_bib10","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1005070","article-title":"Nonlinear Hebbian learning as a unifying principle in receptive field formation","volume":"12","author":"Brito","year":"2016","journal-title":"PLOS Computational Biology"},{"key":"2024082215382307900_bib11","volume-title":"Contraction theory for dynamical systems","author":"Bullo","year":"2023"},{"issue":"1","key":"2024082215382307900_bib12","first-page":"83","article-title":"The Dantzig selector: Statistical estimation when p is much larger than n","volume":"54","author":"Cand\u00e8s","year":"2009","journal-title":"Quality Control and Applied Statistics"},{"issue":"2","key":"2024082215382307900_bib13","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/MSP.2007.914731","article-title":"An introduction to compressive sampling","volume":"25","author":"Cand\u00e8s","year":"2008","journal-title":"IEEE Signal Processing Magazine"},{"key":"2024082215382307900_bib14","doi-asserted-by":"crossref","DOI":"10.1109\/CDC51059.2022.9993009","article-title":"Contraction analysis of Hopfield neural networks with Hebbian learning","volume-title":"Proceedings of the IEEE Conference on Decision and Control","author":"Centorrino","year":"2022"},{"key":"2024082215382307900_bib15","doi-asserted-by":"publisher","first-page":"1724","DOI":"10.1109\/LCSYS.2023.3278250","article-title":"Euclidean contractivity of neural networks with symmetric weights","volume":"7","author":"Centorrino","year":"2023","journal-title":"IEEE Control Systems Letters"},{"key":"2024082215382307900_bib16","doi-asserted-by":"crossref","DOI":"10.1016\/j.ifacol.2023.10.1052","article-title":"Fast sparse optimization via adaptive shrinkage","volume-title":"Proceedings of the IFAC World Congress","author":"Cerone","year":"2023"},{"issue":"12","key":"2024082215382307900_bib17","doi-asserted-by":"publisher","first-page":"3317","DOI":"10.1162\/NECO_a_00372","article-title":"A common network architecture efficiently implements a variety of sparsity-based inference problems","volume":"24","author":"Charles","year":"2012","journal-title":"Neural Computation"},{"key":"2024082215382307900_bib18","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1007\/978-1-4419-9569-8_10","article-title":"Proximal splitting methods in signal processing","volume-title":"Fixed-point algorithms for inverse problems in science and engineering","author":"Combettes","year":"2011"},{"issue":"1","key":"2024082215382307900_bib19","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1007\/BF02124750","article-title":"On the Lambert W function","volume":"5","author":"Corless","year":"1996","journal-title":"Advances in Computational Mathematics"},{"key":"2024082215382307900_bib20","article-title":"Contracting dynamics for time-varying convex optimization","author":"Davydov","year":"2023","journal-title":"IEEE Transactions on Automatic Control"},{"key":"2024082215382307900_bib21","article-title":"Non-Euclidean contraction analysis of continuous-time neural networks","author":"Davydov","year":"2022"},{"key":"2024082215382307900_bib22","first-page":"1527","article-title":"Non-Euclidean contractivity of recurrent neural networks","volume-title":"Proceedings of the American Control Conference","author":"Davydov","year":"2022"},{"key":"2024082215382307900_bib23","volume-title":"Theoretical neuroscience: Computational and mathematical modeling of neural systems","author":"Dayan","year":"2005"},{"issue":"3","key":"2024082215382307900_bib24","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1088\/0954-898X_3_3_002","article-title":"Dynamic properties of neural networks with adapting synapses","volume":"3","author":"Dong","year":"1992","journal-title":"Network: Computation in Neural Systems"},{"issue":"6","key":"2024082215382307900_bib25","doi-asserted-by":"publisher","first-page":"972","DOI":"10.1109\/JPROC.2009.2037655","article-title":"On the role of sparse and redundant representations in image processing","volume":"98","author":"Elad","year":"2010","journal-title":"Proceedings of the IEEE"},{"issue":"12","key":"2024082215382307900_bib26","doi-asserted-by":"publisher","first-page":"2379","DOI":"10.1364\/JOSAA.4.002379","article-title":"Relations between the statistics of natural images and the response properties of cortical cells","volume":"4","author":"Field","year":"1987","journal-title":"Journal of the Optical Society of America A"},{"key":"2024082215382307900_bib27","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1007\/s00422-002-0353-y","article-title":"Mathematical formulations of Hebbian learning","volume":"87","author":"Gerstner","year":"2003","journal-title":"Biological Cybernetics"},{"key":"2024082215382307900_bib28","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2020.109311","article-title":"Proximal gradient flow and Douglas-Rachford splitting dynamics: Global exponential stability via integral quadratic constraints","volume":"123","author":"Hassan-Moghaddam","year":"2021","journal-title":"Automatica"},{"key":"2024082215382307900_bib29","volume-title":"The organization of behavior: A neuropsychological theory","author":"Hebb","year":"1949"},{"issue":"10","key":"2024082215382307900_bib30","doi-asserted-by":"publisher","first-page":"3088","DOI":"10.1073\/pnas.81.10.3088","article-title":"Neurons with graded response have collective computational properties like those of two-state neurons","volume":"81","author":"Hopfield","year":"1984","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"3","key":"2024082215382307900_bib31","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1007\/BF00339943","article-title":"\u201cNeural\u201d computation of decisions in optimization problems","volume":"52","author":"Hopfield","year":"1985","journal-title":"Biological Cybernetics"},{"key":"2024082215382307900_bib32","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1109\/NNSP.2002.1030067","article-title":"Non-negative sparse coding","volume-title":"Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing","author":"Hoyer","year":"2002"},{"key":"2024082215382307900_bib33","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1016\/S0925-2312(02)00782-8","article-title":"Modeling receptive fields with non-negative sparse coding","volume":"52","author":"Hoyer","year":"2003","journal-title":"Neurocomputing"},{"issue":"1","key":"2024082215382307900_bib34","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1113\/jphysiol.1968.sp008455","article-title":"Receptive fields and functional architecture of monkey striate cortex","volume":"195","author":"Hubel","year":"1968","journal-title":"Journal of Physiology"},{"issue":"3","key":"2024082215382307900_bib35","doi-asserted-by":"publisher","first-page":"1285","DOI":"10.1109\/TAC.2021.3073096","article-title":"Weak and semi-contraction for network systems and diffusively-coupled oscillators","volume":"67","author":"Jafarpour","year":"2022","journal-title":"IEEE Transactions on Automatic Control"},{"key":"2024082215382307900_bib36","article-title":"RNNs of RNNs: Recursive construction of stable assemblies of recurrent neural networks","volume-title":"Advances in neural information processing systems","author":"Kozachkov","year":"2022"},{"issue":"8","key":"2024082215382307900_bib37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1007659","article-title":"Achieving stable dynamics in neural circuits","volume":"16","author":"Kozachkov","year":"2020","journal-title":"PLOS Computational Biology"},{"key":"2024082215382307900_bib38","doi-asserted-by":"crossref","DOI":"10.1007\/s00422-022-00943-8","article-title":"Biologically plausible single-layer networks for nonnegative independent component analysis","volume-title":"Biological cybernetics","author":"Lipshutz","year":"2022"},{"issue":"6","key":"2024082215382307900_bib39","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/S0005-1098(98)00019-3","article-title":"On contraction analysis for non-linear systems","volume":"34","author":"Lohmiller","year":"1998","journal-title":"Automatica"},{"issue":"1","key":"2024082215382307900_bib40","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1162\/NECO_a_00221","article-title":"Mathematical equivalence of two common forms of firing rate models of neural networks","volume":"24","author":"Miller","year":"2012","journal-title":"Neural Computation"},{"key":"2024082215382307900_bib41","first-page":"551","article-title":"\u00dcber die Lage der Integralkurven gew\u00f6hnlicher Differentialgleichungen","volume-title":"Proceedings of the Physico-Mathematical Society of Japan. 3rd Series","author":"Nagumo","year":"1942"},{"issue":"6583","key":"2024082215382307900_bib42","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"issue":"23","key":"2024082215382307900_bib43","doi-asserted-by":"publisher","first-page":"3311","DOI":"10.1016\/S0042-6989(97)00169-7","article-title":"Sparse coding with an overcomplete basis set: A strategy employed by V1?","volume":"37","author":"Olshausen","year":"1997","journal-title":"Vision Research"},{"issue":"4","key":"2024082215382307900_bib44","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1016\/j.conb.2004.07.007","article-title":"Sparse coding of sensory inputs","volume":"14","author":"Olshausen","year":"2004","journal-title":"Current Opinion in Neurobiology"},{"issue":"3","key":"2024082215382307900_bib45","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1561\/2400000003","article-title":"Proximal algorithms","volume":"1","author":"Parikh","year":"2014","journal-title":"Foundations and Trends in Optimization"},{"issue":"10","key":"2024082215382307900_bib46","doi-asserted-by":"publisher","first-page":"2526","DOI":"10.1162\/neco.2008.03-07-486","article-title":"Sparse coding via thresholding and local competition in neural circuits","volume":"20","author":"Rozell","year":"2008","journal-title":"Neural Computation"},{"issue":"4","key":"2024082215382307900_bib47","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000739","article-title":"Global entrainment of transcriptional systems to periodic inputs","volume":"6","author":"Russo","year":"2010","journal-title":"PLOS Computational Biology"},{"key":"2024082215382307900_bib48","doi-asserted-by":"crossref","DOI":"10.1017\/9781108779302","volume-title":"High-dimensional data analysis with low-dimensional models: Principles, computation, and applications","author":"Wright","year":"2022"},{"issue":"2","key":"2024082215382307900_bib49","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1109\/TPAMI.2008.79","article-title":"Robust face recognition via sparse representation","volume":"31","author":"Wright","year":"2008","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"3","key":"2024082215382307900_bib50","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1109\/TCNS.2021.3058934","article-title":"Scalability in nonlinear network systems affected by delays and disturbances","volume":"8","author":"Xie","year":"2021","journal-title":"IEEE Transactions on Control of Network Systems"},{"issue":"7","key":"2024082215382307900_bib51","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1109\/TNNLS.2014.2317880","article-title":"A comprehensive review of stability analysis of continuous-time recurrent neural networks","volume":"25","author":"Zhang","year":"2014","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"issue":"8","key":"2024082215382307900_bib52","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1003191","article-title":"Visual nonclassical receptive field effects emerge from sparse coding in a dynamical system","volume":"9","author":"Zhu","year":"2013","journal-title":"PLOS Computational Biology"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/6\/1163\/2467142\/neco_a_01657.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/6\/1163\/2467142\/neco_a_01657.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T16:20:27Z","timestamp":1724343627000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/36\/6\/1163\/120319\/Positive-Competitive-Networks-for-Sparse"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,10]]},"references-count":52,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,5,10]]},"published-print":{"date-parts":[[2024,5,10]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01657","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,6]]},"published":{"date-parts":[[2024,5,10]]}}}