{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T09:24:54Z","timestamp":1780046694976,"version":"3.53.1"},"reference-count":37,"publisher":"MIT Press - Journals","issue":"11","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2020,11]]},"abstract":"<jats:p> Hierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension. <\/jats:p>","DOI":"10.1162\/neco_a_01325","type":"journal-article","created":{"date-parts":[[2020,9,18]],"date-time":"2020-09-18T21:11:10Z","timestamp":1600463470000},"page":"2279-2309","source":"Crossref","is-referenced-by-count":17,"title":["Effect of Top-Down Connections in Hierarchical Sparse Coding"],"prefix":"10.1162","volume":"32","author":[{"given":"Victor","family":"Boutin","sequence":"first","affiliation":[{"name":"CNRS, INT, Institut de Neurosciences de la Timone, Aix-Marseille Universit\u00e9, Marseille, France and CNRS, ISM, Aix Marseille Universit\u00e9, Marseille, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angelo","family":"Franciosini","sequence":"additional","affiliation":[{"name":"CNRS, Institut de Neurosciences de la Timone, Aix-Marseille Universit\u00e9, 13005 Marseille, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Franck","family":"Ruffier","sequence":"additional","affiliation":[{"name":"CNRS, Institut des Sciences du Mouvement, Aix-Marseille Universit\u00e9, 13009 Marseille, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurent","family":"Perrinet","sequence":"additional","affiliation":[{"name":"CNRS, Institut de Neurosciences de la Timone, Aix-Marseille Universit\u00e9, 13005 Marseille, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1137\/18M1183352"},{"key":"B2","volume-title":"Face database.","author":"AT&amp;T","year":"1994"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1137\/080716542"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2013.6706854"},{"key":"B5","first-page":"215","volume-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics","author":"Coates A.","year":"2011"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-7011-4"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2787"},{"key":"B8","first-page":"399","volume-title":"Proceedings of the 27th International Conference on Machine Learning","author":"Gregor K.","year":"2010"},{"key":"B9","first-page":"9221","volume-title":"Advances in neural information processing systems","volume":"31","author":"Han K.","year":"2018"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299149"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.1962.sp006837"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459469"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1162\/089976603762552951"},{"key":"B14","author":"LeCun Y.","year":"1998","journal-title":"The MNIST database of handwritten digits."},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553453"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.3934\/ipi.2009.3.487"},{"key":"B17","author":"Lotter W.","year":"2016","journal-title":"Deep predictive coding networks for video prediction and unsupervised learning."},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-014-0532-5"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553463"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459452"},{"key":"B21","author":"Makhzani A.","year":"2013","journal-title":"K-sparse autoencoders."},{"key":"B22","first-page":"2791","volume-title":"Advances in neural information processing systems, 28","author":"Makhzani A.","year":"2015"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1109\/78.258082"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(97)00169-7"},{"issue":"1","key":"B25","first-page":"2887","volume":"18","author":"Papyan V.","year":"2017","journal-title":"Journal of Machine Learning Research"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1038\/srep11400"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1016\/S0895-7177(01)00109-1"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1038\/4580"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2010.2040551"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-016-9445-1"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2904255"},{"issue":"15","key":"B32","first-page":"4090","volume":"66","author":"Sulam J.","year":"2018","journal-title":"IEEE Transactions on Signal Processing"},{"key":"B33","author":"Szlam A.","year":"2010","journal-title":"Convolutional matching pursuit and dictionary training."},{"key":"B34","author":"Wen H.","year":"2018","journal-title":"Deep predictive coding network for object recognition"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995393"},{"key":"B36","author":"Zeiler M. D.","year":"2012","journal-title":"Differentiable pooling for hierarchical feature learning."},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126474"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01325","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:43:51Z","timestamp":1615585431000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/32\/11\/2279-2309\/95639"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11]]},"references-count":37,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2020,11]]}},"alternative-id":["10.1162\/neco_a_01325"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01325","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11]]}}}