{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:33:51Z","timestamp":1787322831852,"version":"build-2736575974"},"reference-count":57,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>We present a gradient-based heuristic method for supervised learning of sparsity-promoting regularizers for denoising signals and images. Sparsity-promoting regularization is a key ingredient in solving modern signal reconstruction problems; however, the operators underlying these regularizers are usually either designed by hand or learned from data in an unsupervised way. The recent success of supervised learning (e.g., with convolutional neural networks) in solving image reconstruction problems suggests that it could be a fruitful approach to designing regularizers. Towards this end, we propose to denoise signals using a variational formulation with a parametric, sparsity-promoting regularizer, where the parameters of the regularizer are learned to minimize the mean squared error of reconstructions on a training set of ground truth image and measurement pairs. Training involves solving a challenging bilevel optimization problem; we derive an expression for the gradient of the training loss using the closed-form solution of the denoising problem and provide an accompanying gradient descent algorithm to minimize it. Our experiments with structured 1D signals and natural images indicate that the proposed method can learn an operator that outperforms well-known regularizers (total variation, DCT-sparsity, and unsupervised dictionary learning) and collaborative filtering for denoising.<\/jats:p>","DOI":"10.1137\/22m1506547","type":"journal-article","created":{"date-parts":[[2024,1,10]],"date-time":"2024-01-10T03:59:44Z","timestamp":1704859184000},"page":"31-60","source":"Crossref","is-referenced-by-count":5,"title":["Learning Sparsity-Promoting Regularizers Using Bilevel Optimization"],"prefix":"10.1137","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8336-3426","authenticated-orcid":true,"given":"Avrajit","family":"Ghosh","sequence":"first","affiliation":[{"name":"Computational Mathematics, Science and Engineering, Michigan State University, East Lansing, MI 48824 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"McCann","sequence":"additional","affiliation":[{"name":"Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Madeline","family":"Mitchell","sequence":"additional","affiliation":[{"name":"Computational Mathematics, Science and Engineering, Michigan State University, East Lansing, MI 48824 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saiprasad","family":"Ravishankar","sequence":"additional","affiliation":[{"name":"Computational Mathematics, Science and Engineering, Michigan State University, East Lansing, MI 48824 USA."},{"name":"Biomedical Engineering, Michigan State University, East Lansing, MI 48824 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,1,10]]},"reference":[{"key":"ref1","first-page":"107","volume":"1","author":"Agrawal A.","year":"2019","journal-title":"J. Appl. Numer. Optim."},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2006.881199"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.3847\/2041-8213\/ab0e85"},{"key":"ref4","first-page":"1","volume":"1","author":"Alessio A.","year":"2006","journal-title":"Nuclear Med."},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1515\/9781400833344"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1007\/s11590-018-1319-x"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-2580-5_1"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511804441"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1137\/17M1122451"},{"key":"ref10","volume-title":"Introduction to Remote Sensing","author":"Campbell J. B.","year":"2011"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2016.2629286"},{"key":"ref12","unstructured":"Y. Chen , \nT. Pock , and \nH. Bischof  , Learning \\(\\ell_1\\)-Based Analysis and Synthesis Sparsity Priors Using Bi-Level Optimization, https:\/\/arxiv.org\/abs\/1401.4105, 2014."},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2299065"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"C. Crockett  and \nJ. A. Fessler  , Bilevel Methods for Image Reconstruction, preprint, arXiv:2109.09610, 2021.","DOI":"10.1561\/9781638280033"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1117\/12.643267"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-016-0662-8"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2006.871582"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1109\/42.993128"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2019.2943645"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2017.1333613"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"A. Ghosh , \nM. T. Mccann , and \nS. Ravishankar  , Bilevel learning of \\(\\ell_1\\) regularizers with closed-form gradients (BLORC), in ICASSP 2022, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 1491\u20131495.","DOI":"10.1109\/ICASSP43922.2022.9747201"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2021.3118944"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1137\/0710036"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1016\/0024-3795(75)90118-4"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2713099"},{"key":"ref26","first-page":"167","volume":"34","author":"Kim D.","year":"2014","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"V. Kouni , \nG. Paraskevopoulos , \nH. Rauhut , and \nG. C. Alexandropoulos  , ADMM-DAD net: A deep unfolding network for analysis compressed sensing, in ICASSP 2022, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 1506\u20131510.","DOI":"10.1109\/ICASSP43922.2022.9747096"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1137\/120882706"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1088\/0031-9155\/52\/7\/R01"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-46805-6_19"},{"key":"ref31","first-page":"6719","volume":"33","author":"Lee W.","year":"2020","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref32","doi-asserted-by":"crossref","unstructured":"A. Levis , \nD. Lee , \nJ. A. Tropp , \nC. F. Gammie , and \nK. L. Bouman  , Inference of black hole fluid-dynamics from sparse interferometric measurements, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2021, pp. 2340\u20132349.","DOI":"10.1109\/ICCV48922.2021.00234"},{"key":"ref33","volume":"34","author":"Liu J.","year":"2021","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1155\/2011\/693795"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.156"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1305"},{"key":"ref37","unstructured":"T. P. Minka  , Old and New Matrix Algebra Useful for Statistics, Technical report, MIT Media Lab, 2000."},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2020.3016905"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-18461-6_52"},{"key":"ref40","first-page":"4","volume-title":"Sampling Theory and Applications","author":"Peyr\u00e9 G.","year":"2011"},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2010.2090538"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2012.2226449"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2016.2567299"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2019.2936204"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1016\/j.cpet.2007.08.001"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1137\/16M1102884"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2012.2226445"},{"key":"ref48","volume-title":"Real and Complex Analysis","author":"Rudin W.","year":"1974","edition":"2"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289451"},{"key":"ref50","volume-title":"Remote Sensing: Models and Methods for Image Processing","author":"Schowengerdt R. A.","year":"2006"},{"key":"ref51","first-page":"908","volume-title":"Advances in Neural Information Processing Systems 26","author":"Sprechmann P.","year":"2013"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1214\/11-AOS878"},{"key":"ref53","doi-asserted-by":"crossref","unstructured":"S. V. Venkatakrishnan , \nC. A. Bouman , and \nB. Wohlberg  , Plug-and-play priors for model based reconstruction, in 2013 IEEE Global Conference on Signal and Information Processing, IEEE, 2013, pp. 945\u2013948.","DOI":"10.1109\/GlobalSIP.2013.6737048"},{"key":"ref54","volume-title":"Magnetic Resonance Imaging: Theory and Practice","author":"Vlaardingerbroek M. T.","year":"2013"},{"key":"ref55","unstructured":"Y. Yang , \nJ. Sun , \nH. Li , and \nZ. Xu  , Deep ADMM-net for compressive sensing MRI, in Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS\u201916, Curran Associates, Red Hook, NY, 2016, pp.\u00a010\u201318."},{"key":"ref56","volume-title":"NIPS Workshop","author":"Yunjin C.","year":"2012"},{"key":"ref57","doi-asserted-by":"crossref","unstructured":"J. Zhang  and \nB. Ghanem  , ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1828\u20131837.","DOI":"10.1109\/CVPR.2018.00196"}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/22M1506547","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:27:54Z","timestamp":1787318874000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/22M1506547"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,10]]},"references-count":57,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1137\/22M1506547"],"URL":"https:\/\/doi.org\/10.1137\/22m1506547","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,10]]}}}