{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:33:59Z","timestamp":1787322839993,"version":"build-2736575974"},"reference-count":50,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"1","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12371527"],"award-info":[{"award-number":["12371527"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42293272"],"award-info":[{"award-number":["42293272"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12101058"],"award-info":[{"award-number":["12101058"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004826","name":"Natural Science Foundation of Beijing Municipality","doi-asserted-by":"publisher","award":["1232011"],"award-info":[{"award-number":["1232011"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>High-dimensional deep features extracted by convolutional neural networks have nonlocal self-similarity. However, incorporating this nonlocal prior of deep features into deep network architectures with an interpretable variational framework is rarely explored. In this paper, we propose a learnable nonlocal self-similarity deep feature network for image denoising. Our method is motivated by the fact that the high-dimensional deep features obey a mixture probability distribution based on the Parzen\u2013Rosenblatt window method. Then a regularizer with learnable nonlocal weights is proposed by considering the dual representation of the log-probability prior of the deep features. Specifically, the nonlocal weights are introduced as dual variables that can be learned by unrolling the associated numerical scheme. This leads to nonlocal modules (NLMs) in newly designed networks. Our method provides a statistical and variational interpretation for the nonlocal self-attention mechanism widely used in various networks. By adopting nonoverlapping window and region decomposition techniques, we can significantly reduce the computational complexity of nonlocal self-similarity, thus enabling parallel computation of the NLM. The solution to the proposed variational problem can be formulated as a learnable nonlocal self-similarity network for image denoising. This work offers a novel approach for constructing network structures that consider self-similarity and nonlocality. The improvements achieved by this method are predictable and partially controllable. Compared with several closely related denoising methods, the experimental results show the effectiveness of the proposed method in image denoising.<\/jats:p>","DOI":"10.1137\/22m1536996","type":"journal-article","created":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T05:37:17Z","timestamp":1708666637000},"page":"441-475","source":"Crossref","is-referenced-by-count":7,"title":["Learnable Nonlocal Self-Similarity of Deep Features for Image Denoising"],"prefix":"10.1137","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-3143-8857","authenticated-orcid":true,"given":"Junying","family":"Meng","sequence":"first","affiliation":[{"name":"Laboratory of Mathematics and Complex Systems (Ministry of Education of China), School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Faqiang","family":"Wang","sequence":"additional","affiliation":[{"name":"Laboratory of Mathematics and Complex Systems (Ministry of Education of China), School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8697-3089","authenticated-orcid":true,"given":"Jun","family":"Liu","sequence":"additional","affiliation":[{"name":"Laboratory of Mathematics and Complex Systems (Ministry of Education of China), School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, China."},{"name":"Corresponding author. Laboratory of Mathematics and Complex Systems (Ministry of Education of China), School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,2,23]]},"reference":[{"key":"ref1","doi-asserted-by":"crossref","unstructured":"A. Abdelhamed, S. Lin, and M. S. Brown, A high-quality denoising dataset for smartphone cameras, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 1692\u20131700.","DOI":"10.1109\/CVPR.2018.00182"},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"E. Agustsson and R. Timofte, NTIRE 2017 challenge on single image super-resolution: Dataset and study, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 126\u2013135.","DOI":"10.1109\/CVPRW.2017.150"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2018.01.012"},{"key":"ref4","doi-asserted-by":"crossref","unstructured":"S. Anwar and N. Barnes, Real image denoising with feature attention, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2019, pp. 3155\u20133164.","DOI":"10.1109\/ICCV.2019.00325"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.161"},{"key":"ref6","doi-asserted-by":"crossref","unstructured":"A. Buades, B. Coll, and J.M. Morel, A non-local algorithm for image denoising, in Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR\u201905), Vol. 2, IEEE, 2005, pp. 60\u201365.","DOI":"10.1109\/CVPR.2005.38"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"M. Chang, Q. Li, H. Feng, and Z. Xu, Spatial-adaptive network for single image denoising, in European Conference on Computer Vision, Springer, New York, 2020, pp. 171\u2013187.","DOI":"10.1007\/978-3-030-58577-8_11"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"P. Charbonnier, L. Blanc-Feraud, G. Aubert, and M. Barlaud, Two deterministic half-quadratic regularization algorithms for computed imaging, in Proceedings of the 1st International Conference on Image Processing, Vol. 2, IEEE, 1994, pp. 168\u2013172.","DOI":"10.1109\/ICIP.1994.413553"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2596743"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2018.2850222"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2007.901238"},{"key":"ref12","unstructured":"R. Franzen, Kodak Lossless True Color Image Suite, 1999, https:\/\/r0k.us\/graphics\/kodak\/."},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3087424"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1137\/070698592"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"S. Gu, L. Zhang, W. Zuo, and X. Feng, Weighted nuclear norm minimization with application to image denoising, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2862\u20132869.","DOI":"10.1109\/CVPR.2014.366"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"S. Guo, Z. Yan, K. Zhang, W. Zuo, and L. Zhang, Toward convolutional blind denoising of real photographs, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 1712\u20131722.","DOI":"10.1109\/CVPR.2019.00181"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128018"},{"key":"ref18","doi-asserted-by":"crossref","unstructured":"J.B. Huang, A. Singh, and N. Ahuja, Single image super-resolution from transformed self-exemplars, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 5197\u20135206.","DOI":"10.1109\/CVPR.2015.7299156"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3124086"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"W.S. Lai, J.B. Huang, N. Ahuja, and M.H. Yang, Deep Laplacian pyramid networks for fast and accurate super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 624\u2013632.","DOI":"10.1109\/CVPR.2017.618"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"S. Lefkimmiatis, Non-local color image denoising with convolutional neural networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3587\u20133596.","DOI":"10.1109\/CVPR.2017.623"},{"key":"ref22","doi-asserted-by":"crossref","unstructured":"S. Lefkimmiatis, Universal denoising networks: A novel CNN architecture for image denoising, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3204\u20133213.","DOI":"10.1109\/CVPR.2018.00338"},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte, SwinIR: Image restoration using swin transformer, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2021, pp. 1833\u20131844.","DOI":"10.1109\/ICCVW54120.2021.00210"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"K. Lin, T. H. Li, S. Liu, and G. Li, Real photographs denoising with noise domain adaptation and attentive generative adversarial network, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2019.","DOI":"10.1109\/CVPRW.2019.00221"},{"key":"ref25","volume":"31","author":"Liu D.","year":"2018","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1137\/16M1074163"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2021, pp. 10012\u201310022.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2631888"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"D. Martin, C. Fowlkes, D. Tal, and J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in Proceedings of the 8th International Conference on Computer Vision, Vol. 2, IEEE, 2001, pp. 416\u2013423.","DOI":"10.1109\/ICCV.2001.937655"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3063916"},{"key":"ref31","doi-asserted-by":"crossref","unstructured":"C. Mou, J. Zhang, and Z. Wu, Dynamic attentive graph learning for image restoration, in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2021, pp. 4328\u20134337.","DOI":"10.1109\/ICCV48922.2021.00429"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177704472"},{"key":"ref33","volume":"31","author":"Pl\u00f6tz T.","year":"2018","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref34","doi-asserted-by":"crossref","unstructured":"P. Qiao, Y. Dou, W. Feng, R. Li, and Y. Chen, Learning non-local image diffusion for image denoising, in Proceedings of the 25th ACM International Conference on Multimedia, 2017, pp. 1847\u20131855.","DOI":"10.1145\/3123266.3123370"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"C. Ren, X. He, C. Wang, and Z. Zhao, Adaptive consistency prior based deep network for image denoising, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8596\u20138606.","DOI":"10.1109\/CVPR46437.2021.00849"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177728190"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-008-0197-6"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(92)90242-F"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.12.024"},{"key":"ref40","doi-asserted-by":"crossref","unstructured":"R. Timofte, E. Agustsson, L. Van Gool, M.H. Yang, and L. Zhang, NTIRE 2017 challenge on single image super-resolution: Methods and results, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 114\u2013125.","DOI":"10.1109\/CVPRW.2017.150"},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3013166"},{"key":"ref42","doi-asserted-by":"crossref","unstructured":"X. Wang, R. Girshick, A. Gupta, and K. He, Non-local neural networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7794\u20137803.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1145\/3599730"},{"key":"ref44","first-page":"1","author":"Zhang K.","year":"2023","journal-title":"Mach. Intell. Res."},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2662206"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2839891"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1117\/1.3600632"},{"key":"ref48","unstructured":"Y. Zhang, K. Li, K. Li, B. Zhong, and Y. Fu, Residual Non-Local Attention Networks for Image Restoration, preprint, arXiv:1903.10082, 2019."},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2968521"},{"key":"ref50","doi-asserted-by":"crossref","unstructured":"H. Zheng, H. Yong, and L. Zhang, Deep convolutional dictionary learning for image denoising, in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 630\u2013641.","DOI":"10.1109\/CVPR46437.2021.00069"}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/22M1536996","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:28:43Z","timestamp":1787318923000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/22M1536996"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,23]]},"references-count":50,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1137\/22M1536996"],"URL":"https:\/\/doi.org\/10.1137\/22m1536996","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,23]]}}}