{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,22]],"date-time":"2026-08-22T13:42:50Z","timestamp":1787406170644,"version":"build-2736575974"},"reference-count":67,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"2","funder":[{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF-16-1-0265"],"award-info":[{"award-number":["W911NF-16-1-0265"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1319140"],"award-info":[{"award-number":["CCF-1319140"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1516988"],"award-info":[{"award-number":["1516988"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2017,1]]},"abstract":"<jats:p>We propose an image representation scheme combining the local and nonlocal characterization of patches in an image. Our representation scheme can be shown to be equivalent to a tight frame constructed from convolving local bases (e.g., wavelet frames, discrete cosine transforms, etc.) with nonlocal bases (e.g., spectral basis induced by nonlinear dimension reduction on patches), and we call the resulting frame elements convolution framelets. Insight gained from analyzing the proposed representation leads to a novel interpretation of a recent high-performance patch-based image processing algorithm using the point integral method (PIM) and the low dimensional manifold model (LDMM) [S. Osher, Z. Shi, and W. Zhu, Low Dimensional Manifold Model for Image Processing, Tech. Rep., CAM report 16-04, UCLA, Los Angeles, CA, 2016]. In particular, we show that LDMM is a weighted $\\ell_2$-regularization on the coefficients obtained by decomposing images into linear combinations of convolution framelets; based on this understanding, we extend the original LDMM to a reweighted version that yields further improved results. In addition, we establish the energy concentration property of convolution framelet coefficients for the setting where the local basis is constructed from a given nonlocal basis via a linear reconstruction framework; a generalization of this framework to unions of local embeddings can provide a natural setting for interpreting BM3D, one of the state-of-the-art image denoising algorithms.<\/jats:p>","DOI":"10.1137\/16m1091447","type":"journal-article","created":{"date-parts":[[2017,5,16]],"date-time":"2017-05-16T12:06:17Z","timestamp":1494936377000},"page":"711-750","source":"Crossref","is-referenced-by-count":42,"title":["A Tale of Two Bases: Local-Nonlocal Regularization on Image Patches with Convolution Framelets"],"prefix":"10.1137","volume":"10","author":[{"given":"Rujie","family":"Yin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tingran","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yue M.","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ingrid","family":"Daubechies","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2017,5,16]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1137\/140977680"},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1016\/j.laa.2005.06.035"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1109\/T-C.1974.223784"},{"key":"atypb4","first-page":"941","volume":"4","author":"Barnsley M. F.","year":"1990","journal-title":"US Patent"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1162\/089976603321780317"},{"key":"atypb6","first-page":"129","volume":"19","author":"Belkin M.","year":"2007","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb7","first-page":"60","author":"Buades A.","year":"2005","journal-title":"Washington, DC"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1137\/040616024"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-007-0056-x"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2009.2018575"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2011.2172799"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1137\/S0036139994279153"},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.1016\/j.acha.2006.04.006"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0500334102"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0500896102"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2007.901238"},{"key":"atypb17","unstructured":"K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian,\n                      BM3D image denoising with shape-adaptive principal component analysis\n                      , in SPARS'09, Signal Processing with Adaptive Sparse Structured Representations, Saint Malo, France, 2009."},{"key":"atypb18","doi-asserted-by":"crossref","unstructured":"I. Daubechies,\n                      Ten Lectures on Wavelets\n                      , CBMS-NSF Regional Conf. Ser. in Appl. Math. 61, SIAM, Philadelphia, 1992.","DOI":"10.1137\/1.9781611970104"},{"key":"atypb19","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1995.tb02032.x"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/81.3.425"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2006.881969"},{"key":"atypb22","first-page":"2443","author":"Engan K.","year":"1999","journal-title":"Washington, DC"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1137\/060669358"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1137\/070698592"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1137\/090771806"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1126\/science.1127647"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2015.1094393"},{"key":"atypb28","doi-asserted-by":"publisher","DOI":"10.1109\/83.128028"},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2015.2446943"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2362059"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1162\/089976603762552951"},{"key":"atypb32","first-page":"459","author":"Kreutz-Delgado K.","year":"2000","journal-title":"WA"},{"key":"atypb33","unstructured":"S. S. Lafon,\n                      Diffusion Maps and Geometric Harmonics\n                      , Ph.D. thesis, Yale University, New Haven, CT, 2004."},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1214\/16-EJS1112"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023705401078"},{"key":"atypb36","doi-asserted-by":"crossref","unstructured":"J. A. Lee and M. Verleysen,\n                      Nonlinear Dimensionality Reduction\n                      , Springer, New York, 2007.","DOI":"10.1007\/978-0-387-39351-3"},{"key":"atypb37","first-page":"293","author":"Lesage S.","year":"2005","journal-title":"Washington, DC"},{"key":"atypb38","unstructured":"Z. Li, Z. Shi, and J. Sun,\n                      Point Integral Method for Solving Poisson-Type Equations on Manifolds from Point Clouds with Convergence Guarantees\n                      , preprint,https:\/\/arxiv.org\/abs\/1409.2623, 2014."},{"key":"atypb39","doi-asserted-by":"crossref","unstructured":"A. V. Little, M. Maggioni, and L. Rosasco,\n                      Multiscale geometric methods for data sets I: Multiscale SVD, noise and curvature\n                      , Appl. Comput. Harmon. Anal., to appear,https:\/\/doi.org\/10.1016\/j.acha.2015.09.009.","DOI":"10.1016\/j.acha.2015.09.009"},{"key":"atypb40","first-page":"2579","volume":"9","author":"Maaten L. v. d.","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"atypb41","first-page":"2272","author":"Mairal J.","year":"2009","journal-title":"Washington, DC"},{"key":"atypb42","unstructured":"S. Mallat,\n                      A Wavelet Tour of Signal Processing: The Sparse Way\n                      , Academic Press, New York, 2008."},{"key":"atypb43","unstructured":"A. Newson, A. Almansa, Y. Gousseau, and P. Pe\u0301rez,\n                      Non-local patch-based image inpainting\n                      , Image Processing On Line (IPOL), preprint, 2015."},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1016\/S0042-6989(97)00169-7"},{"key":"atypb45","unstructured":"S. Osher, Z. Shi, and W. Zhu,\n                      Low Dimensional Manifold Model for Image Processing\n                      , Tech. Rep., CAM report 16-04, UCLA, Los Angeles, CA, 2016."},{"key":"atypb46","doi-asserted-by":"publisher","DOI":"10.1080\/14786440109462720"},{"key":"atypb47","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0676-2"},{"key":"atypb48","doi-asserted-by":"publisher","DOI":"10.1137\/07068881X"},{"key":"atypb49","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2011.2120592"},{"key":"atypb50","unstructured":"X. Qi,\n                      Vector Nonlocal Mean Filter\n                      , Ph.D. thesis, University of Toronto, Toronto, Canada, 2015."},{"key":"atypb51","unstructured":"L. Rosasco,\n                      Data Representation: From Signal Processing to Machine Learning\n                      , Mini-Tutorial, SIAM Conference on Image Science, Albuquerque, NM, 2016,http:\/\/meetings.siam.org\/sess\/dsp_programsess.cfm?SESSIONCODE=22699&_ga=1.165936247.87958178.1459945433."},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1126\/science.290.5500.2323"},{"key":"atypb53","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(92)90242-F"},{"key":"atypb54","doi-asserted-by":"crossref","unstructured":"B. Scho\u0308lkopf and A. J. Smola,\n                      Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond\n                      , MIT Press, Cambridge, MA, 2002.","DOI":"10.7551\/mitpress\/4175.001.0001"},{"key":"atypb55","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289630"},{"key":"atypb56","unstructured":"Z. Shi, S. Osher, and W. Zhu,\n                      Low Dimensional Manifold Model with Semi-local Patches\n                      , Technical Report, CAM report 16-63, UCLA, Los Angeles, CA, 2016."},{"key":"atypb57","unstructured":"Z. Shi and J. Sun,\n                      Convergence of the Point Integral Method for the Poisson Equation with Dirichlet Boundary on Point Cloud\n                      , preprint,https:\/\/arxiv.org\/abs\/1312.4424, 2013."},{"key":"atypb58","unstructured":"Z. Shi and J. Sun,\n                      Convergence of the Point Integral Method for Poisson Equation on Point Cloud\n                      , preprint,https:\/\/arxiv.org\/abs\/1403.2141, 2014."},{"key":"atypb59","doi-asserted-by":"publisher","DOI":"10.1137\/070712146"},{"key":"atypb60","unstructured":"A. Singer and H.t. Wu,\n                      Spectral Convergence of the Connection Laplacian from Random Samples\n                      , preprint,https:\/\/arxiv.org\/abs\/1306.1587, 2013."},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1109\/79.952804"},{"key":"atypb62","unstructured":"S. Smale, T. Poggio, A. Caponnetto, and J. Bouvrie,\n                      Derived distance: Towards a mathematical theory of visual cortex\n                      , Artificial Intelligence (2007)."},{"key":"atypb63","first-page":"1711","volume":"9","author":"Szlam A. D.","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"atypb64","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2293425"},{"key":"atypb65","doi-asserted-by":"publisher","DOI":"10.1126\/science.290.5500.2319"},{"key":"atypb66","doi-asserted-by":"publisher","DOI":"10.1007\/BF02288916"},{"key":"atypb67","doi-asserted-by":"publisher","DOI":"10.1109\/30.125072"}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/16M1091447","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:03:53Z","timestamp":1787317433000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/16M1091447"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,1]]},"references-count":67,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2017,1]]}},"alternative-id":["10.1137\/16M1091447"],"URL":"https:\/\/doi.org\/10.1137\/16m1091447","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,1]]}}}