{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:07:33Z","timestamp":1760234853192,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,6,17]],"date-time":"2021-06-17T00:00:00Z","timestamp":1623888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>In this paper we propose a novel learning-based wavelet transform and demonstrate its utility as a representation in solving a number of linear inverse problems\u2014these are asymmetric problems, where the forward problem is easy to solve, but the inverse is difficult and often ill-posed. The wavelet decomposition is comprised of the application of an invertible 2D wavelet filter-bank comprising symmetric and anti-symmetric filters, in combination with a set of 1\u00d71 convolution filters learnt from Principal Component Analysis (PCA). The 1\u00d71 filters are needed to control the size of the decomposition. We show that the application of PCA across wavelet subbands in this way produces an architecture equivalent to a separable Convolutional Neural Network (CNN), with the principal components forming the 1\u00d71 filters and the subtraction of the mean forming the bias terms. The use of an invertible filter bank and (approximately) invertible PCA allows us to create a deep autoencoder very simply, and avoids issues of overfitting. We investigate the construction and learning of such networks, and their application to linear inverse problems via the Alternating Direction of Multipliers Method (ADMM). We use our network as a drop-in replacement for traditional discrete wavelet transform, using wavelet shrinkage as the projection operator. The results show good potential on a number of inverse problems such as compressive sensing, in-painting, denoising and super-resolution, and significantly close the performance gap with Generative Adversarial Network (GAN)-based methods.<\/jats:p>","DOI":"10.3390\/sym13061083","type":"journal-article","created":{"date-parts":[[2021,6,17]],"date-time":"2021-06-17T21:29:16Z","timestamp":1623965356000},"page":"1083","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Principal Component Wavelet Networks for Solving Linear Inverse Problems"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7570-1192","authenticated-orcid":false,"given":"Bernard","family":"Tiddeman","sequence":"first","affiliation":[{"name":"Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6423-6475","authenticated-orcid":false,"given":"Morteza","family":"Ghahremani","sequence":"additional","affiliation":[{"name":"A.I. Virtanen Institute for Molecular Sciences, University of Eastern Finland, 70150 Kuopio, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1109\/18.382009","article-title":"De-noising by soft-thresholding","volume":"41","author":"Donoho","year":"1995","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1338","DOI":"10.1109\/TIP.2003.818640","article-title":"Image denoising using scale mixtures of Gaussians in the wavelet domain","volume":"12","author":"Portilla","year":"2003","journal-title":"IEEE Trans. Image Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1137\/070697653","article-title":"Learning multiscale sparse representations for image and video restoration","volume":"7","author":"Mairal","year":"2008","journal-title":"Multiscale Model. Simul."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s10851-006-5257-3","article-title":"Total variation wavelet inpainting","volume":"25","author":"Chan","year":"2006","journal-title":"J. Math. Imaging Vis."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1838","DOI":"10.1109\/TIP.2011.2108306","article-title":"Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization","volume":"20","author":"Dong","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., and Efros, A.A. (2016, January 27\u201330). Context encoders: Feature learning by inpainting. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.278"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Mousavi, A., and Baraniuk, R.G. (2017, January 5\u20139). Learning to invert: Signal recovery via deep convolutional networks. Proceedings of the 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952561"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., and Ashok, A. (2016, January 27\u201330). Reconnet: Non-iterative reconstruction of images from compressively sensed measurements. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.55"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Husz\u00e1r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21\u201326). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Dong, C., Loy, C.C., He, K., and Tang, X. (2014, January 6\u201312). Learning a deep convolutional network for image super-resolution. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10593-2_13"},{"key":"ref_11","unstructured":"Xu, L., Ren, J.S., Liu, C., and Jia, J. (2014, January 8\u201313). Deep convolutional neural network for image deconvolution. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rick Chang, J., Li, C.L., Poczos, B., Vijaya Kumar, B., and Sankaranarayanan, A.C. (2017, January 22\u201329). One Network to Solve Them All\u2013Solving Linear Inverse Problems Using Deep Projection Models. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.627"},{"key":"ref_13","unstructured":"Lucic, M., Kurach, K., Michalski, M., Bousquet, O., and Gelly, S. (2018, January 3\u20138). Are GANs Created Equal? A Large-Scale Study. Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS\u201918), Montreal, QC, Canada."},{"key":"ref_14","unstructured":"Arjovsky, M., and Bottou, L. (2017, January 24\u201326). Towards Principled Methods for Training Generative Adversarial Networks. Proceedings of the 5th International Conference on Learning Representations (ICLR 2017), Toulon, France."},{"key":"ref_15","unstructured":"Milacski, Z.A., P\u00f3czos, B., and Lorincz, A. (2019, January 9\u201312). Differentiable Unrolled Alternating Direction Method of Multipliers for OneNet. Proceedings of the British Machine Vision Conference, Cardiff, UK."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Raj, A., Li, Y., and Bresler, Y. (November, January 27). GAN-Based Projector for Faster Recovery With Convergence Guarantees in Linear Inverse Problems. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00570"},{"key":"ref_17","unstructured":"Mallat, S. (2008). A Wavelet Tour of Signal Processing, Academic Press. [3rd ed.]."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1109\/34.93808","article-title":"The design and use of steerable filters","volume":"13","author":"Freeman","year":"1991","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1006\/acha.2000.0343","article-title":"Complex wavelets for shift invariant analysis and filtering of signals","volume":"10","author":"Kingsbury","year":"2001","journal-title":"J. Appl. Comput. Harmon. Anal."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1109\/MSP.2005.1550194","article-title":"The dual-tree complex wavelet transform","volume":"22","author":"Selesnick","year":"2005","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1109\/TSP.2004.826174","article-title":"The double-density dual-tree DWT","volume":"52","author":"Selesnick","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_22","unstructured":"Kingma, D.P., and Dhariwal, P. (2018, January 3\u20138). Glow: Generative flow with invertible 1x1 convolutions. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1109\/TPAMI.2012.230","article-title":"Invariant Scattering Convolution Networks","volume":"35","author":"Bruna","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2208","DOI":"10.1109\/TPAMI.2018.2855738","article-title":"Scattering networks for hybrid representation learning","volume":"41","author":"Oyallon","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","unstructured":"Angles, T., and Mallat, S. (2018). Generative networks as inverse problems with scattering transforms. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Gupta, M.R., and Jacobson, N.P. (2006, January 26\u201329). Wavelet Principal Component Analysis and its Application to Hyperspectral Images. Proceedings of the 2006 International Conference on Image Processing, Las Vegas, NV, USA.","DOI":"10.1109\/ICIP.2006.312611"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1117\/1.482742","article-title":"Human face recognition using PCA on wavelet subband","volume":"9","author":"Feng","year":"2000","journal-title":"J. Electron. Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Naik, G.R., Pendharkar, G., and Nguyen, H.T. (2016, January 16\u201320). Wavelet PCA for automatic identification of walking with and without an exoskeleton on a treadmill using pressure and accelerometer sensors. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591117"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"5017","DOI":"10.1109\/TIP.2015.2475625","article-title":"PCANet: A simple deep learning baseline for image classification?","volume":"24","author":"Chan","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"45153","DOI":"10.1109\/ACCESS.2018.2865425","article-title":"Face recognition based on CSGF (2D) 2 PCANet","volume":"6","author":"Kong","year":"2018","journal-title":"IEEE Access"},{"key":"ref_31","unstructured":"Rick Chang, J., Li, C.L., Poczos, B., Vijaya Kumar, B., and Sankaranarayanan, A.C. (2021, February 17). OneNet Tensorflow Implementation. Available online: https:\/\/github.com\/rick-chang\/OneNet\/tree\/master\/admm."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1007\/s10915-018-0757-z","article-title":"Global convergence of ADMM in nonconvex nonsmooth optimization","volume":"78","author":"Wang","year":"2019","journal-title":"J. Sci. Comput."},{"key":"ref_33","unstructured":"Huang, G.B., Ramesh, M., Berg, T., and Learned-Miller, E. (2007). Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, University of Massachusetts. Technical Report 07-49."},{"key":"ref_34","unstructured":"Harvey, A., and LaPlace, J. (2021, February 17). MegaPixels: Origins, Ethics, and Privacy Implications of Publicly Available Face Recognition Image Datasets. Available online: https:\/\/megapixels.cc."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Joshi, M.R., Nkenyereye, L., Joshi, G.P., Islam, S.M.R., Abdullah-Al-Wadud, M., and Shrestha, S. (2020). Auto-Colorization of Historical Images Using Deep Convolutional Neural Networks. Mathematics, 8.","DOI":"10.3390\/math8122258"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Rajendran, G.B., Kumarasamy, U.M., Zarro, C., Divakarachari, P.B., and Ullo, S.L. (2020). Land-Use and Land-Cover Classification Using a Human Group-Based Particle Swarm Optimization Algorithm with an LSTM Classifier on Hybrid Pre-Processing Remote-Sensing Images. Remote Sens., 12.","DOI":"10.3390\/rs12244135"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"J\u00e4ntschi, L. (2019). A Test Detecting the Outliers for Continuous Distributions Based on the Cumulative Distribution Function of the Data Being Tested. Symmetry, 11.","DOI":"10.3390\/sym11060835"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"J\u00e4ntschi, L. (2020). Detecting Extreme Values with Order Statistics in Samples from Continuous Distributions. Mathematics, 8.","DOI":"10.3390\/math8020216"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/6\/1083\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:17:50Z","timestamp":1760163470000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/6\/1083"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,17]]},"references-count":39,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["sym13061083"],"URL":"https:\/\/doi.org\/10.3390\/sym13061083","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2021,6,17]]}}}