{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:40:41Z","timestamp":1760190041675,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2019,9,17]],"date-time":"2019-09-17T00:00:00Z","timestamp":1568678400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Foundation\u2002of\u2002Guangdong Educational\u2002Committee, China\u2002(innovative and Strong University Project)","award":["2018KQNCX247"],"award-info":[{"award-number":["2018KQNCX247"]}]},{"name":"the Doctoral Scientific Research Foundation of Huizhou University","award":["2018JB024","2018JB025"],"award-info":[{"award-number":["2018JB024","2018JB025"]}]},{"name":"the Science and Technology Planning Project of Guangdong Province","award":["2017A020214011"],"award-info":[{"award-number":["2017A020214011"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Image recovery from compressive sensing (CS) measurement data, especially noisy data has always been challenging due to its implicit ill-posed nature, thus, to seek a domain where a signal can exhibit a high degree of sparsity and to design an effective algorithm have drawn increasingly more attention. Among various sparsity-based models, structured or group sparsity often leads to more powerful signal reconstruction techniques. In this paper, we propose a novel entropy-based algorithm for CS recovery to enhance image sparsity through learning the group sparsity of residual. To reduce the residual of similar packed patches, the group sparsity of residual is described by a Laplacian scale mixture (LSM) model, therefore, each singular value of the residual of similar packed patches is modeled as a Laplacian distribution with a variable scale parameter, to exploit the benefits of high-order dependency among sparse coefficients. Due to the latent variables, the maximum a posteriori (MAP) estimation of the sparse coefficients cannot be obtained, thus, we design a loss function for expectation\u2013maximization (EM) method based on relative entropy. In the frame of EM iteration, the sparse coefficients can be estimated with the denoising-based approximate message passing (D-AMP) algorithm. Experimental results have shown that the proposed algorithm can significantly outperform existing CS techniques for image recovery.<\/jats:p>","DOI":"10.3390\/e21090900","type":"journal-article","created":{"date-parts":[[2019,9,23]],"date-time":"2019-09-23T04:50:21Z","timestamp":1569214221000},"page":"900","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["An Entropy-Based Algorithm with Nonlocal Residual Learning for Image Compressive Sensing Recovery"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6625-3949","authenticated-orcid":false,"given":"Zhonghua","family":"Xie","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Huizhou University, Huizhou 516007, China"}]},{"given":"Lingjun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Huizhou University, Huizhou 516007, China"}]},{"given":"Cui","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,9,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1289","DOI":"10.1109\/TIT.2006.871582","article-title":"Compressed Sensing","volume":"52","author":"Donoho","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"7136","DOI":"10.3390\/s150407136","article-title":"Informational Analysis for Compressive Sampling in Radar Imaging","volume":"15","author":"Zhang","year":"2015","journal-title":"Sensors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4655","DOI":"10.1109\/TIT.2007.909108","article-title":"Signal Recovery from Random Measurements Via Orthogonal Matching Pursuit","volume":"53","author":"Tropp","year":"2007","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1194","DOI":"10.1007\/s00034-018-0906-5","article-title":"Robust Frequency Estimation of Multi-sinusoidal Signals Using Orthogonal Matching Pursuit with Weak Derivatives Criterion","volume":"38","author":"Fu","year":"2019","journal-title":"Circuits Syst. Signal Process."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1137\/080716542","article-title":"A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems","volume":"2","author":"Beck","year":"2009","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1137\/090756855","article-title":"NESTA: A Fast and Accurate First Order Method for Sparse Recovery","volume":"4","author":"Becker","year":"2009","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1109\/LSP.2009.2037532","article-title":"Tree-Structured Compressive Sensing with Variational Bayesian Analysis","volume":"17","author":"He","year":"2010","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Shekaramiz, M., Moon, T.K., and Gunther, J.H. (2019). Bayesian Compressive Sensing of Sparse Signals with Unknown Clustering Patterns. Entropy, 21.","DOI":"10.3390\/e21030247"},{"key":"ref_9","unstructured":"Chen, Z., Ma, S., and Wen, G. (2014, January 14\u201318). Image Compressive-Sensing Recovery Using Structured Laplacian Sparsity in DCT Domain and Multi-Hypothesis Prediction. Proceedings of the 2014 IEEE International Conference on Multimedia & Expo (ICME), Chengdu, China."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3483","DOI":"10.1109\/TIP.2011.2150231","article-title":"Multivariate Compressive Sensing for Image Reconstruction in the Wavelet Domain: Using Scale Mixture Models","volume":"20","author":"Wu","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","article-title":"Nonlinear Total Variation Based Noise Removal Algorithms","volume":"60","author":"Rudin","year":"1992","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1016\/j.media.2011.06.001","article-title":"Efficient MR Image Reconstruction for Compressed MR Imaging","volume":"15","author":"Huang","year":"2011","journal-title":"Med. Image Anal."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"25889","DOI":"10.1007\/s11042-018-5824-9","article-title":"Compressive Sensing Based Simultaneous Fusion and Compression of Multi-Focus Images Using Learned Dictionary","volume":"77","author":"Ashwini","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/LSP.2012.2229976","article-title":"Improving Dictionary Learning: Multiple Dictionary Updates and Coefficient Reuse","volume":"20","author":"Smith","year":"2012","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/TIP.2007.911473","article-title":"Image Restoration Using Space-Variant Gaussian Scale Mixtures in Overcomplete Pyramids","volume":"17","author":"Mancera","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3042","DOI":"10.1109\/TSP.2010.2044837","article-title":"Block-Sparse Signals: Uncertainty Relations and Efficient Recovery","volume":"58","author":"Eldar","year":"2010","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1377","DOI":"10.1016\/j.mri.2014.07.016","article-title":"Exploiting the Wavelet Structure in Compressed Sensing MRI","volume":"32","author":"Chen","year":"2014","journal-title":"Magn. Reson. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3439","DOI":"10.1109\/TSP.2012.2191780","article-title":"Compressive Imaging Using Approximate Message Passing and A Markov-Tree Prior","volume":"60","author":"Som","year":"2012","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1109\/LGRS.2013.2284288","article-title":"Compressive Sensing-Based ISAR Imaging via the Combination of the Sparsity and Nonlocal Total Variation","volume":"11","author":"Zhang","year":"2014","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1016\/j.jvcir.2013.06.019","article-title":"A learning-based method for compressive image recovery","volume":"24","author":"Dong","year":"2013","journal-title":"J. Vis. Commun. Image Represent"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Seitzer, M., Yang, G., Schlemper, J., Oktay, O., W\u00fcrfl, T., Christlein, V., Wong, T., Mohiaddin, R., Firmin, D., and Keegan, J. (2018, January 16\u201320). Adversarial and Perceptual Refinement for Compressed Sensing MRI Reconstruction. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Granada, Spain.","DOI":"10.1007\/978-3-030-00928-1_27"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mousavi, A., and Baraniuk, R.G. (2017, January 5\u20139). Learning to Invert: Signal Recovery via Deep Convolutional Networks. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7952561"},{"key":"ref_23","unstructured":"Yao, H., Dai, F., Zhang, D., Ma, Y., Zhang, S., and Zhang, Y. (2017). DR2-net: Deep Residual Reconstruction Network for Image Compressive Sensing. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1109\/TCI.2018.2846413","article-title":"Convolutional Neural Networks for Non-iterative Reconstruction of Compressively Sensed Images","volume":"4","author":"Lohit","year":"2018","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"ref_25","unstructured":"Yang, Y., Sun, J., Li, H., and Xu, Z. (2018). ADMM-CS Net: A Deep Learning Approach for Image Compressive Sensing. IEEE Trans. Pattern Anal. Mach. Intell., 1."},{"key":"ref_26","unstructured":"Metzler, C.A., Mousavi, A., and Baraniuk, R.G. (2017). Learned D-AMP: Principled Neural Network based Compressive Image Recovery. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1310","DOI":"10.1109\/TMI.2017.2785879","article-title":"DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction","volume":"37","author":"Yang","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_28","unstructured":"Yu, S., Dong, H., Yang, G., Slabaugh, G., Dragotti, P.L., Ye, X., Liu, F., Arridge, S., Keegan, J., and Firmin, D. (2017). Deep De-Aliasing for Fast Compressive Sensing MRI. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Schlemper, J., Yang, G., Ferreira, P., Scott, A., McGill, L.A., Khalique, Z., Gorodezky, M., Roehl, M., Keegan, J., and Pennell, D. (2018, January 16\u201320). Stochastic Deep Compressive Sensing for the Reconstruction of Diffusion Tensor Cardiac MRI. Proceedings of the 2018 International Conference on Medical Image Computing and Computer-Assisted Intervention, Granada, Spain.","DOI":"10.1007\/978-3-030-00928-1_34"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3618","DOI":"10.1109\/TIP.2014.2329449","article-title":"Compressive Sensing via Nonlocal Low-rank Regularization","volume":"23","author":"Dong","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5117","DOI":"10.1109\/TIT.2016.2556683","article-title":"From Denoising to Compressed Sensing","volume":"62","author":"Metzler","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Metzler, C.A., Maleki, A., and Baraniuk, R.G. (2015, January 27\u201330). BM3D-AMP: A New Image Recovery Algorithm Based on BM3D Denoising. Proceedings of the 2015 IEEE International Conference Image Processing (ICIP), Quebec City, QC, Canada.","DOI":"10.1109\/ICIP.2015.7351377"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2080","DOI":"10.1109\/TIP.2007.901238","article-title":"Image Denoising By Sparse 3-D Transform-Domain Collaborative Filtering","volume":"16","author":"Dabov","year":"2007","journal-title":"IEEE Trans. Image Process."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1109\/JETCAS.2012.2220391","article-title":"Image Compressive Sensing Recovery via Collaborative Sparsity","volume":"2","author":"Zhang","year":"2012","journal-title":"IEEE J. Emerg. Sel. Top. Circuits Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1109","DOI":"10.1016\/j.image.2012.09.003","article-title":"Image Reconstruction with Locally Adaptive Sparsity and Nonlocal Robust Regularization","volume":"27","author":"Dong","year":"2012","journal-title":"Signal Process. Image Commun."},{"key":"ref_36","first-page":"18","article-title":"An Approximate Message Passing Algorithm with Composite Sparse Constraint for CS Reconstruction","volume":"45","author":"Xie","year":"2017","journal-title":"J. S. China Univ. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2459","DOI":"10.1109\/TMI.2015.2437894","article-title":"LRTV: MR Image Super-Resolution with Low-Rank and Total Variation Regularizations","volume":"34","author":"Shi","year":"2015","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_38","unstructured":"Garrigues, P., and Olshausen, B.A. (2010, January 6\u20139). Group Sparse Coding with A Laplacian Scale Mixture Prior. Proceedings of the Advances in Neural Information Processing Systems (NIPS), Vancouver, BC, Canada."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum-likelihood from Incomplete Data via the EM Algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"J. R. Stat. Soc."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"18914","DOI":"10.1073\/pnas.0909892106","article-title":"Message Passing Algorithms for Compressed Sensing","volume":"106","author":"Donoho","year":"2009","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1002\/cpa.20042","article-title":"An Iterative Thresholding Algorithm for Linear Inverse Problems with A Sparsity Constraint","volume":"57","author":"Daubechies","year":"2004","journal-title":"Commun. Pure Appl. Math."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"3869","DOI":"10.1109\/TIP.2014.2332755","article-title":"High-Accuracy Total Variation with Application to Compressed Video Sensing","volume":"23","author":"Hosseini","year":"2014","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1137\/070703983","article-title":"Bregman Iterative Algorithms For l1-minimization with Applications to Compressed Sensing","volume":"1","author":"Yin","year":"2008","journal-title":"SIAM J. Imaging Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3396","DOI":"10.1109\/TIT.2013.2239356","article-title":"Accurate Prediction of Phase Transitions in Compressed Sensing via A Connection to Minimax Denoising","volume":"59","author":"Donoho","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, X., and Liang, J. (2014, January 4\u20139). Side Information-aided Compressed Sensing Reconstruction Via Approximate Message Passing. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2014.6854217"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hill, P.R., Kim, J.H., Basarab, A., Kouam\u00e9, D., Bull, D.R., and Achim, A. (2016, January 25\u201328). Compressive Imaging Using Approximate Message Passing and A Cauchy Prior in The Wavelet Domain. Proceedings of the IEEE International Conference Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532812"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1540","DOI":"10.1109\/TIP.2008.2001404","article-title":"Monte-Carlo Sure: A Black-Box Optimization of Regularization Parameters for General Denoising Algorithms","volume":"17","author":"Ramani","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s11263-016-0930-5","article-title":"Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision","volume":"121","author":"Gu","year":"2017","journal-title":"Int. J. Comput. Vis."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1109\/TIP.2012.2221729","article-title":"Nonlocal Image Restoration with Bilateral Variance Estimation: A Low-Rank Approach","volume":"22","author":"Dong","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rodger, J.A. (2019). QuantumIS: A Qualia Consciousness Awareness and Information Theory Quale Approach to Reducing Strategic Decision-Making Entropy. Entropy, 21.","DOI":"10.3390\/e21020125"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/9\/900\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:21:02Z","timestamp":1760188862000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/9\/900"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,17]]},"references-count":50,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["e21090900"],"URL":"https:\/\/doi.org\/10.3390\/e21090900","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2019,9,17]]}}}