{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T18:25:47Z","timestamp":1775931947983,"version":"3.50.1"},"reference-count":43,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2015,5,18]],"date-time":"2015-05-18T00:00:00Z","timestamp":1431907200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Recently, sparse adaptive learning algorithms have been developed to exploit system sparsity as well as to mitigate various noise disturbances in many applications. In particular, in sparse channel estimation, the parameter vector with sparsity characteristic can be well estimated from noisy measurements through a sparse adaptive filter. In previous studies, most works use the mean square error (MSE) based cost to develop sparse filters, which is rational under the assumption of Gaussian distributions. However, Gaussian assumption does not always hold in real-world environments. To address this issue, we incorporate in this work an l1-norm or a reweighted l1-norm into the minimum error entropy (MEE) criterion to develop new sparse adaptive filters, which may perform much better than the MSE based methods, especially in heavy-tailed non-Gaussian situations, since the error entropy can capture higher-order statistics of the errors. In addition, a new approximator of l0-norm, based on the correntropy induced metric (CIM), is also used as a sparsity penalty term (SPT). We analyze the mean square convergence of the proposed new sparse adaptive filters. An energy conservation relation is derived and a sufficient condition is obtained, which ensures the mean square convergence. Simulation results confirm the superior performance of the new algorithms.<\/jats:p>","DOI":"10.3390\/e17053419","type":"journal-article","created":{"date-parts":[[2015,5,18]],"date-time":"2015-05-18T10:56:37Z","timestamp":1431946597000},"page":"3419-3437","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Minimum Error Entropy Algorithms with Sparsity Penalty Constraints"],"prefix":"10.3390","volume":"17","author":[{"given":"Zongze","family":"Wu","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510640, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siyuan","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510640, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wentao","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1710-3818","authenticated-orcid":false,"given":"Badong","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jose","family":"Principe","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, China"},{"name":"Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2015,5,18]]},"reference":[{"key":"ref_1","unstructured":"Chen, Y., Gu, Y., and Hero, A.O. (2009, January 19\u201324). Sparse LMS for system identification, Taipei, Taiwan."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1109\/LSP.2009.2024736","article-title":"l0 norm constraint LMS algorithm for sparse system identification","volume":"16","author":"Gu","year":"2009","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1049\/iet-spr.2012.0125","article-title":"Robust Zero-point Attraction Least Mean Square Algorithm on Near Sparse System Identification","volume":"7","author":"Jin","year":"2013","journal-title":"IET Signal Process."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3289","DOI":"10.1016\/j.sigpro.2010.05.015","article-title":"Convergence analysis of sparse LMS algorithms with l1-norm penalty based on white input signal","volume":"90","author":"Shi","year":"2010","journal-title":"Signal Process"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yin, D., So, H.C., and Gu, Y. (2014, January 4\u20139). Sparse Constraint Affine Projection Algorithm with Parallel Implementation and Application in Compressive Sensing, Florence, Italy.","DOI":"10.1109\/ICASSP.2014.6855005"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4013","DOI":"10.1109\/TSP.2010.2048103","article-title":"SPARLS: The sparse RLS algorithm","volume":"58","author":"Babadi","year":"2010","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1016\/j.sigpro.2012.10.008","article-title":"Gradient optimization p-norm-like constraint LMS algorithm for sparse system estimation","volume":"93","author":"Wu","year":"2013","journal-title":"Signal Process"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1002\/acs.2428","article-title":"Sparse leaky LMS algorithm for system identification and its convergence analysis","volume":"28","author":"Salman","year":"2014","journal-title":"Int. J. Adapt. Control Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Aliyu, M.L., Alkassim, M.A., and Salman, M.S. (2014). A p-norm variable step-size LMS algorithm for sparse system identification. Signal Image Video Process.","DOI":"10.1007\/s11760-013-0610-7"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1109\/LCOMM.2013.011113.121586","article-title":"Non-Uniform Norm Constraint LMS Algorithm for Sparse System Identification","volume":"17","author":"Wu","year":"2013","journal-title":"IEEE Commun. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/TCSI.2013.2289407","article-title":"Sparse Adaptive Filtering by an Adaptive Convex Combination of the LMS and the ZA-LMS Algorithms","volume":"61","author":"Das","year":"2014","journal-title":"IEEE Trans. Circuits Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4480","DOI":"10.1109\/TSP.2012.2198468","article-title":"Diffusion sparse least-mean squares over networks","volume":"60","author":"Liu","year":"2012","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1023\/A:1007974400149","article-title":"Nonlinear filtering of non-Gaussian noise","volume":"19","author":"Plataniotis","year":"1997","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2588","DOI":"10.1109\/TSP.2005.849213","article-title":"Nonlinear system identification in impulsive environments","volume":"53","author":"Weng","year":"2005","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_15","unstructured":"Golub, G.H., and van Loan, C.F. (1983). Matrix Computation, the Johns Hopkins University Press."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Principe, J.C. (2010). Information Theoretic Learning: Renyi\u2019s Entropy and Kernel Perspectives, Springer.","DOI":"10.1007\/978-1-4419-1570-2"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, B., Zhu, Y., Hu, J., and Principe, J.C. (2013). System Parameter Identification: Information Criteria and Algorithms, Elsevier.","DOI":"10.1016\/B978-0-12-404574-3.00005-1"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1109\/SP-M.2006.248709","article-title":"From linear adaptive filtering to nonlinear information processing","volume":"23","author":"Erdogmus","year":"2006","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/TSP.2002.1011217","article-title":"An error-entropy minimization for supervised training of nonlinear adaptive systems","volume":"50","author":"Erdogmus","year":"2002","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1007\/s00034-007-9004-9","article-title":"Stochastic gradient algorithm under (h, \u03d5)-entropy criterion","volume":"26","author":"Chen","year":"2007","journal-title":"Circuit Syst. Signal Process"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1016\/j.sigpro.2004.11.028","article-title":"Minimum-entropy estimation in semi-parametric models","volume":"85","author":"Wolsztynski","year":"2005","journal-title":"Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/LSP.2009.2031716","article-title":"The Equivalency of Minimum Error Entropy Criterion and Minimum Dispersion Criterion for Symmetric Stable Signal Processing","volume":"17","author":"Song","year":"2010","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"966","DOI":"10.3390\/e14050966","article-title":"Some further results on the minimum error entropy estimation","volume":"14","author":"Chen","year":"2012","journal-title":"Entropy"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1168","DOI":"10.1109\/TNN.2010.2050212","article-title":"Mean-square convergence analysis of ADALINE training with minimum error entropy criterion","volume":"21","author":"Chen","year":"2010","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2311","DOI":"10.3390\/e14112311","article-title":"On the Smoothed Minimum Error Entropy Criterion","volume":"14","author":"Chen","year":"2012","journal-title":"Entropy"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4011","DOI":"10.1109\/TSP.2013.2265221","article-title":"Diffusion information theoretic learning for distributed estimation over network","volume":"61","author":"Li","year":"2013","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/LCOMM.2002.1010863","article-title":"The minimum error entropy based robust wireless channel tracking in impulsive noise","volume":"6","author":"Xue","year":"2002","journal-title":"IEEE Commun. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5286","DOI":"10.1109\/TSP.2007.896065","article-title":"Correntropy: Properties and Applications in Non-Gaussian Signal Processing","volume":"55","author":"Liu","year":"2007","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_29","unstructured":"Seth, S., and Principe, J.C. (April, January 31). Compressed signal reconstruction using the correntropy induced metric, Las Vegas, NV, USA."},{"key":"ref_30","unstructured":"Wipf, D.P., and Nagarajan, S.S. A new view of automatic relevance determination. Available online: http:\/\/papers.nips.cc\/paper\/3372-a-new-view-of-automatic-relevance-determination."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1184","DOI":"10.1109\/TSP.2011.2178406","article-title":"Survival information potential: A new criterion for adaptive system training","volume":"60","author":"Chen","year":"2012","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1023\/A:1008143417156","article-title":"Learning from examples with information theoretic criteria","volume":"26","author":"Principe","year":"2000","journal-title":"J.VLSI Signal Process. Syst. Signal Image Video Technol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"880","DOI":"10.1109\/LSP.2014.2319308","article-title":"Steady-state Mean-square Error Analysis for Adaptive Filtering under the Maximum Correntropy Criterion","volume":"21","author":"Chen","year":"2014","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1109\/LSP.2012.2204435","article-title":"Maximum correntropy estimation is a smoothed MAP estimation","volume":"19","author":"Chen","year":"2012","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_35","first-page":"192","article-title":"Adaptive filters with error nonlinearities: Mean-square analysis and optimum design","volume":"4","author":"Sayed","year":"2001","journal-title":"EURASIP J. Appl. Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1109\/78.286951","article-title":"Stochastic gradient adaptation under general error criteria","volume":"42","author":"Douglas","year":"1994","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_37","unstructured":"Sayed, A.H. (2003). Fundamentals of Adaptive Filtering, Wiley."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/TSP.2003.820077","article-title":"Mean-square performance of a family of affine projection algorithms","volume":"52","author":"Shin","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"5286","DOI":"10.1109\/LSP.2003.819353","article-title":"A normalized robust mixed-norm adaptive algorithm for system identification","volume":"11","author":"Papoulis","year":"2004","journal-title":"Signal Process. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1109\/5.231338","article-title":"Signal processing with fractional lower order moments: Stable processes and their applications","volume":"81","author":"Shao","year":"1993","journal-title":"Proc. IEEE"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2588","DOI":"10.1109\/TSP.2005.849213","article-title":"Nonlinear system identification in impulsive environments","volume":"53","author":"Weng","year":"2005","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_42","unstructured":"Georgiadis, A.T., and Mulgrew, B. (1999, January 6\u20138). A family of recursive algorithms for channel identification in alpha-stable noise, Bayona, Spain."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1107","DOI":"10.1109\/LCOMM.2011.081011.110813","article-title":"Alpha-stable channel capacity","volume":"15","author":"Wang","year":"2011","journal-title":"IEEE Commun. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3419\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:46:27Z","timestamp":1760215587000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3419"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,18]]},"references-count":43,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,5]]}},"alternative-id":["e17053419"],"URL":"https:\/\/doi.org\/10.3390\/e17053419","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,5,18]]}}}