{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:20:51Z","timestamp":1760145651041,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T00:00:00Z","timestamp":1722902400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"State Grid Information and Telecommunication Group scientific and technological innovation projects \u201cResearch on Power Digital Space Technology System and Key Technologies\u201d","award":["SGIT0000XMJS2310456"],"award-info":[{"award-number":["SGIT0000XMJS2310456"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The minimum error entropy (MEE) criterion finds extensive utility across diverse applications, particularly in contexts characterized by non-Gaussian noise. However, its computational demands are notable, and are primarily attributable to the double summation operation involved in calculating the probability density function (PDF) of the error. To address this, our study introduces a novel approach, termed the fast minimum error entropy (FMEE) algorithm, aimed at mitigating computational complexity through the utilization of polynomial expansions of the error PDF. Initially, the PDF approximation of a random variable is derived via the Gram\u2013Charlier expansion. Subsequently, we proceed to ascertain and streamline the entropy of the random variable. Following this, the error entropy inherent to the linear regression model is delineated and expressed as a function of the regression coefficient vector. Lastly, leveraging the gradient descent algorithm, we compute the regression coefficient vector corresponding to the minimum error entropy. Theoretical scrutiny reveals that the time complexity of FMEE stands at O(n), in stark contrast to the O(n2) complexity associated with MEE. Experimentally, our findings underscore the remarkable efficiency gains afforded by FMEE, with time consumption registering less than 1\u2030 of that observed with MEE. Encouragingly, this efficiency leap is achieved without compromising accuracy, as evidenced by negligible differentials observed between the accuracies of FMEE and MEE. Furthermore, comprehensive regression experiments on real-world electric datasets in northwest China demonstrate that our FMEE outperforms baseline methods by a clear margin.<\/jats:p>","DOI":"10.3390\/a17080341","type":"journal-article","created":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T11:54:19Z","timestamp":1722945259000},"page":"341","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Fast Minimum Error Entropy for Linear Regression"],"prefix":"10.3390","volume":"17","author":[{"given":"Qiang","family":"Li","sequence":"first","affiliation":[{"name":"State Grid Information & Telecommunication Group Co., Ltd., Beijing 102209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Liao","sequence":"additional","affiliation":[{"name":"State Grid Information & Telecommunication Group Co., Ltd., Beijing 102209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Cui","sequence":"additional","affiliation":[{"name":"State Grid Information & Telecommunication Group Co., Ltd., Beijing 102209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Wang","sequence":"additional","affiliation":[{"name":"State Grid Information & Telecommunication Group Co., Ltd., Beijing 102209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3301-2713","authenticated-orcid":false,"given":"Hui","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5253-5598","authenticated-orcid":false,"given":"Qingshu","family":"Guan","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, Xi\u2019an Jiaotong University, Xi\u2019an 710049, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2815","DOI":"10.1109\/TSP.2015.2419189","article-title":"Maximum Entropy PDF Design Using Feature Density Constraints: Applications in Signal Processing","volume":"11","author":"Baggenstoss","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3171","DOI":"10.1038\/s41467-023-38889-5","article-title":"Cu-based high-entropy two-dimensional oxide as stable and active photothermal catalyst","volume":"14","author":"Li","year":"2023","journal-title":"Nat. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bisikalo, O., Kharchenko, V., Kovtun, V., Krak, I., and Pavlov, S. (2023). Parameterization of the stochastic model for evaluating variable small data in the Shannon entropy basis. Entropy, 25.","DOI":"10.3390\/e25020184"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chen, Q., Zhang, F., Su, L., Lin, B., Chen, S., and Zhang, Y. (2024). State Parameter Fusion Estimation for Intelligent Vehicles Based on IMM-MCCKF. Appl. Sci., 14.","DOI":"10.3390\/app14114495"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1471","DOI":"10.1109\/TAC.2016.2582642","article-title":"Maximum entropy kernels for system identification","volume":"62","author":"Carli","year":"2016","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2401887","DOI":"10.1002\/adfm.202401887","article-title":"From Single Metals to High-Entropy Alloys: How Machine Learning Accelerates the Development of Metal Electrocatalysts","volume":"1","author":"Fan","year":"2024","journal-title":"Adv. Funct. Mater."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1109\/TAC.2009.2034927","article-title":"Persistent Dwell-Time Switched Nonlinear Systems: Variation Paradigm and Gauge Design","volume":"55","author":"Han","year":"2010","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1109\/TAC.2016.2585743","article-title":"Stabilization and Entropy Reduction via SDP-Based Design of Fixed-Order Output Feedback Controllers and Tuning Parameters","volume":"62","author":"Chesi","year":"2017","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"12851","DOI":"10.1109\/TIE.2023.3234128","article-title":"Spatiotemporal entropy for abnormality detection and localization of Li-ion battery packs","volume":"70","author":"Wei","year":"2023","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1016\/j.isatra.2023.03.022","article-title":"Intelligent fault detection scheme for constant-speed wind turbines based on improved multiscale fuzzy entropy and adaptive chaotic Aquila optimization-based support vector machine","volume":"138","author":"Wang","year":"2023","journal-title":"ISA Trans."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.patrec.2014.06.012","article-title":"Entropy-based outlier detection using semi-supervised approach with few positive examples","volume":"49","author":"Daneshpazhouh","year":"2014","journal-title":"Pattern Recogn. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"111010","DOI":"10.1016\/j.automatica.2023.111010","article-title":"Decentralized dynamic state estimation for multi-machine power systems with non-Gaussian noises: Outlier detection and localization","volume":"153","author":"Qu","year":"2023","journal-title":"Automatica"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"108806","DOI":"10.1016\/j.sigpro.2022.108806","article-title":"Novel robust minimum error entropy wasserstein distribution kalman filter under model uncertainty and non-gaussian noise","volume":"203","author":"Feng","year":"2023","journal-title":"Signal Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1109\/TSP.2023.3250829","article-title":"Adaptive kernel Kalman filter","volume":"71","author":"Sun","year":"2023","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"8497","DOI":"10.1109\/TNNLS.2022.3230417","article-title":"Beyond homophily and homogeneity assumption: Relation-based frequency adaptive graph neural networks","volume":"35","author":"Wu","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1109\/TAC.2006.872771","article-title":"Minimum entropy filtering for multivariate stochastic systems with non-Gaussian noises","volume":"51","author":"Guo","year":"2006","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4010","DOI":"10.1109\/TSP.2015.2425803","article-title":"Minimum error entropy based sparse representation for robust subspace clustering","volume":"63","author":"Wang","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.neucom.2020.10.081","article-title":"Deep face recognition: A survey","volume":"429","author":"Wang","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"116141","DOI":"10.1016\/j.jsv.2021.116141","article-title":"Nonlinear dynamical system identification using the sparse regression and separable least squares methods","volume":"505","author":"Lin","year":"2021","journal-title":"J. Sound Vib."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3419","DOI":"10.3390\/e17053419","article-title":"Minimum error entropy algorithms with sparsity penalty constraints","volume":"17","author":"Wu","year":"2015","journal-title":"Entropy"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1322","DOI":"10.1109\/TSP.2018.2889951","article-title":"Sparse signal recovery via generalized entropy functions minimization","volume":"67","author":"Huang","year":"2018","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1109\/TAC.2002.806663","article-title":"Minimum entropy control of closed-loop tracking errors for dynamic stochastic systems","volume":"48","author":"Yue","year":"2003","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/9.983388","article-title":"Minimum entropy control of non-Gaussian dynamic stochastic systems","volume":"47","author":"Wang","year":"2002","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.1109\/TVT.2018.2882640","article-title":"Minimum error entropy criterion based channel estimation for massive-MIMO in VLC","volume":"68","author":"Mitra","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1966","DOI":"10.1109\/TSP.2003.812843","article-title":"Convergence properties and data efficiency of the minimum error entropy criterion in adaline training","volume":"51","author":"Erdogmus","year":"2003","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.neucom.2013.04.037","article-title":"Kernel minimum error entropy algorithm","volume":"121","author":"Chen","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"6571","DOI":"10.1109\/TSP.2016.2612169","article-title":"Convergence of gradient descent for minimum error entropy principle in linear regression","volume":"64","author":"Hu","year":"2016","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5549","DOI":"10.3390\/e17085549","article-title":"Convergence of a fixed-point minimum error entropy algorithm","volume":"17","author":"Zhang","year":"2015","journal-title":"Entropy"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/j.jfranklin.2009.11.009","article-title":"On optimal estimations with minimum error entropy criterion","volume":"347","author":"Chen","year":"2010","journal-title":"J. Frankl. Inst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1109\/LSP.2018.2797079","article-title":"A new information theoretic relation between minimum error entropy and maximum correntropy","volume":"25","author":"Heravi","year":"2018","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Yang, S., Tan, J., and Chen, B. (2022). Robust spike-based continual meta-learning improved by restricted minimum error entropy criterion. Entropy, 24.","DOI":"10.3390\/e24040455"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"8797","DOI":"10.1109\/TIM.2020.2999757","article-title":"Robust power system state estimation with minimum error entropy unscented Kalman filter","volume":"69","author":"Dang","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Wang, B., and Hu, T. (2018). Semi-Supervised Minimum Error Entropy Principle with Distributed Method. Entropy, 20.","DOI":"10.3390\/e20120968"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1109\/TNNLS.2016.2636160","article-title":"Insights into the robustness of minimum error entropy estimation","volume":"29","author":"Chen","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"4079","DOI":"10.1109\/TSP.2015.2437836","article-title":"Minimum total error entropy method for parameter estimation","volume":"63","author":"Shen","year":"2015","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"7386","DOI":"10.1109\/TITS.2021.3069362","article-title":"Human trajectory forecasting in crowds: A deep learning perspective","volume":"23","author":"Kothari","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kim, N., and Kwon, K. (2016). Normalized Minimum Error Entropy Algorithm with Recursive Power Estimation. Entropy, 18.","DOI":"10.3390\/e18070239"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1109\/LSP.2019.2890973","article-title":"Granger causality analysis based on quantized minimum error entropy criterion","volume":"26","author":"Chen","year":"2019","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"080401","DOI":"10.1103\/PhysRevLett.127.080401","article-title":"Experimental detection of the correlation R\u00e9nyi entropy in the central spin model","volume":"127","author":"Niknam","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"102359","DOI":"10.1016\/j.aam.2022.102359","article-title":"Representations of degenerate Hermite polynomials","volume":"139","author":"Kim","year":"2022","journal-title":"Adv. Appl. Math. Mech."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"113945","DOI":"10.1016\/j.cam.2021.113945","article-title":"The valid regions of Gram\u2013Charlier densities with high-order cumulants","volume":"407","author":"Lin","year":"2022","journal-title":"J. Comput. Appl. Math."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"129850","DOI":"10.1016\/j.physa.2024.129850","article-title":"Differential entropy estimation with a Paretian kernel: Tail heaviness and smoothing","volume":"1","author":"Matsushita","year":"2024","journal-title":"Phys. A Stat. Mech. Appl."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"13496","DOI":"10.1021\/acsomega.1c01762","article-title":"Testing the test: A comparative study of marine microbial corrosion under laboratory and field conditions","volume":"6","author":"Canales","year":"2021","journal-title":"ACS Omega"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/8\/341\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:30:43Z","timestamp":1760110243000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/8\/341"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,6]]},"references-count":43,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["a17080341"],"URL":"https:\/\/doi.org\/10.3390\/a17080341","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2024,8,6]]}}}