{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T02:39:54Z","timestamp":1772678394255,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2017,11,30]],"date-time":"2017-11-30T00:00:00Z","timestamp":1512000000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The nonparametric Sample Entropy (SE) estimator has become a standard for the quantification of structural complexity of nonstationary time series, even in critical cases of unfavorable noise levels. The SE has proven very successful for signals that exhibit a certain degree of the underlying structure, but do not obey standard probability distributions, a typical case in real-world scenarios such as with physiological signals. However, the SE estimates structural complexity based on uncertainty rather than on (self) correlation, so that, for reliable estimation, the SE requires long data segments, is sensitive to spikes and erratic peaks in data, and owing to its amplitude dependence it exhibits lack of precision for signals with long-term correlations. To this end, we propose a class of new entropy estimators based on the similarity of embedding vectors, evaluated through the angular distance, the Shannon entropy and the coarse-grained scale. Analysis of the effects of embedding dimension, sample size and tolerance shows that the so introduced Cosine Similarity Entropy (CSE) and the enhanced Multiscale Cosine Similarity Entropy (MCSE) are amplitude-independent and therefore superior to the SE when applied to short time series. Unlike the SE, the CSE is shown to yield valid entropy values over a broad range of embedding dimensions. By evaluating the CSE and the MCSE over a variety of benchmark synthetic signals as well as for real-world data (heart rate variability of three different cardiovascular pathologies), the proposed algorithms are demonstrated to be able to quantify degrees of structural complexity in the context of self-correlation over small to large temporal scales, thus offering physically meaningful interpretations and rigor in the understanding the intrinsic properties of the structural complexity of a system, such as the number of its degrees of freedom.<\/jats:p>","DOI":"10.3390\/e19120652","type":"journal-article","created":{"date-parts":[[2017,11,30]],"date-time":"2017-11-30T12:02:51Z","timestamp":1512043371000},"page":"652","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["Cosine Similarity Entropy: Self-Correlation-Based Complexity Analysis of Dynamical Systems"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4270-7683","authenticated-orcid":false,"given":"Theerasak","family":"Chanwimalueang","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Imperial College, SW7 2AZ London, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8432-3963","authenticated-orcid":false,"given":"Danilo","family":"Mandic","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Imperial College, SW7 2AZ London, UK"}]}],"member":"1968","published-online":{"date-parts":[[2017,11,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1073\/pnas.88.6.2297","article-title":"Approximate entropy as a measure of system complexity","volume":"88","author":"Pincus","year":"1991","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1111\/j.1749-6632.2001.tb02755.x","article-title":"Assessing serial irregularity and its implications for health","volume":"954","author":"Pincus","year":"2001","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Rand, D., and Young, L.S. (1981). Detecting Strange Attractors in Turbulence. Dynamical Systems and Turbulence, Springer.","DOI":"10.1007\/BFb0091903"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1103\/PhysRevLett.45.712","article-title":"Geometry from a time series","volume":"45","author":"Packard","year":"1980","journal-title":"Phys. Rev. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.physd.2003.11.001","article-title":"The delay vector variance method for detecting determinism and nonlinearity in time series","volume":"190","author":"Gautama","year":"2004","journal-title":"Phys. D"},{"key":"ref_6","first-page":"H2039","article-title":"Physiological time-series analysis using approximate entropy and sample entropy","volume":"278","author":"Richman","year":"2000","journal-title":"Am. J. Physiol."},{"key":"ref_7","first-page":"1027","article-title":"Study of sample entropy ideal computational parameters in the estimation of atrial fibrillation organization from the ECG","volume":"37","author":"Alcaraz","year":"2010","journal-title":"Comput. Cardiol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.3390\/e15031069","article-title":"Time series analysis using composite multiscale entropy","volume":"15","author":"Wu","year":"2013","journal-title":"Entropy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.artmed.2011.06.007","article-title":"Comparative study of approximate entropy and sample entropy robustness to spikes","volume":"53","author":"Aboy","year":"2011","journal-title":"Artif. Intell. Med."},{"key":"ref_10","first-page":"R789","article-title":"Sample entropy analysis of neonatal heart rate variability","volume":"283","author":"Lake","year":"2002","journal-title":"Am. J. Physiol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1109\/TNSRE.2007.897025","article-title":"Characterization of surface EMG signal based on fuzzy entropy","volume":"15","author":"Chen","year":"2007","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7140","DOI":"10.1016\/j.physleta.2008.10.049","article-title":"Measuring time series regularity using nonlinear similarity-based sample entropy","volume":"372","author":"Xie","year":"2008","journal-title":"Phys. Lett. A"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.medengphy.2008.04.005","article-title":"Measuring complexity using FuzzyEn, ApEn, and SampEn","volume":"31","author":"Chen","year":"2009","journal-title":"Med. Eng. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3926","DOI":"10.1016\/j.physleta.2010.07.058","article-title":"Using the modified sample entropy to detect determinism","volume":"374","author":"Xie","year":"2010","journal-title":"Phys. Lett. A"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"16","DOI":"10.3389\/fncom.2015.00016","article-title":"EEG entropy measures in anesthesia","volume":"9","author":"Liang","year":"2015","journal-title":"Front. Comput. Neurosci."},{"key":"ref_16","unstructured":"Gan, C., and Learmonth, G. (arXiv, 2015). Comparing entropy with tests for randomness as a measure of complexity in time series, arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s40708-016-0040-8","article-title":"The structure function as new integral measure of spatial and temporal properties of multichannel EEG","volume":"3","author":"Trifonov","year":"2016","journal-title":"Brain Inform."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1103\/PhysRevLett.89.068102","article-title":"Multiscale entropy analysis of complex physiologic time series","volume":"89","author":"Costa","year":"2002","journal-title":"Phys. Rev. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.physa.2003.08.022","article-title":"Multiscale entropy analysis of human gait dynamics","volume":"330","author":"Costa","year":"2003","journal-title":"Phys. A"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"21906","DOI":"10.1103\/PhysRevE.71.021906","article-title":"Multiscale entropy analysis of biological signals","volume":"71","author":"Costa","year":"2005","journal-title":"Phys. Rev. E"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"20901","DOI":"10.1103\/PhysRevE.78.020901","article-title":"Complex dynamics of human red blood cell flickering: Alterations with in vivo aging","volume":"78","author":"Costa","year":"2008","journal-title":"Phys. Rev. E"},{"key":"ref_22","unstructured":"Carter, T. (2017, September 30). An Introduction to Information Theory and Entropy. Available online: http:\/\/astarte.csustan.edu\/~tom\/SFI-CSSS\/info-theory\/info-lec.pdf."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Steele, M.J. (2004). The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities, The Mathematical Association of America.","DOI":"10.1017\/CBO9780511817106"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Deza, E., and Deza, M.M. (2009). Encyclopedia of Distances, Springer.","DOI":"10.1007\/978-3-642-00234-2"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yao, J., Yang, Y., S\u0142owi\u0144ski, R., Greco, S., Li, H., Mitra, S., and Polkowski, L. (2012). The Triangle Inequality Versus Projection onto a Dimension in Determining Cosine Similarity Neighborhoods of Non-negative Vectors. Rough Sets and Current Trends in Computing, Springer.","DOI":"10.1007\/978-3-642-32115-3"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Kryszkiewicz, M. (2014). The Cosine Similarity in Terms of the Euclidean Distance. Encyclopedia of Business Analytics and Optimization, IGI Global.","DOI":"10.4018\/978-1-4666-5202-6.ch223"},{"key":"ref_27","unstructured":"Abbad, A., Abbad, K., and Tairi, H. (April, January 29). Face Recognition Based on City-block and Mahalanobis Cosine Distance. Proceedings of the International Conference on Computer Graphics, Imaging and Visualization (CGiV), Beni Mellal, Morocco."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1109\/TASLP.2013.2285474","article-title":"A Study of the cosine distance-based mean shift for telephone speech diarization","volume":"22","author":"Senoussaoui","year":"2014","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sahu, L., and Mohan, B.R. (2014, January 15\u201317). An Improved K-means Algorithm Using Modified Cosine Distance Measure for Document Clustering Using Mahout with Hadoop. Proceedings of the International Conference on Industrial and Information Systems (ICIIS), Gwalior, India.","DOI":"10.1109\/ICIINFS.2014.7036661"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"4372","DOI":"10.1109\/TIP.2015.2451173","article-title":"Angular-similarity-preserving binary signatures for linear subspaces","volume":"24","author":"Ji","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","first-page":"240","article-title":"Note on regression and inheritance in the case of two parents","volume":"58","author":"Pearson","year":"1985","journal-title":"Proc. R. Soc. Lond."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1214\/ss\/1177012580","article-title":"Francis Galton\u2019s account of the invention of correlation","volume":"4","author":"Stigler","year":"1989","journal-title":"Stat. Sci."},{"key":"ref_33","first-page":"2769","article-title":"Measuring and testing dependence by correlation of distances","volume":"35","author":"Rizzo","year":"2007","journal-title":"Ann. Stat."},{"key":"ref_34","unstructured":"Josh Patterson, A.G. (2015). Deep Learning a Practitioner\u2019s Approach, O\u2019Reilly Media. [1st ed.]."},{"key":"ref_35","first-page":"H1643","article-title":"Physiological time-series analysis: What does regularity quantify?","volume":"266","author":"Pincus","year":"1994","journal-title":"Am. J. Physiol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3069","DOI":"10.1016\/j.physd.2008.06.005","article-title":"The effect of time delay on approximate & sample Entropy calculations","volume":"237","author":"Kaffashi","year":"2008","journal-title":"Phys. D"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/S0076-6879(04)84011-4","article-title":"Sample entropy","volume":"384","author":"Richman","year":"2004","journal-title":"Methods Enzymol."},{"key":"ref_38","unstructured":"Gautama, T., Mandic, D.P., and Van Hulle, M.M. (2003, January 6\u201310). A Differential Entropy Based Method for Determining the Optimal Embedding Parameters of a Signal. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Hong Kong, China."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Deisboeck, T.S., and Kresh, J.Y. (2006). Methods and Techniques of Complex Systems Science: An Overview. Complex Systems Science in Biomedicine, Springer.","DOI":"10.1007\/978-0-387-33532-2"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1806","DOI":"10.1001\/jama.1992.03480130122036","article-title":"Loss of complexity and aging. Potential applications of fractals and chaos theory to senescence","volume":"267","author":"Lipsitz","year":"1992","journal-title":"J. Am. Med. Assoc."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"E215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_42","unstructured":"Moody, G.B., and Mark, R.G. (1983, January 4\u20137). A new method for detecting atrial fibrillation using RR intervals. Proceedings of the International Conference on Computers in Cardiology, Aachen, Germany."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/12\/652\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:52:04Z","timestamp":1760208724000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/12\/652"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,30]]},"references-count":42,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2017,12]]}},"alternative-id":["e19120652"],"URL":"https:\/\/doi.org\/10.3390\/e19120652","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,30]]}}}