{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T13:39:42Z","timestamp":1780061982199,"version":"3.54.0"},"reference-count":79,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2016,5,19]],"date-time":"2016-05-19T00:00:00Z","timestamp":1463616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive uncertainty associated with a specific state, rather than averaged over all states. We provide a data-driven approach for estimating the specific entropy rate of an observed time series. Finally, we consider three case studies of estimating the specific entropy rate from synthetic and physiological data relevant to the analysis of heart rate variability.<\/jats:p>","DOI":"10.3390\/e18050190","type":"journal-article","created":{"date-parts":[[2016,5,19]],"date-time":"2016-05-19T20:43:57Z","timestamp":1463690637000},"page":"190","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Specific Differential Entropy Rate Estimation for Continuous-Valued Time Series"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4624-3951","authenticated-orcid":false,"given":"David","family":"Darmon","sequence":"first","affiliation":[{"name":"Department of Military and Emergency Medicine, Uniformed Services University, Bethesda, MD 20814, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2016,5,19]]},"reference":[{"key":"ref_1","unstructured":"Shalizi, C.R. (2006). Complex Systems Science in Biomedicine, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Peliti, L., and Vulpiani, A. (1988). Measures of Complexity, Springer-Verlag.","DOI":"10.1007\/3-540-50316-1"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, M., and Vit\u00e1nyi, P. (1993). An Introduction to Kolmogorov Complexity and Its Applications, Springer Science & Business Media.","DOI":"10.1007\/978-1-4757-3860-5"},{"key":"ref_4","unstructured":"Rissanen, J. (1989). Stochastic Complexity in Statistical Inquiry, World Scientific."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1007\/BF00668821","article-title":"Toward a quantitative theory of self-generated complexity","volume":"25","author":"Grassberger","year":"1986","journal-title":"Int. J. Theor. Phys."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Crutchfield, J.P., and Young, K. (1989). Inferring statistical complexity. Phys. Rev. Lett., 63.","DOI":"10.1103\/PhysRevLett.63.105"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1023\/A:1010388907793","article-title":"Computational mechanics: Pattern and prediction, structure and simplicity","volume":"104","author":"Shalizi","year":"2001","journal-title":"J. Stat. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"037109","DOI":"10.1063\/1.3637494","article-title":"Anatomy of a bit: Information in a time series observation","volume":"21","author":"James","year":"2011","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_9","first-page":"861","article-title":"A new metric invariant of transient dynamical systems and automorphisms in Lebesgue spaces","volume":"119","author":"Kolmogorov","year":"1958","journal-title":"Dokl. Akad. Nauk SSSR"},{"key":"ref_10","first-page":"768","article-title":"On the concept of entropy for a dynamic system","volume":"124","author":"Sinai","year":"1959","journal-title":"Dokl. Akad. Nauk SSSR"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kantz, H., and Schreiber, T. (2004). Nonlinear Time Series Analysis, Cambridge University Press.","DOI":"10.1017\/CBO9780511755798"},{"key":"ref_13","first-page":"417","article-title":"Equations of motion from a data series","volume":"1","author":"Crutchfield","year":"1987","journal-title":"Complex Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/TBME.2005.859782","article-title":"Renyi entropy measures of heart rate Gaussianity","volume":"53","author":"Lake","year":"2006","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2018","DOI":"10.1103\/PhysRevE.62.2018","article-title":"Dynamical entropy for systems with stochastic perturbation","volume":"62","author":"Ostruszka","year":"2000","journal-title":"Phys. Rev. E"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1109\/18.32121","article-title":"Information and entropy in strange attractors","volume":"35","author":"Fraser","year":"1989","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_17","unstructured":"Badii, R., and Politi, A. (1999). Complexity: Hierarchical Structures and Scaling in Physics, Cambridge University Press."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Fan, J., and Yao, Q. (2003). Nonlinear Time Series: Nonparametric and Parametric Methods, Springer Science & Business Media.","DOI":"10.1007\/b97702"},{"key":"ref_19","unstructured":"Chan, K.S., and Tong, H. (2013). Chaos: A Statistical Perspective, Springer Science & Business Media."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Michalowicz, J.V., Nichols, J.M., and Bucholtz, F. (2013). Handbook of Differential Entropy, CRC Press.","DOI":"10.1201\/b15991"},{"key":"ref_21","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ihara, S. (1993). Information Theory for Continuous Systems, World Scientific.","DOI":"10.1142\/9789814355827"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Grimmett, G., and Stirzaker, D. (2001). Probability and Random Processes, Oxford University Press.","DOI":"10.1093\/oso\/9780198572237.001.0001"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s11203-004-0383-2","article-title":"On the non-parametric prediction of conditionally stationary sequences","volume":"8","author":"Caires","year":"2005","journal-title":"Stat. Inference Stoch. Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1111\/j.2517-6161.1994.tb02009.x","article-title":"Quantifying the influence of initial values on non-linear prediction","volume":"56","author":"Yao","year":"1994","journal-title":"J. R. Stat. Soc. Ser. B Methodol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1098\/rsta.1994.0096","article-title":"On prediction and chaos in stochastic systems","volume":"348","author":"Yao","year":"1994","journal-title":"Philos. Trans. R. Soc. Lond. A Math. Phys. Eng. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1088\/0954-898X_10_4_303","article-title":"How to measure the information gained from one symbol","volume":"10","author":"DeWeese","year":"1999","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"026110","DOI":"10.1103\/PhysRevE.77.026110","article-title":"Local information transfer as a spatiotemporal filter for complex systems","volume":"77","author":"Lizier","year":"2008","journal-title":"Phys. Rev. E"},{"key":"ref_29","unstructured":"Lizier, J.T. (2014). Directed Information Measures in Neuroscience, Springer."},{"key":"ref_30","first-page":"9","article-title":"Sample estimate of the entropy of a random vector","volume":"23","author":"Kozachenko","year":"1987","journal-title":"Probl. Peredachi Inf."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4374","DOI":"10.1109\/TIT.2013.2251456","article-title":"Ensemble Estimators for Multivariate Entropy Estimation","volume":"59","author":"Sricharan","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","unstructured":"Gao, S., Ver Steeg, G., and Galstyan, A. (2015). Estimating Mutual Information by Local Gaussian Approximation."},{"key":"ref_34","unstructured":"Singh, S., and P\u00f3czos, B. (2016). Analysis of k-Nearest Neighbor Distances with Application to Entropy Estimation."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"013310","DOI":"10.1103\/PhysRevE.93.013310","article-title":"Nonparametric k-nearest-neighbor entropy estimator","volume":"93","author":"Lombardi","year":"2016","journal-title":"Phys. Rev. E"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1214\/aos\/1176348768","article-title":"Variable Kernel Density Estimation","volume":"20","author":"Terrell","year":"1992","journal-title":"Ann. Stat."},{"key":"ref_37","first-page":"31","article-title":"Conditional probability density and regression estimators","volume":"Volume 25","author":"Rosenblatt","year":"1969","journal-title":"Multivariate Analysis II"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1198\/016214504000000548","article-title":"Cross-validation and the estimation of conditional probability densities","volume":"99","author":"Hall","year":"2004","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v027.i05","article-title":"Nonparametric Econometrics: The np Package","volume":"27","author":"Hayfield","year":"2008","journal-title":"J. Stat. Softw."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Bosq, D. (2012). Nonparametric Statistics for Stochastic Processes: Estimation and Prediction, Springer Science & Business Media.","DOI":"10.1002\/9781118562024.ch15"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/S0167-2789(02)00432-3","article-title":"Information transfer in continuous processes","volume":"166","author":"Kaiser","year":"2002","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1093\/biomet\/81.2.351","article-title":"A cross-validatory method for dependent data","volume":"81","author":"Burman","year":"1994","journal-title":"Biometrika"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1063\/1.1530990","article-title":"Regularities unseen, randomness observed: Levels of entropy convergence","volume":"13","author":"Crutchfield","year":"2003","journal-title":"Chaos Interdiscip. J. Nonlinear Sci."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1198\/jasa.2010.tm09426","article-title":"Dimension reduction and adaptation in conditional density estimation","volume":"105","author":"Efromovich","year":"2010","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_45","unstructured":"Lahiri, S.N. (2013). Resampling Methods for Dependent Data, Springer Science & Business Media."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2297","DOI":"10.1073\/pnas.88.6.2297","article-title":"Approximate entropy as a measure of system complexity","volume":"88","author":"Pincus","year":"1991","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"H2039","DOI":"10.1152\/ajpheart.2000.278.6.H2039","article-title":"Physiological time-series analysis using approximate entropy and sample entropy","volume":"278","author":"Richman","year":"2000","journal-title":"Am. J. Physiol. Heart Circ. Physiol."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"4273","DOI":"10.1109\/TIT.2012.2192713","article-title":"Conditional r\u00e9nyi entropies","volume":"58","author":"Teixeira","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"R789","DOI":"10.1152\/ajpregu.00069.2002","article-title":"Sample entropy analysis of neonatal heart rate variability","volume":"283","author":"Lake","year":"2002","journal-title":"Am. J. Physiol. Regul. Integr. Comp. Physiol."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1016\/S0076-6879(09)67020-8","article-title":"Nonparametric entropy estimation using kernel densities","volume":"467","author":"Lake","year":"2009","journal-title":"Methods Enzymol."},{"key":"ref_51","unstructured":"Lake, D.E. (September, January 30). Improved entropy rate estimation in physiological data. Proceedings of the 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"197","DOI":"10.2307\/3315450","article-title":"Gaussian-based kernels","volume":"18","author":"Wand","year":"1990","journal-title":"Can. J. Stat."},{"key":"ref_53","first-page":"63","article-title":"Reconstruction of integrate-and-fire dynamics","volume":"11","author":"Sauer","year":"1997","journal-title":"Nonlinear Dyn. Time Ser."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1007\/s10439-012-0668-3","article-title":"The appropriate use of approximate entropy and sample entropy with short data sets","volume":"41","author":"Yentes","year":"2013","journal-title":"Ann. Biomed. Eng."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1007\/BF01053745","article-title":"Embedology","volume":"65","author":"Sauer","year":"1991","journal-title":"J. Stat. Phys."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/0167-7152(88)90050-8","article-title":"Canonical kernels for density estimation","volume":"7","author":"Marron","year":"1988","journal-title":"Stat. Probab. Lett."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1007\/s11517-006-0119-0","article-title":"Heart rate variability: A review","volume":"44","author":"Acharya","year":"2006","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1111\/j.1469-8986.1997.tb02140.x","article-title":"Heart rate variability: Origins, methods, and interpretive caveats","volume":"34","author":"Berntson","year":"1997","journal-title":"Psychophysiology"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Billman, G.E. (2011). Heart rate variability\u2014A historical perspective. Front. Physiol., 2.","DOI":"10.3389\/fphys.2011.00086"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1098\/rsta.2008.0232","article-title":"Methods derived from nonlinear dynamics for analysing heart rate variability","volume":"367","author":"Voss","year":"2009","journal-title":"Philos. Trans. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1109\/TBME.1984.325351","article-title":"Comparing spectra of a series of point events particularly for heart rate variability data","volume":"4","author":"Deboer","year":"1984","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1016\/j.cmpb.2013.07.024","article-title":"Kubios HRV\u2014Heart rate variability analysis software","volume":"113","author":"Tarvainen","year":"2014","journal-title":"Comput. Methods Progr. Biomed."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/10.43620","article-title":"A comparison of the noise sensitivity of nine QRS detection algorithms","volume":"37","author":"Friesen","year":"1990","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1016\/j.clinph.2005.01.010","article-title":"Sympathetic and parasympathetic activities evaluated by heart-rate variability in head injury of various severities","volume":"116","author":"Su","year":"2005","journal-title":"Clin. Neurophysiol."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/j.jcrc.2007.04.006","article-title":"Investigation of heart rate and blood pressure variability, baroreflex sensitivity, and approximate entropy in acute brain injury patients","volume":"23","author":"Papaioannou","year":"2008","journal-title":"J. Crit. Care"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Tanizaki, H. (1996). Nonlinear Filters: Estimation and Applications, Springer Science & Business Media.","DOI":"10.1007\/978-3-662-03223-7"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Zuo, K., Bellanger, J.J., Yang, C., Shu, H., and Le Jeannes, R.B. (2013, January 3\u20137). Exploring neural directed interactions with transfer entropy based on an adaptive kernel density estimator. Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan.","DOI":"10.1109\/EMBC.2013.6610507"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/0165-1684(94)90029-9","article-title":"Independent component analysis, a new concept?","volume":"36","author":"Comon","year":"1994","journal-title":"Signal Process."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"068102","DOI":"10.1103\/PhysRevLett.89.068102","article-title":"Multiscale entropy analysis of complex physiologic time series","volume":"89","author":"Costa","year":"2002","journal-title":"Phys. Rev. Lett."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"H424","DOI":"10.1152\/ajpheart.00482.2003","article-title":"A point-process model of human heartbeat intervals: New definitions of heart rate and heart rate variability","volume":"288","author":"Barbieri","year":"2004","journal-title":"AJP Heart Circ. Physiol."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"1335","DOI":"10.1109\/TBME.2010.2041002","article-title":"Characterizing nonlinear heartbeat dynamics within a point process framework","volume":"57","author":"Chen","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"2914","DOI":"10.1109\/TSP.2013.2253775","article-title":"Point-process nonlinear models with laguerre and volterra expansions: Instantaneous assessment of heartbeat dynamics","volume":"61","author":"Valenza","year":"2013","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"052803","DOI":"10.1103\/PhysRevE.89.052803","article-title":"Inhomogeneous point-process entropy: An instantaneous measure of complexity in discrete systems","volume":"89","author":"Valenza","year":"2014","journal-title":"Phys. Rev. E"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1103\/PhysRevLett.85.461","article-title":"Measuring Information Transfer","volume":"85","author":"Schreiber","year":"2000","journal-title":"Phys. Rev. Lett."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.physd.2013.07.001","article-title":"Causation entropy identifies indirect influences, dominance of neighbors and anticipatory couplings","volume":"267","author":"Sun","year":"2014","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_76","unstructured":"Bell, A.J. (2003, January 1\u20134). The co-information lattice. Proceedings of the Fifth International Workshop on Independent Component Analysis and Blind Signal Separation: ICA, Nara, Japan."},{"key":"ref_77","unstructured":"Kandasamy, K., Krishnamurthy, A., Poczos, B., Wasserman, L., and Robins, J.M. (2014). Influence Functions for Machine Learning: Nonparametric Estimators for Entropies, Divergences and Mutual Informations."},{"key":"ref_78","unstructured":"Darmon, D. spenra GitHub Repository. Available online: http:\/\/github.com\/ddarmon\/spenra."},{"key":"ref_79","unstructured":"Kandasamy, K., Krishnamurthy, A., and Poczos, B. (2015). Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/5\/190\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:24:08Z","timestamp":1760210648000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/18\/5\/190"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,19]]},"references-count":79,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2016,5]]}},"alternative-id":["e18050190"],"URL":"https:\/\/doi.org\/10.3390\/e18050190","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,5,19]]}}}