{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T16:39:53Z","timestamp":1780677593277,"version":"3.54.1"},"reference-count":56,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2015,1,19]],"date-time":"2015-01-19T00:00:00Z","timestamp":1421625600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Information-theoretic quantities, such as entropy and mutual information (MI), can be used to quantify the amount of information needed to describe a dataset or the information shared between two datasets. In the case of a dynamical system, the behavior of the relevant variables can be tightly coupled, such that information about one variable at a given instance in time may provide information about other variables at later instances in time. This is often viewed as a flow of information, and tracking such a flow can reveal relationships among the system variables. Since the MI is a symmetric quantity; an asymmetric quantity, called Transfer Entropy (TE), has been proposed to estimate the directionality of the coupling. However, accurate estimation of entropy-based measures is notoriously difficult. Every method has its own free tuning parameter(s) and there is no consensus on an optimal way of estimating the TE from a dataset. We propose a new methodology to estimate TE and apply a set of methods together as an accuracy cross-check to provide a reliable mathematical tool for any given data set. We demonstrate both the variability in TE estimation across techniques as well as the benefits of the proposed methodology to reliably estimate the directionality of coupling among variables.     <\/jats:p>","DOI":"10.3390\/e17010438","type":"journal-article","created":{"date-parts":[[2015,1,19]],"date-time":"2015-01-19T10:53:54Z","timestamp":1421664834000},"page":"438-470","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":55,"title":["A Recipe for the Estimation of Information Flow in a  Dynamical System"],"prefix":"10.3390","volume":"17","author":[{"given":"Deniz","family":"Gencaga","sequence":"first","affiliation":[{"name":"NOAA-CREST, The City College of New York, New York, NY, 10031, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7702-3807","authenticated-orcid":false,"given":"Kevin","family":"Knuth","sequence":"additional","affiliation":[{"name":"Depts. of Physics and Informatics, University at Albany (SUNY), Albany, NY 12222, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Rossow","sequence":"additional","affiliation":[{"name":"NOAA-CREST, The City College of New York, New York, NY, 10031, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2015,1,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2533","DOI":"10.1152\/jn.01106.2006","article-title":"Evaluating information transfer between auditory cortical neurons","volume":"97","author":"Eggermont","year":"2007","journal-title":"J. Neurophysiol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1016\/j.jsv.2008.11.025","article-title":"Dynamic system change detection using a modification of the transfer entropy","volume":"322","author":"Overbey","year":"2009","journal-title":"J. Sound Vibr."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2178","DOI":"10.1016\/j.ymssp.2009.03.016","article-title":"Effects of noise on transfer entropy estimation for damage detection","volume":"23","author":"Overbey","year":"2009","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_4","unstructured":"Pearl, J. (2000). Causality: Models, Reasoning and Inference, MIT Press."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1002\/joc.1499","article-title":"Empirical orthogonal functions and related techniques in atmospheric science: A review","volume":"27","author":"Hannachi","year":"2007","journal-title":"Int. J. Climatol."},{"key":"ref_6","first-page":"265","article-title":"Information theoretic learning","volume":"1","author":"Principe","year":"2000","journal-title":"Unsuperv. Adapt. Filter."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1103\/PhysRevLett.85.461","article-title":"Measuring information transfer","volume":"85","author":"Schreiber","year":"2000","journal-title":"Phys. Rev. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"424","DOI":"10.2307\/1912791","article-title":"Investigating causal relations by econometric models and cross-spectral methods","volume":"37","author":"Granger","year":"1969","journal-title":"Econometrica"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"056221","DOI":"10.1103\/PhysRevE.70.056221","article-title":"Radial basis function approach to nonlinear Granger causality of time series","volume":"70","author":"Ancona","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physrep.2006.12.004","article-title":"Causality detection based on information-theoretic approaches in time series analysis","volume":"441","author":"Vejmelka","year":"2007","journal-title":"Phys. Rep."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1140\/epjb\/e2010-00034-5","article-title":"Differentiating information transfer and causal effect","volume":"73","author":"Lizier","year":"2010","journal-title":"Eur. Phys. J. B"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1175\/JAS3857.1","article-title":"Information flow in ensemble weather predictions","volume":"64","author":"Kleeman","year":"2007","journal-title":"J. Atmos. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"327","DOI":"10.3390\/e15010327","article-title":"Information Flow: Theory and Applications","volume":"15","author":"Liang","year":"2013","journal-title":"Entropy"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1007\/978-0-387-69319-4_24","article-title":"Information flow in coupled nonlinear systems: Application to the epileptic human brain","volume":"7","author":"Sabesan","year":"2007","journal-title":"Data Mining Biomed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"9558","DOI":"10.1073\/pnas.0703499104","article-title":"Information flow between subspaces of complex dynamical systems","volume":"104","author":"Majda","year":"2007","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"244101","DOI":"10.1103\/PhysRevLett.95.244101","article-title":"Information transfer between dynamical system components","volume":"95","author":"Liang","year":"2005","journal-title":"Phys. Rev. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ruddell, B.L., and Kumar, P. (2009). Ecohydrologic process networks: 1. Identification. Water Resour. Res., 45.","DOI":"10.1029\/2008WR007279"},{"key":"ref_18","first-page":"W03420","article-title":"Ecohydrologic process networks: 2. Analysis and characterization","volume":"45","author":"Ruddell","year":"2009","journal-title":"Water Resour. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/S0167-2789(02)00432-3","article-title":"Information transfer in continuous processes","volume":"166","author":"Kaiser","year":"2002","journal-title":"Physica D"},{"key":"ref_20","unstructured":"Knuth, K.H., Gotera, A., Curry, C.T., Huyser, K.A., Wheeler, K.R., and Rossow, W.B. (2005, January 27\u201330). Revealing relationships among relevant climate variables with information theory, Adelphi, MD, USA."},{"key":"ref_21","unstructured":"Shannon, C.E., and Weaver, W. (1949). The Mathematical Theory of Communication, University of Illinois Press."},{"key":"ref_22","unstructured":"Knuth, K.H. Optimal data-based binning for histograms. Available online: http:\/\/knuthlab.rit.albany.edu\/index.php\/Products\/Code & http:\/\/www.astroml.org\/."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6841","DOI":"10.1103\/PhysRevE.52.6841","article-title":"Estimating functions of probability distributions from a finite set of samples","volume":"52","author":"Wolpert","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_24","first-page":"471","article-title":"Bialek13, W. Entropy and Inference, Revisited","volume":"1","author":"Nemenman","year":"2002","journal-title":"Adv. Neur. Inf. Process. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1016\/0167-2789(95)00041-2","article-title":"Generalized redundancies for time series analysis","volume":"84","author":"Prichard","year":"1995","journal-title":"Physica D"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1016\/S0167-2789(98)00269-3","article-title":"Estimating the errors on measured entropy and mutual information","volume":"125","author":"Roulston","year":"1999","journal-title":"Physica D"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1016\/0375-9601(88)90193-4","article-title":"Finite sample corrections to entropy and dimension estimates","volume":"128","author":"Grassberger","year":"1988","journal-title":"Phys. Lett. A"},{"key":"ref_28","unstructured":"Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, Chapman & Hall\/CRC."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"612","DOI":"10.3390\/e13030612","article-title":"Information theory and dynamical system predictability","volume":"13","author":"Kleeman","year":"1986","journal-title":"Entropy"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2057","DOI":"10.1175\/1520-0469(2002)059<2057:MDPUUR>2.0.CO;2","article-title":"Measuring dynamical prediction utility using relative entropy","volume":"59","author":"Kleeman","year":"2002","journal-title":"J. Atmos. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1103\/PhysRevLett.50.346","article-title":"Characterization of strange attractors","volume":"50","author":"Grassberger","year":"1983","journal-title":"Phys. Rev. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/0167-2789(83)90298-1","article-title":"Measuring the strangeness of strange attractors","volume":"9","author":"Grassberger","year":"1983","journal-title":"Physica D"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kantz, H., and Schreiber, T. (2003). Nonlinear time Series Analysis, Cambridge University Press.","DOI":"10.1017\/CBO9780511755798"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1103\/PhysRevA.33.1134","article-title":"Independent coordinates for strange attractors from mutual information","volume":"33","author":"Fraser","year":"1986","journal-title":"Phys. Rev. A"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1315","DOI":"10.1109\/18.761290","article-title":"Estimation of the information by an adaptive partitioning of the observation space","volume":"45","author":"Darbellay","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_37","unstructured":"Gomez-Herrero, G., Wu, W., Rutanen, K., Soriano, M.C., Pipa, G., and Vicente, R. (2010). Assessing coupling dynamics from an ensemble of time series, arXiv:1008.0539."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s10827-010-0262-3","article-title":"Transfer entropy-a model-free measure of effective connectivity for the neurosciences","volume":"30","author":"Vicente","year":"2011","journal-title":"J. Comput. Neurosci."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wibral, M., Pampu, N., Priesemann, V., Siebenh\u00fchner, F., Seiwert, H., Lindner, M., Lizier, J.T., and Vicente, R. (2013). Measuring Information-Transfer Delays. PLoS One, 8.","DOI":"10.1371\/journal.pone.0055809"},{"key":"ref_40","unstructured":"Steeg, G., and Galstyan, V.A. (2013, January 4\u20138). Information-theoretic measures of influence based on content dynamics, Rome, Italy."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1175\/1520-0469(1963)020<0130:DNF>2.0.CO;2","article-title":"Deterministic nonperiodic flow","volume":"20","author":"Lorenz","year":"1963","journal-title":"J. Atmos. Sci."},{"key":"ref_42","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, Wiley."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"026110","DOI":"10.1103\/PhysRevE.77.026110","article-title":"Local information transfer as a spatiotemporal filter for complex systems","volume":"77","author":"Lizier","year":"2008","journal-title":"Phys. Rev. E"},{"key":"ref_44","unstructured":"Williams, P.L., and Beer, R.D. Available online: http:\/\/arxiv.org\/abs\/1102.1507."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1093\/biomet\/66.3.605","article-title":"On optimal and data-based histograms","volume":"66","author":"Scott","year":"1979","journal-title":"Biometrika"},{"key":"ref_46","first-page":"453","article-title":"On the histogram as a density estimator: L 2 theory","volume":"57","author":"Freedman","year":"1981","journal-title":"Probab. Theory Relat. Fields"},{"key":"ref_47","unstructured":"Box, G.E., and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis, Addison-Wesley."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"066208","DOI":"10.1103\/PhysRevE.71.066208","article-title":"Statistical validation of mutual information calculations: Comparison of alternative numerical algorithms","volume":"71","author":"Cellucci","year":"2005","journal-title":"Phys. Rev. E"},{"key":"ref_49","unstructured":"Bertelle, C., Liu, X., and Aziz-Alaoui, M.A. (July, January 29). Improvement of Symbolic Transfer Entropy, Le Havre, France."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"5394","DOI":"10.1038\/srep05394","article-title":"Transfer Entropy and Transient Limits of Computation","volume":"4","author":"Prokopenko","year":"2014","journal-title":"Sci. Rep."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1142\/S0218127407017628","article-title":"Methods for quantifying the causal structure of bivariate time series","volume":"17","author":"Lungarella","year":"2007","journal-title":"Int. J. Bifurc. Chaos"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"3043","DOI":"10.4249\/scholarpedia.3043","article-title":"Grassberger-Procaccia algorithm","volume":"2","author":"Grassberger","year":"2007","journal-title":"Scholarpedia"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.physa.2004.06.144","article-title":"Mutual information: A measure of dependency for nonlinear time series","volume":"344","author":"Dionisio","year":"2004","journal-title":"Physica A"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/0165-1684(89)90132-1","article-title":"On estimation of entropy and mutual information of continuous distributions","volume":"16","author":"Moddemeijer","year":"1989","journal-title":"Signal Process."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"158101","DOI":"10.1103\/PhysRevLett.100.158101","article-title":"Symbolic Transfer Entropy","volume":"100","author":"Staniek","year":"2008","journal-title":"Phys. Rev. Lett."},{"key":"ref_56","unstructured":"Abramowitz, M., and Stegun, I. (1972). Handbook of Mathematical Functions, Dover Publications."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/1\/438\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:41:47Z","timestamp":1760215307000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/1\/438"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,1,19]]},"references-count":56,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2015,1]]}},"alternative-id":["e17010438"],"URL":"https:\/\/doi.org\/10.3390\/e17010438","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,1,19]]}}}