{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:23:13Z","timestamp":1760242993780,"version":"build-2065373602"},"reference-count":23,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2015,4,23]],"date-time":"2015-04-23T00:00:00Z","timestamp":1429747200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we model discrete time series as discrete Markov processes of arbitrary order and derive the approximate distribution of the Kullback-Leibler divergence between a known transition probability matrix and its sample estimate. We introduce two new information-theoretic measurements: information memory loss and information codependence structure. The former measures the memory content within a Markov process and determines its optimal order. The latter assesses the codependence among Markov processes. Both measurements are evaluated on toy examples and applied on high frequency foreign exchange data, focusing on 2008 financial crisis and 2010\/2011 Euro crisis.<\/jats:p>","DOI":"10.3390\/e17052606","type":"journal-article","created":{"date-parts":[[2015,4,23]],"date-time":"2015-04-23T11:40:29Z","timestamp":1429789229000},"page":"2606-2623","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Uncovering Discrete Non-Linear Dependence with Information Theory"],"prefix":"10.3390","volume":"17","author":[{"given":"Anton","family":"Golub","sequence":"first","affiliation":[{"name":"Olsen Ltd, Eierbrechtstrasse 50, 8053 Z\u00fcrich, Switzerland"}]},{"given":"Gregor","family":"Chliamovitch","sequence":"additional","affiliation":[{"name":"Computer Science Department, University of Geneva, rte de Drize 7, 1227 Carouge, Switzerland"}]},{"given":"Alexandre","family":"Dupuis","sequence":"additional","affiliation":[{"name":"Olsen Ltd, Eierbrechtstrasse 50, 8053 Z\u00fcrich, Switzerland"},{"name":"Computer Science Department, University of Geneva, rte de Drize 7, 1227 Carouge, Switzerland"}]},{"given":"Bastien","family":"Chopard","sequence":"additional","affiliation":[{"name":"Computer Science Department, University of Geneva, rte de Drize 7, 1227 Carouge, Switzerland"}]}],"member":"1968","published-online":{"date-parts":[[2015,4,23]]},"reference":[{"key":"ref_1","first-page":"135","article-title":"Rasprostranenie zakona bol\u2019shih chisel na velichiny zavisgaschie drug ot druga","volume":"15","author":"Markov","year":"1906","journal-title":"Izvestiya Fiziko-matematicheskogo obschhestra pri Kazanskom universitete, 2-ya seriya"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Levin, D.A., Peres, Y., and Wilmer, E.L. (2008). Markov Chains and Mixing Time, American Mathematical Society.","DOI":"10.1090\/mbk\/058"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1063\/1.1699114","article-title":"Equations of State Calculations by Fast Computing Machines","volume":"21","author":"Metropolis","year":"1953","journal-title":"J. Chem. Phys"},{"key":"ref_4","unstructured":"Rabiner, L.R. Available online: http:\/\/ethw.org\/First-Hand:The_Hidden_Markov_Model."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"357","DOI":"10.2307\/1912559","article-title":"A new approach to the economic analysis of nonstationary time series and the business cycle","volume":"57","author":"Hamilton","year":"1989","journal-title":"Econometrica"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"488","DOI":"10.2307\/3212863","article-title":"Determination of the order of Markov Chain by Akaikes Information Criterion","volume":"12","author":"Tong","year":"1975","journal-title":"J. Appl. Probab"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1093\/biomet\/68.1.45","article-title":"An optimal selection of regression variables","volume":"68","author":"Katz","year":"1981","journal-title":"Biometrika"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","article-title":"A new look at the statistical mode indentification","volume":"19","author":"Akaike","year":"1974","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_9","first-page":"461","article-title":"Estimating the dimension of a model","volume":"6","author":"Schwartz","year":"1978","journal-title":"Ann. Stat"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1214\/aoms\/1177707039","article-title":"Statistical Inference about Markov chains","volume":"28","author":"Anderson","year":"1957","journal-title":"Ann. Math. Stat"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1017\/S0305004100026402","article-title":"The frequency goodness of fit test for probability chains","volume":"47","author":"Bartlett","year":"1951","journal-title":"Proc. Camb. Philos. Soc"},{"key":"ref_12","first-page":"187","article-title":"A multivariate Markov Chain model for categorical data sequences and its applications in demand predictions","volume":"13","author":"Ching","year":"2002","journal-title":"IMA J. Manag. Math"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1142\/S0129065705000256","article-title":"On Construction of Stochastic Genetic Networks Based on Gene Expression Sequences","volume":"15","author":"Ching","year":"2005","journal-title":"Int. J. Neural Syst"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1080\/14697680500383714","article-title":"On multivariate credibility approach for portfolio credit risk measurement","volume":"5","author":"Siu","year":"2005","journal-title":"Quant. Finance"},{"key":"ref_15","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley."},{"key":"ref_16","unstructured":"Feller, W. (1968). An Introduction to Probability Theory and Its Applications, Wiley."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1109\/TIT.2004.826687","article-title":"The Kullback-Leibler divergence rate between Markov Sources","volume":"50","author":"Rached","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"L\u00e9vy-V\u00e9hel, J., and Lutton, E. (2005). Fractals in Engineering, Springer.","DOI":"10.1007\/b137729"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1080\/07350015.1999.10524830","article-title":"Nonlinear Predictability of Stock Returns Using Financial and Economic Variables","volume":"17","author":"Min","year":"1999","journal-title":"J. Bus. Econ. Stat"},{"key":"ref_20","first-page":"291","article-title":"Comovements in Stock Prices in the Very Short Run","volume":"74","author":"Epps","year":"1979","journal-title":"J. Am. Stat. Assoc"},{"key":"ref_21","first-page":"4828","article-title":"Impact of the tick-size on financial returns and correlations","volume":"389","author":"Guhr","year":"2014","journal-title":"Physica A"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"053040","DOI":"10.1088\/1367-2630\/16\/5\/053040","article-title":"Modelling the short term herding behavior of stock markets","volume":"16","author":"Shapira","year":"2014","journal-title":"New J. Phys"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1080\/14697688.2010.481632","article-title":"Patterns in high-frequency FX data: discovery of 12 empirical scaling laws","volume":"11","author":"Glattfelder","year":"2011","journal-title":"Quant. Finance"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/2606\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:45:14Z","timestamp":1760215514000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/2606"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,4,23]]},"references-count":23,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,5]]}},"alternative-id":["e17052606"],"URL":"https:\/\/doi.org\/10.3390\/e17052606","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2015,4,23]]}}}