{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T22:11:10Z","timestamp":1766268670732,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2018,1,26]],"date-time":"2018-01-26T00:00:00Z","timestamp":1516924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As we discuss, a stationary stochastic process is nonergodic when a random persistent topic can be detected in the infinite random text sampled from the process, whereas we call the process strongly nonergodic when an infinite sequence of independent random bits, called probabilistic facts, is needed to describe this topic completely. Replacing probabilistic facts with an algorithmically random sequence of bits, called algorithmic facts, we adapt this property back to ergodic processes. Subsequently, we call a process perigraphic if the number of algorithmic facts which can be inferred from a finite text sampled from the process grows like a power of the text length. We present a simple example of such a process. Moreover, we demonstrate an assertion which we call the theorem about facts and words. This proposition states that the number of probabilistic or algorithmic facts which can be inferred from a text drawn from a process must be roughly smaller than the number of distinct word-like strings detected in this text by means of the Prediction by Partial Matching (PPM) compression algorithm. We also observe that the number of the word-like strings for a sample of plays by Shakespeare follows an empirical stepwise power law, in a stark contrast to Markov processes. Hence, we suppose that natural language considered as a process is not only non-Markov but also perigraphic.<\/jats:p>","DOI":"10.3390\/e20020085","type":"journal-article","created":{"date-parts":[[2018,1,29]],"date-time":"2018-01-29T07:46:20Z","timestamp":1517211980000},"page":"85","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Is Natural Language a Perigraphic Process? The Theorem about Facts and Words Revisited"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7136-5283","authenticated-orcid":false,"given":"\u0141ukasz","family":"D\u0119bowski","sequence":"first","affiliation":[{"name":"Institute of Computer Science, Polish Academy of Sciences, ul. Jana Kazimierza 5, 01-248 Warszawa, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,1,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"30","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1002\/j.1538-7305.1951.tb01366.x","article-title":"Prediction and entropy of printed English","volume":"30","author":"Shannon","year":"1951","journal-title":"Bell Syst. Tech. J."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley. [2nd ed.].","DOI":"10.1002\/047174882X"},{"key":"ref_4","unstructured":"Jelinek, F. (1997). Statistical Methods for Speech Recognition, The MIT Press."},{"key":"ref_5","unstructured":"Manning, C.D., and Sch\u00fctze, H. (1999). Foundations of Statistical Natural Language Processing, The MIT Press."},{"key":"ref_6","first-page":"173","article-title":"Twice-universal coding","volume":"20","author":"Ryabko","year":"1984","journal-title":"Probl. Inf. Transm."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1109\/TCOM.1984.1096090","article-title":"Data compression using adaptive coding and partial string matching","volume":"32","author":"Cleary","year":"1984","journal-title":"IEEE Trans. Commun."},{"key":"ref_8","unstructured":"Zipf, G.K. (1965). The Psycho-Biology of Language: An Introduction to Dynamic Philology, The MIT Press. [2nd ed.]."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/00437956.1954.11659509","article-title":"Structure formelle des textes et communication","volume":"10","author":"Mandelbrot","year":"1954","journal-title":"Word"},{"key":"ref_10","first-page":"168","article-title":"The number of different words as a function of text length","volume":"42","author":"Kuraszkiewicz","year":"1951","journal-title":"Pami\u0119t. Lit."},{"key":"ref_11","unstructured":"Guiraud, P. (1954). Les Caract\u00e8res Statistiques du Vocabulaire, Presses Universitaires de France."},{"key":"ref_12","unstructured":"Herdan, G. (1964). Quantitative Linguistics, Butterworths."},{"key":"ref_13","unstructured":"Heaps, H.S. (1978). Information Retrieval\u2014Computational and Theoretical Aspects, Academic Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1515\/FREQ.1990.44.9-10.243","article-title":"Der bekannte Grenzwert der redundanzfreien Information in Texten\u2014Eine Fehlinterpretation der Shannonschen Experimente?","volume":"44","author":"Hilberg","year":"1990","journal-title":"Frequenz"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1209\/0295-5075\/14\/3\/001","article-title":"Entropy of Symbolic Sequences: The Role of Correlations","volume":"14","author":"Ebeling","year":"1991","journal-title":"Europhys. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1209\/0295-5075\/26\/4\/001","article-title":"Entropy and long-range correlations in literary English","volume":"26","author":"Ebeling","year":"1994","journal-title":"Europhys. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1016\/S0378-4371(01)00444-7","article-title":"Complexity through nonextensivity","volume":"302","author":"Bialek","year":"2001","journal-title":"Physica A"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1063\/1.1530990","article-title":"Regularities unseen, randomness observed: The entropy convergence hierarchy","volume":"15","author":"Crutchfield","year":"2003","journal-title":"Chaos"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1080\/09296170500500637","article-title":"On Hilberg\u2019s law and its links with Guiraud\u2019s law","volume":"13","year":"2006","journal-title":"J. Quant. Linguist."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1177\/002383098002300303","article-title":"Language acquisition and the discovery of phrase structure","volume":"23","author":"Wolff","year":"1980","journal-title":"Lang. Speech"},{"key":"ref_21","unstructured":"De Marcken, C.G. (1996). Unsupervised Language Acquisition. [Ph.D. Thesis, Massachussetts Institute of Technology]."},{"key":"ref_22","unstructured":"Osborne, M., and Sang, E.T.K. (1999). Unsupervised Learning of Word Boundary with Description Length Gain. Proceedings of the Computational Natural Language Learning ACL Workshop, Bergen, The Association for Computational Linguistics."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1109\/18.841160","article-title":"Grammar-based codes: A new class of universal lossless source codes","volume":"46","author":"Kieffer","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1260","DOI":"10.1016\/j.spl.2009.01.016","article-title":"A general definition of conditional information and its application to ergodic decomposition","volume":"79","year":"2009","journal-title":"Statist. Probab. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4589","DOI":"10.1109\/TIT.2011.2145170","article-title":"On the Vocabulary of Grammar-Based Codes and the Logical Consistency of Texts","volume":"57","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1109\/TIT.2005.850116","article-title":"The Smallest Grammar Problem","volume":"51","author":"Charikar","year":"2005","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"037105","DOI":"10.1063\/1.3630929","article-title":"Excess entropy in natural language: present state and perspectives","volume":"21","year":"2011","journal-title":"Chaos"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1080\/09296174.2015.1106268","article-title":"The Relaxed Hilberg Conjecture: A Review and New Experimental Support","volume":"22","year":"2015","journal-title":"J. Quantit. Linguist."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3392","DOI":"10.1109\/TIT.2012.2190708","article-title":"Mixing, Ergodic, and Nonergodic Processes with Rapidly Growing Information between Blocks","volume":"58","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_30","unstructured":"Billingsley, P. (1979). Probability and Measure, Wiley."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gray, R.M. (2009). Probability, Random Processes, and Ergodic Properties, Springer.","DOI":"10.1007\/978-1-4419-1090-5"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Breiman, L. (1992). Probability, SIAM.","DOI":"10.1137\/1.9781611971286"},{"key":"ref_33","unstructured":"Kallenberg, O. (1997). Foundations of Modern Probability, Springer."},{"key":"ref_34","unstructured":"Yaglom, A.M., and Yaglom, I.M. (1983). Probability and Information. Theory and Decision Library, Springer."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1109\/TIT.1974.1055287","article-title":"The ergodic decomposition of stationary discrete random processses","volume":"20","author":"Gray","year":"1974","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"5716","DOI":"10.1109\/TIT.2015.2470675","article-title":"Hilberg Exponents: New Measures of Long Memory in the Process","volume":"61","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, M., and Vit\u00e1nyi, P.M.B. (2008). An Introduction to Kolmogorov Complexity and Its Applications, Springer. [3rd ed.].","DOI":"10.1007\/978-0-387-49820-1"},{"key":"ref_38","unstructured":"Barron, A.R. (1985). Logically Smooth Density Estimation. [Ph.D. Thesis, Stanford University]."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Woungang, I., Misra, S., and Misra, S.C. (2010). Applications of Universal Source Coding to Statistical Analysis of Time Series. Selected Topics in Information and Coding Theory, World Scientific Publishing.","DOI":"10.1142\/9789812837172"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/S0304-3975(98)00248-5","article-title":"On the combinatorics of finite words","volume":"218","year":"1999","journal-title":"Theor. Comput. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"5903","DOI":"10.3390\/e17085903","article-title":"Maximal Repetitions in Written Texts: Finite Energy Hypothesis vs. Strong Hilberg Conjecture","volume":"17","year":"2015","journal-title":"Entropy"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/2\/85\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:52:42Z","timestamp":1760194362000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/2\/85"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1,26]]},"references-count":41,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2018,2]]}},"alternative-id":["e20020085"],"URL":"https:\/\/doi.org\/10.3390\/e20020085","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,1,26]]}}}