{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:16:19Z","timestamp":1781543779654,"version":"3.54.5"},"reference-count":33,"publisher":"MIT Press","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Linguistics"],"published-print":{"date-parts":[[2015,9]]},"abstract":"<jats:p>This article presents a mathematical and empirical verification of computational constancy measures for natural language text. A constancy measure characterizes a given text by having an invariant value for any size larger than a certain amount. The study of such measures has a 70-year history dating back to Yule's K, with the original intended application of author identification. We examine various measures proposed since Yule and reconsider reports made so far, thus overviewing the study of constancy measures. We then explain how K is essentially equivalent to an approximation of the second-order R\u00e9nyi entropy, thus indicating its signification within language science. We then empirically examine constancy measure candidates within this new, broader context. The approximated higher-order entropy exhibits stable convergence across different languages and kinds of text. We also show, however, that it cannot identify authors, contrary to Yule's intention. Lastly, we apply K to two unknown scripts, the Voynich manuscript and Rongorongo, and show how the results support previous hypotheses about these scripts.<\/jats:p>","DOI":"10.1162\/coli_a_00228","type":"journal-article","created":{"date-parts":[[2015,7,20]],"date-time":"2015-07-20T18:19:23Z","timestamp":1437416363000},"page":"481-502","source":"Crossref","is-referenced-by-count":36,"title":["Computational Constancy Measures of Texts\u2014Yule's<i>K<\/i>and R\u00e9nyi's Entropy"],"prefix":"10.1162","volume":"41","author":[{"given":"Kumiko","family":"Tanaka-Ishii","sequence":"first","affiliation":[{"name":"Kyushu University and JST-PRESTO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shunsuke","family":"Aihara","sequence":"additional","affiliation":[{"name":"Gunosy Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","reference":[{"key":"R3","unstructured":"Bromiley, P. A., N. A. Thacker, and E. Bouhova-Thacker. 2010. Shannon entropy, Renyi entropy, and information. Available at http:\/\/www.tina-vision.net\/docs\/memos\/2004-004.pdf. Accessed June 2015."},{"key":"R4","unstructured":"Brown, P. F., S. A. Della Pietra, V. J. Della Pietra, J. C. Lai, and R. L. Mercer. 1992. An estimate of an upper bound for the entropy of English. Computational Linguistics, 18(1):31\u201340."},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1978.1055912"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1016\/j.spl.2009.01.016"},{"key":"R10","unstructured":"Dbowski, \u0141. 2013. Empirical evidence for Hilberg's conjecture in single author texts. In Methods and Applications of Quantitative Linguistics: Selected papers of the 8th International Conference on Quantitative Linguistics (Qualico), pages 143\u2013151, Belgrade."},{"key":"R12","unstructured":"Dugast, D. 1979. Vocabulaire et Stylistique. I Th\u00e9\u00e2tre et Dialogue. Slatkine-Champion. Travaux de Linguistique Quantitative."},{"key":"R13","unstructured":"Farach, M., M. Noordewier, S. Savari, L. Shepp, A. Wyner, and J. Ziv. 1995. On the entropy of DNA: Algorithms and measurements based on memory and rapid convergence. In Proceedings of the Sixth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 48\u201357, San Francisco, CA."},{"key":"R14","doi-asserted-by":"crossref","unstructured":"Genzel, D. and E. Charniak. 2002. Entropy rate constancy in text. In Annual Meeting of the Association for the ACL, pages 199\u2013206, Philadelphia, PA.","DOI":"10.3115\/1073083.1073117"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/40.3-4.237"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1109\/18.30993"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.1515\/FREQ.1990.44.9-10.243"},{"key":"R22","unstructured":"Honor\u00e9, A. 1979. Some simple measures of richness of vocabulary. Association for Literary and Linguistic Computing Bulletin, 7:172\u2013177."},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.5715\/jnlp.18.119"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.5715\/jnlp.21.877"},{"key":"R26","doi-asserted-by":"crossref","unstructured":"Levy, R. and T. F. Jaeger. 2007. Speakers optimize information density through information density through syntactic reduction. In Annual Conference on Neural Information Processing Systems, pages 1\u20138, Vancouver.","DOI":"10.7551\/mitpress\/7503.003.0111"},{"key":"R27","unstructured":"Maas, H. D. 1972. Zusammenhang zwischen wortschatzumfang und l\u00e4nge eines textes [Relationship between vocabulary and text length]. Zeitschrift f\u00fcr Literaturwissenschaft und Linguistik, 8:73\u201370."},{"key":"R28","unstructured":"Mandelbrot, B. 1953. An informational theory of the statistical structure of language. Communication Theory, 486\u2013500."},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0066344"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.1002\/j.1834-4453.2005.tb00597.x"},{"key":"R32","unstructured":"Orlov, J. K. and R. Y. Chitashvili. 1983. Generalized z-distribution generating the well-known \u2018rank-distributions\u2019. Bulletin of the Academy of Sciences of Georgia, 110:269\u2013272."},{"key":"R33","unstructured":"Pozdniakov, K. and I. Pozdniakov. 2007. Rapanui writing and the Rapanui language: Preliminary results of a statistical analysis. Forum for Anthropology and Culture, 3:3\u201336."},{"key":"R34","unstructured":"Reddy, S. and K. Knight. 2011. What we know about the Voynich Manuscript. In ACL Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, Portland, OR."},{"key":"R35","unstructured":"R\u00e9nyi, A. 1960. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematics, Statistics and Probability, pages 547\u2013561, Berkeley, CA."},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1063\/1.166191"},{"key":"R38","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.1214\/aop\/1176989934"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.2307\/2285930"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1038\/163688a0"},{"key":"R42","doi-asserted-by":"publisher","DOI":"10.1162\/089120100750105920"},{"key":"R43","doi-asserted-by":"publisher","DOI":"10.1007\/s10579-010-9115-y"},{"key":"R44","doi-asserted-by":"publisher","DOI":"10.3115\/1220175.1220299"},{"key":"R45","unstructured":"Tuldava, J. 1977. Quantitative relations between the size of the text and lexical richness. SMIL Quarterly, Journal of Linguistic Calculus, 4:28\u201335."},{"key":"R46","doi-asserted-by":"publisher","DOI":"10.1023\/A:1001749303137"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/COLI_a_00228","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,12]],"date-time":"2023-08-12T10:24:56Z","timestamp":1691835896000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/41\/3\/481-502\/1519"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,9]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2015,9]]}},"alternative-id":["10.1162\/COLI_a_00228"],"URL":"https:\/\/doi.org\/10.1162\/coli_a_00228","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,9]]}}}