{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T13:39:37Z","timestamp":1770817177477,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2013,6,5]],"date-time":"2013-06-05T00:00:00Z","timestamp":1370390400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We characterize the statistical bootstrap for the estimation of informationtheoretic quantities from data, with particular reference to its use in the study of large-scale social phenomena. Our methods allow one to preserve, approximately, the underlying axiomatic relationships of information theory\u2014in particular, consistency under arbitrary coarse-graining\u2014that motivate use of these quantities in the first place, while providing reliability comparable to the state of the art for Bayesian estimators. We show how information-theoretic quantities allow for rigorous empirical study of the decision-making capacities of rational agents, and the time-asymmetric flows of information in distributed systems. We provide illustrative examples by reference to ongoing collaborative work on the semantic structure of the British Criminal Court system and the conflict dynamics of the contemporary Afghanistan insurgency.<\/jats:p>","DOI":"10.3390\/e15062246","type":"journal-article","created":{"date-parts":[[2013,6,5]],"date-time":"2013-06-05T12:40:57Z","timestamp":1370436057000},"page":"2246-2276","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["Bootstrap Methods for the Empirical Study of Decision-Making and Information Flows in Social Systems"],"prefix":"10.3390","volume":"15","author":[{"given":"Simon","family":"DeDeo","sequence":"first","affiliation":[{"name":"Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Hawkins","sequence":"additional","affiliation":[{"name":"Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA"},{"name":"Cognitive Science Program, 819 Eigenmann, 1900 E. 10th St., Indiana University, Bloomington, IN 47406, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sara","family":"Klingenstein","sequence":"additional","affiliation":[{"name":"Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tim","family":"Hitchcock","sequence":"additional","affiliation":[{"name":"Department of History, University of Hertfordshire, Hatfield, Hertfordshire, AL10 9AB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2013,6,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1162\/089976603321780272","article-title":"Estimation of entropy and mutual information","volume":"15","author":"Paninski","year":"2003","journal-title":"Neural Comput."},{"key":"ref_3","unstructured":"Collaboration \u201cInstitution Formation, Semantic Phenomena, and Three Hundred Years of the British Criminal Court System\u201d, with participants Simon DeDeo, Sara Klingenstein, and Tim Hitchcock."},{"key":"ref_4","unstructured":"This is not exactly true, since we use part-of-speech information for our semantic classifier, and the means (a hidden Markov model) for identifying a word\u2019s lexical category are sensitive to order and context: compare \u201cthe dogs bit the sailor\u201d [\u2018dogs\u2019 as plural noun] and \u201cthe sailor dogs the waitress\u201d [\u2018dogs\u2019 as verb]. It is an interesting and open question the extent to which syntactical features convey semantic information."},{"key":"ref_5","unstructured":"It is possible to correct the magic word problem post hoc, merging categories until no magic words remain. If this rule is specified with sufficient generality, it allows for bootstrap estimation. We do not consider this approach here."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_7","unstructured":"Nielsen, F. (2010). A family of statistical symmetric divergences based on Jensen\u2019s inequality. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1109\/TIT.1970.1054466","article-title":"Probability of error, equivocation, and the Chernoff bound","volume":"16","author":"Hellman","year":"1970","journal-title":"IEEE Tran. Inform. Theory"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Jaynes, E., and Bretthorst, G. (2003). Probability Theory: The Logic of Science, Cambridge University Press.","DOI":"10.1017\/CBO9780511790423"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cover, T., and Thomas, J. (2006). Elements of Information Theory, Wiley.","DOI":"10.1002\/047174882X"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/S0020-0255(76)90746-5","article-title":"On information and distance measures, error bounds, and feature selection","volume":"10","author":"Chen","year":"1976","journal-title":"Inform. Sci."},{"key":"ref_12","first-page":"369","article-title":"Approximate error bounds in pattern recognition","volume":"7","author":"Ito","year":"1972","journal-title":"Mach. Intell."},{"key":"ref_13","first-page":"220","article-title":"A tight upper bound on the Bayesian probability of error","volume":"16","author":"Hashlamoun","year":"1994","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_14","unstructured":"We can estimate \u03c1 itself by means of the statistical bootstrap. Resampling suggests that \u03c1 itself may suffer from bias which can be corrected for."},{"key":"ref_15","unstructured":"Consider, for example, the case where the supports of p1 and p2 only partially overlap: p1(x) is zero for some x, but p2(x) is non-zero, and vice versa for some y. The draw {x,y} is not possible for either p1 or p2, but is possible for the average."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"472","DOI":"10.2747\/1539-7216.51.4.472","article-title":"Peering into the Fog of War: The Geography of the WikiLeaks Afghanistan War Logs 2004\u20132009","volume":"51","author":"Witmer","year":"2010","journal-title":"Eurasian Geogr. Econ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"12414","DOI":"10.1073\/pnas.1203177109","article-title":"Point process modelling of the Afghan War Diary","volume":"109","author":"Dewar","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_18","unstructured":"There are more than 78,000 SIGACTs in the original data set. We removed approximately 20,000 because they expressed a level of ambiguity about what happened (\u201cunknown-initiated action\u201d or \u201csuspicious incident\u201d), because their GPS coordinates did not match the description, because they were not time-sensitive (\u201cweapons cache found\u201d) or because they were irrelevant to our study (\u201cIED hoax\u201d or \u201cshow of force\u201d). Information on initiative is explicitly included in the SIGACT data and is extracted by the methods of Ref. [16]."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1080\/1057610X.2010.501425","article-title":"Networks and Armies: Structuring Rebellion in Colombia and Afghanistan","volume":"33","author":"Giustozzi","year":"2010","journal-title":"Stud. Confl. Terror."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0032329207312181","article-title":"Telling the Difference: Guerrillas and Paramilitaries in the Colombian War","volume":"36","year":"2008","journal-title":"Polit. Soc."},{"key":"ref_21","unstructured":"Green, A.H. (2011). Repertoires of Violence Against Non-combatants: The Role of Armed Group Institutions and Ideologies. [Ph.D. Thesis, Yale University]."},{"key":"ref_22","unstructured":"Collaboration \u201cThe Emergence of Insurgency in Afghanistan: An Information Theoretic Analysis\u201d, with participants Simon DeDeo and Robert Hawkins."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hutter, M. (2001). Distribution of Mutual Information. arXiv.","DOI":"10.7551\/mitpress\/1120.003.0056"},{"key":"ref_24","unstructured":"Zaffalon, M., and Hutter, M. (2002, January 1\u20134). Robust Feature Selection by Mutual Information Distributions. Proceedings of the 18th Conference in Uncertainty in Artificial Intelligence, Edmonton, Canada."},{"key":"ref_25","unstructured":"Williams, P.L., and Beer, R.D. (2011). Generalized Measures of Information Transfer. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1103\/PhysRevLett.85.461","article-title":"Measuring Information Transfer","volume":"85","author":"Schreiber","year":"2000","journal-title":"Phys. Rev. Lett."},{"key":"ref_27","unstructured":"THOTH. Available online: http:\/\/thoth-python.org."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dietterich, T.G., Becker, S., and Ghahramani, Z. (2002). Advances in Neural Information Processing Systems 14, MIT Press.","DOI":"10.7551\/mitpress\/1120.001.0001"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"56111:1","DOI":"10.1103\/PhysRevE.69.056111","article-title":"Entropy and information in neural spike trains: Progress on the sampling problem","volume":"69","author":"Nemenman","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"6841","DOI":"10.1103\/PhysRevE.52.6841","article-title":"Estimating functions of probability distributions from a finite set of samples","volume":"52","author":"Wolpert","year":"1995","journal-title":"Phys. Rev. E"},{"key":"ref_31","unstructured":"Nijenhuis, A., and Wilf, H.S. (1978). Combinatorial Algorithms for Computers and Calculators, Academic Press. [2nd ed.]."},{"key":"ref_32","unstructured":"Care needs to be taken, however, since a general stochastic function of the underlying process will not, in general, preserve independence in the empirical distribution. It is always possible, for example, to choose an instantiation of a stochastic re-mapping post hoc that magnifies accidental correlations found in the original empirical distribution. We thank John Geanakoplos for pointing this out to us."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"DeDeo, S. (2011). Effective theories for circuits and automata. Chaos, 21.","DOI":"10.1063\/1.3640747"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3311","DOI":"10.1016\/S0042-6989(97)00169-7","article-title":"Sparse coding with an overcomplete basis set: A strategy employed by V1?","volume":"37","author":"Olshausen","year":"1997","journal-title":"Vision Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.conb.2004.07.007","article-title":"Sparse coding of sensory inputs","volume":"14","author":"Olshausen","year":"2004","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"14259","DOI":"10.1073\/pnas.1203021109","article-title":"Sparse code of conflict in a primate society","volume":"109","author":"Daniels","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_37","unstructured":"Fowler, Harold N. (1925). Phaedrus, Harvard University Press. Available online: http:\/\/www.perseus.tufts.edu\/hopper\/text?doc=Plat.+Phaedrus+265e."},{"key":"ref_38","unstructured":"Afghan War Diary. Available online: http:\/\/wikileaks.org\/afg\/event\/2005\/04\/AFG20050403n68.html."},{"key":"ref_39","unstructured":"Quastler, H. (1955). Information Theory in Psychology II-B, Free Press."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1016\/S0304-4076(97)00099-7","article-title":"Approximate bias correction in econometrics","volume":"85","author":"MacKinnon","year":"1998","journal-title":"J. Econometrics"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"S2","DOI":"10.1111\/j.1475-4932.2006.00328.x","article-title":"Bootstrap Methods in Econometrics","volume":"82","author":"MacKinnon","year":"2006","journal-title":"Econ. Rec."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/15\/6\/2246\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:47:11Z","timestamp":1760219231000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/15\/6\/2246"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,6,5]]},"references-count":41,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2013,6]]}},"alternative-id":["e15062246"],"URL":"https:\/\/doi.org\/10.3390\/e15062246","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,6,5]]}}}