{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:28:10Z","timestamp":1760243290283,"version":"build-2065373602"},"reference-count":23,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2014,7,17]],"date-time":"2014-07-17T00:00:00Z","timestamp":1405555200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The Principle of Maximum Entropy is often used to update probabilities due to evidence instead of performing Bayesian updating using Bayes\u2019 Theorem, and its use often has efficacious results. However, in some circumstances the results seem unacceptable and unintuitive. This paper discusses some of these cases, and discusses how to identify some of the situations in which this principle should not be used. The paper starts by reviewing three approaches to probability, namely the classical approach, the limiting frequency approach, and the Bayesian approach. It then introduces maximum entropy and shows its relationship to the three approaches. Next, through examples, it shows that maximizing entropy sometimes can stand in direct opposition to Bayesian updating based on reasonable prior beliefs. The paper concludes that if we take the Bayesian approach that probability is about reasonable belief based on all available information, then we can resolve the conflict between the maximum entropy approach and the Bayesian approach that is demonstrated in the examples.<\/jats:p>","DOI":"10.3390\/e16074004","type":"journal-article","created":{"date-parts":[[2014,7,17]],"date-time":"2014-07-17T11:26:30Z","timestamp":1405596390000},"page":"4004-4014","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["A Note of Caution on Maximizing Entropy"],"prefix":"10.3390","volume":"16","author":[{"given":"Richard","family":"Neapolitan","sequence":"first","affiliation":[{"name":"Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL 60611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xia","family":"Jiang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, PA 15232, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2014,7,17]]},"reference":[{"key":"ref_1","unstructured":"Neapolitan, R.E. (1989). Probabilistic Reasoning in Expert Systems, Wiley."},{"key":"ref_2","unstructured":"De Laplace, P.S. (1995). Philosophical Essay on Probabilities, Springer."},{"key":"ref_3","unstructured":"Keynes, J.M. (2004). A Treatise on Probability, Dover."},{"key":"ref_4","unstructured":"Weatherford, R. (1982). Philosophical Foundations of Probability Theory, Routledge & Kegan Paul."},{"key":"ref_5","unstructured":"Von Mises, R. (1957). Probability, Statistics, and Truth, George Allen & Unwin."},{"key":"ref_6","unstructured":"Kerrich, J.M. (1946). An Experimental Introduction to the Theory of Probability, Einer Munskgaard."},{"key":"ref_7","first-page":"417","article-title":"Bias and Runs in Dice Throwing and Recording: A Few Million Throws","volume":"36","author":"Iverson","year":"1971","journal-title":"Psychometrika"},{"key":"ref_8","unstructured":"De Finetti, B. (1972). Probability, Induction, and Statistics, Wiley."},{"key":"ref_9","unstructured":"Good, I.J. (1965). The Estimation of Probabilities: An Essay on Modern Bayesian Methods, MIT Press. Research Monograph No. 30."},{"key":"ref_10","first-page":"62","article-title":"46656 Varieties of Bayesianism","volume":"25","author":"Good","year":"1971","journal-title":"Am. Stat"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Skyrms, B., and Harper, W.L. (1988). Causation, Chance, and Credence, Kluwer Academics.","DOI":"10.1007\/978-94-009-2863-3"},{"key":"ref_12","unstructured":"Levine, R.D., and Tribus, M. (1979). The Maximum Entropy Formalism, MIT Press."},{"key":"ref_13","unstructured":"Carnap, R. (1950). Logical Foundations of Probability, University of Chicago Press."},{"key":"ref_14","unstructured":"Nagel, E. (1939). Principles of the Theory of Probability, University of Chicago Press."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1109\/PROC.1982.12425","article-title":"On the Rationale of Maximum-Entropy Methods","volume":"70","author":"Jaynes","year":"1982","journal-title":"Proc. IEEE"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1214\/aos\/1176345975","article-title":"W.E. Johnson\u2019s \u201cSufficientness\u201d Postulate","volume":"10","author":"Zabell","year":"1982","journal-title":"Ann. Stat"},{"key":"ref_17","unstructured":"Neapolitan, R.E. (2004). Learning Bayesian Networks, Prentice Hall."},{"key":"ref_18","first-page":"259","article-title":"Entropy and Uncertainty","volume":"53","author":"Seidenfield","year":"1986","journal-title":"Philos. Sci"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/0196-8858(81)90003-8","article-title":"A Critique of Jaynes\u2019 Maximum Entropy Principle","volume":"2","author":"Dias","year":"1981","journal-title":"Adv. Appl. Math"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF00485957","article-title":"Some Random Observations","volume":"63","author":"Jaynes","year":"1985","journal-title":"Synthese"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/1355-2198(95)00015-1","article-title":"Can the Maximum Entropy Principle be Explained as a Consistency Requirement?","volume":"26","author":"Uffink","year":"1995","journal-title":"Stud. Hist. Philos. Sci. B Stud. Hist. Philos. Mod. Phys"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/1355-2198(95)00022-4","article-title":"The Constraint Rule of the Maximum Entropy Principle","volume":"27","author":"Uffink","year":"1996","journal-title":"Stud. Hist. Philos. Sci. B Stud. Hist. Philos. Mod. Phys"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1080\/01621459.1982.10477893","article-title":"Updating Subjective Probability","volume":"77","author":"Diaconis","year":"1982","journal-title":"J. Am. Stat. Assoc"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/7\/4004\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:13:42Z","timestamp":1760217222000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/7\/4004"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,7,17]]},"references-count":23,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2014,7]]}},"alternative-id":["e16074004"],"URL":"https:\/\/doi.org\/10.3390\/e16074004","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2014,7,17]]}}}