{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T01:10:10Z","timestamp":1790730610332,"version":"4.1.0"},"reference-count":20,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2017,5,19]],"date-time":"2017-05-19T00:00:00Z","timestamp":1495152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Entropy methods enable a convenient general approach to providing a probability distribution with partial information. The minimum cross-entropy principle selects the distribution that minimizes the Kullback\u2013Leibler divergence subject to the given constraints. This general principle encompasses a wide variety of distributions, and generalizes other methods that have been proposed independently. There remains, however, some confusion about the breadth of entropy methods in the literature. In particular, the asymmetry of the Kullback\u2013Leibler divergence provides two important special cases when the target distribution is uniform: the maximum entropy method and the maximum log-probability method. This paper compares the performance of both methods under a variety of conditions. We also examine a generalized maximum log-probability method as a further demonstration of the generality of the entropy approach.<\/jats:p>","DOI":"10.3390\/e19050232","type":"journal-article","created":{"date-parts":[[2017,5,23]],"date-time":"2017-05-23T01:47:33Z","timestamp":1495504053000},"page":"232","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["A Kullback\u2013Leibler View of Maximum Entropy and Maximum Log-Probability Methods"],"prefix":"10.3390","volume":"19","author":[{"given":"Ali","family":"Abbas","sequence":"first","affiliation":[{"name":"Industrial &amp; Systems Engineering and Public Policy, University of Southern California, Los Angeles, CA 90089, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"H. Cadenbach","sequence":"additional","affiliation":[{"name":"Supply Chain & Analytics, University of Missouri-St. Louis, St. Louis, MO 63121, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ehsan","family":"Salimi","sequence":"additional","affiliation":[{"name":"Industrial &amp; Systems Engineering, University of Southern California, Los Angeles, CA 90007, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,5,19]]},"reference":[{"key":"ref_1","unstructured":"Howard, R., and Abbas, A.E. (2015). Foundations of Decision Analysis, Prentice Hall."},{"key":"ref_2","unstructured":"Box, G.E., and Tiao, G.C. (1973). Bayesian Inference in Statistical Analysis, John Wiley & Sons."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1109\/TSSC.1968.300117","article-title":"Prior Probabilities","volume":"4","author":"Jaynes","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern."},{"key":"ref_4","unstructured":"Nelsen, R.B. (2013). An Introduction to Copulas, Springer."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1287\/deca.1080.0133","article-title":"A Kullback\u2013Leibler view of Linear and Log-Linear Pools","volume":"6","author":"Abbas","year":"2009","journal-title":"Decis. Anal."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1287\/deca.1120.0261","article-title":"Approximating Joint Probability Distributions Given Partial Information","volume":"10","author":"Montiel","year":"2013","journal-title":"Decis. Anal."},{"key":"ref_7","unstructured":"Rubenstein, R.Y., and Kroese, D.P. (2004). The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning, Springer."},{"key":"ref_8","unstructured":"Williams, C. (2002). Entropy Methods for Univariate Distributions in Decision Analysis. Bayesian Inference and Maximum Entropy Methods in Science and Engineering, American Institute of Physics."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1109\/TEM.2005.861803","article-title":"Entropy Methods for Joint Distributions in Decision Analysis","volume":"53","author":"Abbas","year":"2006","journal-title":"IEEE Trans. Eng. Manag."},{"key":"ref_10","unstructured":"Castle, J.P., Morris, R.D., Abbas, A.E., and Knuth, K.H. (2005). Maximum Entropy Distributions Between Upper and Lower bounds. Bayesian Inference and Maximum Entropy Methods in Science and Engineering, American Institute of Physics."},{"key":"ref_11","unstructured":"Williams, C. (2002). An Entropy Approach for Utility Assignment in Decision Analysis. Bayesian Inference and Maximum Entropy Methods in Science and Engineering, American Institute of Physics."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/0377-0427(89)90346-4","article-title":"Applications of the notion of analytic center in approximation (estimation) problems","volume":"28","author":"Sonnevend","year":"1989","journal-title":"J. Comput. Appl. Math."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ye, Y. (1997). Interior Point Algorithms: Theory and Analysis, Wiley Interscience.","DOI":"10.1002\/9781118032701"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On Information and Sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_15","unstructured":"Kullback, S. (1997). Information Theory and Statistics, Dover Publications."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Boyd, S., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information Theory and Statistical Mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1002\/j.1538-7305.1949.tb00928.x","article-title":"Communication Theory of Secrecy Systems","volume":"28","author":"Shannon","year":"1949","journal-title":"Bell Syst. Tech. J."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sonnevend, G. (1986). An \u2018analytical centre\u2019 for polyhedrons and new classes of global algorithms for linear (smooth, convex) programming. System Modelling and Optimization, Springer.","DOI":"10.1007\/BFb0043914"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1109\/TSMCA.2003.822269","article-title":"Entropy Methods for Adaptive Utility Elicitation","volume":"34","author":"Abbas","year":"2004","journal-title":"IEEE Trans. Syst. Man Cybern. Part A"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/5\/232\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:36:19Z","timestamp":1760207779000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/5\/232"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,19]]},"references-count":20,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2017,5]]}},"alternative-id":["e19050232"],"URL":"https:\/\/doi.org\/10.3390\/e19050232","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,5,19]]}}}