{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:02:35Z","timestamp":1760241755528,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,8,3]],"date-time":"2018-08-03T00:00:00Z","timestamp":1533254400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We consider the problem of model selection using the Minimum Description Length (MDL) criterion for distributions with parameters on the hypersphere. Model selection algorithms aim to find a compromise between goodness of fit and model complexity. Variables often considered for complexity penalties involve number of parameters, sample size and shape of the parameter space, with the penalty term often referred to as stochastic complexity. Current model selection criteria either ignore the shape of the parameter space or incorrectly penalize the complexity of the model, largely because typical Laplace approximation techniques yield inaccurate results for curved spaces. We demonstrate how the use of a constrained Laplace approximation on the hypersphere yields a novel complexity measure that more accurately reflects the geometry of these spherical parameters spaces. We refer to this modified model selection criterion as spherical MDL. As proof of concept, spherical MDL is used for bin selection in histogram density estimation, performing favorably against other model selection criteria.<\/jats:p>","DOI":"10.3390\/e20080575","type":"journal-article","created":{"date-parts":[[2018,8,3]],"date-time":"2018-08-03T11:03:26Z","timestamp":1533294206000},"page":"575","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Spherical Minimum Description Length"],"prefix":"10.3390","volume":"20","author":[{"given":"Trevor","family":"Herntier","sequence":"first","affiliation":[{"name":"Department of Computer Engineering and Sciences, Florida Institute of Technology, Melbourne, FL 32940, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Koffi Eddy","family":"Ihou","sequence":"additional","affiliation":[{"name":"Concordia Institute for Information Systems Engineering, Concordia University, Montreal, QC H3G 1M8, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anthony","family":"Smith","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering and Sciences, Florida Institute of Technology, Melbourne, FL 32940, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8695-8436","authenticated-orcid":false,"given":"Anand","family":"Rangarajan","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL 32611, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8124-5648","authenticated-orcid":false,"given":"Adrian","family":"Peter","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering and Sciences, Florida Institute of Technology, Melbourne, FL 32940, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1016\/0005-1098(78)90005-5","article-title":"Modeling by shortest data description","volume":"14","author":"Rissanen","year":"1978","journal-title":"Automatica"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1214\/aos\/1176346150","article-title":"A universal prior for integers and estimation by minimum description length","volume":"11","author":"Rissanen","year":"1983","journal-title":"Ann. Stat."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Parzen, E., Tanabe, K., and Kitagawa, G. (1998). Information theory and an extension of the maximum likelihood principle. Selected Papers of Hirotugu Akaike, Springer.","DOI":"10.1007\/978-1-4612-1694-0"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","article-title":"A new look at the statistical model identification","volume":"19","author":"Akaike","year":"1974","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1093\/biomet\/88.2.367","article-title":"Counting degrees of freedom in hierarchical and other richly parameterised models","volume":"88","author":"Hodges","year":"1988","journal-title":"Biometrika"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the dimension of a model","volume":"6","author":"Schwarz","year":"1978","journal-title":"Ann. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1080\/10618600.1999.10474843","article-title":"Bootstrapping likelihood for model selection with small samples","volume":"8","author":"Pan","year":"1999","journal-title":"J. Comput. Gr. Stat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1111\/j.2517-6161.1987.tb01694.x","article-title":"Stochastic complexity","volume":"49","author":"Rissanen","year":"1987","journal-title":"J. R. Stat. Soc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/18.481776","article-title":"Fisher information and stochastic complexity","volume":"42","author":"Rissanen","year":"1996","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gr\u00fcnwald, P., Myung, I., and Pitt, M. (2005). A tutorial introduction to the minimum description length principle. Advances in Minimum Description Length: Theory and Applications, The MIT Press.","DOI":"10.7551\/mitpress\/1114.001.0001"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1162\/neco.1997.9.2.349","article-title":"Statistical inference, Occam\u2019s razor, and statistical mechanics on the space of probability distributions","volume":"9","author":"Balasubramanian","year":"1997","journal-title":"Neural Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1111\/j.2517-6161.1982.tb01189.x","article-title":"The Fisher\u2013Bingham distribution on the sphere","volume":"44","author":"Kent","year":"1982","journal-title":"J. R. Stat. Soc."},{"key":"ref_13","unstructured":"Boothby, W.M. (2002). An Introduction to Differentiable Manifolds and Riemannian Geometry, Academic Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2743","DOI":"10.1109\/18.720554","article-title":"The minimum description length principle in coding and modeling","volume":"44","author":"Barron","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2537","DOI":"10.1109\/18.887861","article-title":"MDL denoising","volume":"46","author":"Rissanen","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1093\/comjnl\/42.4.330","article-title":"Refinements of MDL and MML coding","volume":"42","author":"Wallace","year":"1999","journal-title":"Comput. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1093\/comjnl\/11.2.185","article-title":"An information measure for classification","volume":"11","author":"Wallace","year":"1968","journal-title":"Comput. J."},{"key":"ref_18","unstructured":"Wallace, C.S. (2005). Statistical and Inductive Inference by Minimum Message Length, Springer. Information Science and Statistics."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/TPAMI.2006.77","article-title":"Metric learning for text documents","volume":"28","author":"Lebanon","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Srivastava, A., Jermyn, I., and Joshi, S. (2007, January 17\u201322). Riemannian analysis of probability density functions with applications in vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383188"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/TPAMI.2008.69","article-title":"Information geometry for landmark shape analysis: Unifying shape representation and deformation","volume":"31","author":"Peter","year":"2009","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1109\/TIP.2008.918038","article-title":"Maximum likelihood wavelet density estimation with applications to image and shape matching","volume":"17","author":"Peter","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_23","unstructured":"Bingham, C. (1964). Distributions on the Sphere and on the Projective Plane. [Ph.D. Thesis, Yale University]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"67","DOI":"10.2307\/3315537","article-title":"Laplace approximations to posterior moments and marginal distributions on circles, spheres, and cylinders","volume":"19","author":"Parthasarathy","year":"1991","journal-title":"Can. J. Stat."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1093\/biomet\/92.2.465","article-title":"Saddlepoint approximations for the Bingham and Fisher-Bingham normalising constants","volume":"92","author":"Kume","year":"2005","journal-title":"Biometrika"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1093\/biomet\/75.4.705","article-title":"On stochastic complexity and nonparametric density estimation","volume":"75","author":"Hall","year":"1988","journal-title":"Biometrika"},{"key":"ref_27","unstructured":"Kontkanen, P. (2009). Computationally Efficient Methods for MDL-Optimal Density Dstimation and Data Clustering. [Ph.D. Thesis, University of Helsinki]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1051\/ps:2008005","article-title":"A comparison of automatic histogram constructions","volume":"13","author":"Davies","year":"2009","journal-title":"ESAIM"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1162\/neco.1992.4.3.448","article-title":"A practical Bayesian framework for backpropation networks","volume":"4","author":"McKay","year":"1992","journal-title":"Neural Comput."},{"key":"ref_30","unstructured":"Stigler, S.M. (1986). The History of Statistics: The Measurement of Uncertainty before 1900, Harvard University Press."},{"key":"ref_31","first-page":"81","article-title":"Information and accuracy attainable in estimation of statistical parameters","volume":"37","author":"Rao","year":"1945","journal-title":"Bull. Calcutta Math. Soc."},{"key":"ref_32","unstructured":"Jeffreys, H. (1961). Theory of Probability, Oxford University Press. [3rd ed.]."},{"key":"ref_33","first-page":"188","article-title":"The geometry of asymptotic inference","volume":"4","author":"Kass","year":"1989","journal-title":"Stat. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"83","DOI":"10.2307\/1403260","article-title":"The role of differential geometry in statistical theory","volume":"54","author":"Cox","year":"1986","journal-title":"Int. Stat. Rev."},{"key":"ref_35","unstructured":"Amari, S.I., and Nagaoka, H. (2001). Methods of Information Geometry, American Mathematical Society."},{"key":"ref_36","unstructured":"Cover, T., and Thomas, J. (2006). Elements of Information Theory, Wiley Interscience. [2nd ed.]."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1214\/ss\/1177013621","article-title":"Memoir on the probability of the causes of events","volume":"1","author":"Laplace","year":"1986","journal-title":"Stat. Sci."},{"key":"ref_38","unstructured":"Bertsekas, D.P. (1999). Nonlinear Programming, Athena Scientific. [2nd ed.]."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1090\/S0002-9947-1941-0003498-3","article-title":"The accuracy of the Gaussian approximation to the sum of independent variates","volume":"49","author":"Berry","year":"1941","journal-title":"Trans. Am. Math. Soc."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Robert, C., and Casella, G. (2004). Monte Carlo Statistical Methods, Springer. [2nd ed.].","DOI":"10.1007\/978-1-4757-4145-2"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.jmp.2014.06.002","article-title":"Model selection by minimum description length: Lower-bound sample sizes for the Fisher information approximation","volume":"60","author":"Heck","year":"2014","journal-title":"J. Math. Psychol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1763","DOI":"10.1162\/0899766041336378","article-title":"A note on the applied use of MDL approximations","volume":"16","author":"Navarro","year":"2004","journal-title":"Neural Comput."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Peter, A., and Rangarajan, A. (2011, January 6\u201313). An information geometry approach to shape density minimum description length model selection. Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops, Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130419"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1093\/biomet\/74.3.636","article-title":"Akaike\u2019s information criterion and the histogram","volume":"74","author":"Taylor","year":"1987","journal-title":"Biometrika"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Abramowitz, M., and Stegun, I.A. (1965). Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables, Dover Publications.","DOI":"10.1063\/1.3047921"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1214\/aos\/1176348653","article-title":"Exact mean integrated squared error","volume":"20","author":"Marron","year":"1992","journal-title":"Ann. Stat."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1080\/01621459.1993.10476303","article-title":"Comparison of smoothing parameterizations in bivariate kernel density estimation","volume":"88","author":"Wand","year":"1993","journal-title":"J. Am. Stat. Assoc."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/575\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:16:22Z","timestamp":1760195782000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/8\/575"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,3]]},"references-count":47,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["e20080575"],"URL":"https:\/\/doi.org\/10.3390\/e20080575","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2018,8,3]]}}}