{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T20:06:39Z","timestamp":1780085199177,"version":"3.54.0"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2015,6,18]],"date-time":"2015-06-18T00:00:00Z","timestamp":1434585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we study Amari\u2019s natural gradient flows of real functions defined on the densities belonging to an exponential family on a finite sample space. Our main example is the minimization of the expected value of a real function defined on the sample space. In such a case, the natural gradient flow converges to densities with reduced support that belong to the border of the exponential family. We have suggested in previous works to use the natural gradient evaluated in the mixture geometry. Here, we show that in some cases, the differential equation can be extended to a bigger domain in such a way that the densities at the border of the exponential family are actually internal points in the extended problem. The extension is based on the algebraic concept of an exponential variety. We study in full detail a toy example and obtain positive partial results in the important case of a binary sample space.<\/jats:p>","DOI":"10.3390\/e17064215","type":"journal-article","created":{"date-parts":[[2015,6,18]],"date-time":"2015-06-18T10:08:54Z","timestamp":1434622134000},"page":"4215-4254","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Natural Gradient Flow in the Mixture Geometry of a Discrete Exponential Family"],"prefix":"10.3390","volume":"17","author":[{"given":"Luigi","family":"Malag\u00f2","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Shinshu University, Nagano, Japan"},{"name":"Inria Saclay, \u00cele-de-France, Orsay Cedex, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giovanni","family":"Pistone","sequence":"additional","affiliation":[{"name":"De Castro Statistics, Collegio Carlo Alberto, Moncalieri, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2015,6,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/978-3-642-40020-9_3","article-title":"Nonparametric information geometry","volume":"8085","author":"Nielsen","year":"2013","journal-title":"Geometric Science of Information"},{"key":"ref_2","unstructured":"Malag\u00f2, L., Matteucci, M., and Pistone, G. (2009, January 11\u201312). Stochastic Relaxation as a Unifying Approach in 0\/1 Programming, 2009, Whistler Resort & Spa, BC, Canada."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Malag\u00f2, L., Matteucci, M., and Pistone, G. (2011, January 5\u20138). Towards the geometry of estimation of distribution algorithms based on the exponential family. Schwarzenberg, Austria.","DOI":"10.1145\/1967654.1967675"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Malag\u00f2, L., Matteucci, M., and Pistone, G. (2011, January 5\u20138). Stochastic Natural Gradient Descent by estimation of empirical covariances. New Orleans, LA, USA.","DOI":"10.1109\/CEC.2011.5949720"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Malag\u00f2, L., Matteucci, M., and Pistone, G. (2013, January 20\u201323). Natural gradient, fitness modelling and model selection: A unifying perspective, Cancun, Mexico.","DOI":"10.1109\/CEC.2013.6557608"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wierstra, D., Schaul, T., Peters, J., and Schmidhuber, J. (2008, January 1\u20136). Natural evolution strategies. Hong Kong, China.","DOI":"10.1109\/CEC.2008.4631255"},{"key":"ref_7","unstructured":"Ollivier, Y., Arnold, L., Auger, A., and Hansen, N. (2011). Information-Geometric Optimization Algorithms: A Unifying Picture via Invariance Principles, arXiv: 1106.3708."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4260","DOI":"10.3390\/e16084260","article-title":"Combinatorial Optimization with Information Geometry: Newton method","volume":"16","author":"Pistone","year":"2014","journal-title":"Entropy"},{"key":"ref_9","unstructured":"Amari, S., and Nagaoka, H. (2000). Methods of Information Geometry, American Mathematical Society. Translated from the 1993 Japanese original by Daishi Harada."},{"key":"ref_10","unstructured":"Bourbaki, N. (1971). Vari\u00e9t\u00e9s differentielles et analytiques. Fascicule de r\u00e9sultats \/ Paragraphes 1 \u00e0 7, Hermann."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1214\/aos\/1176324311","article-title":"An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one","volume":"23","author":"Pistone","year":"1995","journal-title":"Ann. Stat."},{"key":"ref_12","unstructured":"Mohammad-Djafari, A., and Barbaresco, F. (2014, January 21\u201326). Gradient Flow of the Stochastic Relaxation on a Generic Exponential Family. Clos Luc\u00e9, Amboise, France."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Brown, L.D. (1986). Fundamentals of Statistical Exponential Families With Applications in Statistical Decision Theory, Institute of Mathematical Statistics.","DOI":"10.1214\/lnms\/1215466757"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rockafellar, R.T. (1970). Convex Analysis, Princeton University Press.","DOI":"10.1515\/9781400873173"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Do Carmo, M.P. (1992). Riemannian Geometry, Birkh\u00e4user Boston Inc.. Translated from the second Portuguese edition by Francis Flaherty.","DOI":"10.1007\/978-1-4757-2201-7"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1162\/089976698300017746","article-title":"Natural gradient works efficiently in learning","volume":"10","author":"Amari","year":"1998","journal-title":"Neur. Comput."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Shima, H. (2007). The Geometry of Hessian Structures, World Scientific Publishing Co. Pte. Ltd.","DOI":"10.1142\/9789812707536"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1214\/08-EJS350","article-title":"On the geometry of discrete exponential families with application to exponential random graph models.","volume":"3","author":"Rinaldo","year":"2009","journal-title":"Electron. J. Stat."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1016\/j.ijar.2011.01.013","article-title":"Support Sets in Exponential Families and Oriented Matroid Theory","volume":"52","author":"Rauh","year":"2011","journal-title":"Int. J. Approx. Reas."},{"key":"ref_20","unstructured":"Malag\u00f2, L., and Pistone, G. (2010). A note on the border of an exponential family, arXiv:1012.0637v1."},{"key":"ref_21","unstructured":"Pistone, G., and Rogantin, M. (2015). The gradient flow of the polarization measure. With an appendix, arXiv:1502.06718."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1214\/aos\/1030563990","article-title":"Algebraic algorithms for sampling from conditional distributions","volume":"26","author":"Diaconis","year":"1998","journal-title":"Ann. Stat."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"653","DOI":"10.1093\/biomet\/83.3.653","article-title":"Generalised confounding with Gr\u00f6bner bases","volume":"83","author":"Pistone","year":"1996","journal-title":"Biometrika"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pistone, G., Riccomagno, E., and Wynn, H.P. (2001). Algebraic Statistics: Computational Commutative Algebra in Statistics, Chapman & Hall\/CRC.","DOI":"10.1201\/9781420035766"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Drton, M., Sturmfels, B., and Sullivant, S. (2009). Lectures on Algebraic Statistics, Birkh\u00e4user Verlag.","DOI":"10.1007\/978-3-7643-8905-5"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Pachter, L., and Sturmfels, B. (2005). Algebraic Statistics for Computational Biology, Cambridge University Press.","DOI":"10.1017\/CBO9780511610684"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gibilisco, P., Riccomagno, E., Rogantin, M.P., and Wynn, H.P. (2010). Algebraic and Geometric Methods in Statistics, Cambridge University Press.","DOI":"10.1017\/CBO9780511642401"},{"key":"ref_28","unstructured":"4ti2 team 4ti2\u2014A software package for algebraic, geometric and combinatorial problems on linear spaces. Available online: http:\/\/www.4ti2.de."},{"key":"ref_29","unstructured":"Micha\u0142ek, M., Sturmfels, B., Uhler, C., and Zwiernik, P. (2014). Exponential Varieties, arXiv:1412.6185."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sturmfels, B. (1996). Gr\u00f6bner Bases and Convex Polytopes, American Mathematical Society.","DOI":"10.1090\/ulect\/008"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1463","DOI":"10.1214\/009053606000000263","article-title":"On the toric algebra of graphical models","volume":"34","author":"Geiger","year":"2006","journal-title":"Ann. Stat."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1007\/s10463-006-0079-z","article-title":"Toric statistical models: Parametric and binomial representations","volume":"59","author":"Rapallo","year":"2007","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Beltrametti, M., Carletti, E., Gallarati, D., and Monti Bragadin, G. (2009). Lectures on Curves, Surfaces and Projective Varieties: A Classical View of Algebraic Geometry, European Mathematical Society.","DOI":"10.4171\/064"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1214\/08-EJS350","article-title":"On the geometry of discrete exponential families with application to exponential random graph models","volume":"3","author":"Rinaldo","year":"2009","journal-title":"Electron. J. Stat."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gibilisco, P., Riccomagno, E., Rogantin, M., and Wynn, H.P. (2009). Algebraic and Geometric Methods in Statistics, Cambridge University Press. Chapter 21.","DOI":"10.1017\/CBO9780511642401"},{"key":"ref_36","unstructured":"Abbott, J., Bigatti, A., and Lagorio, G. CoCoA-5: A system for doing Computations in Commutative Algebra. Available online: http:\/\/cocoa.dima.unige.it."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/4215\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:48:05Z","timestamp":1760215685000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/6\/4215"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,6,18]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2015,6]]}},"alternative-id":["e17064215"],"URL":"https:\/\/doi.org\/10.3390\/e17064215","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,6,18]]}}}