{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T06:59:11Z","timestamp":1792738751879,"version":"4.5.0"},"reference-count":55,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2020,2,16]],"date-time":"2020-02-16T00:00:00Z","timestamp":1581811200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The Jensen\u2013Shannon divergence is a renown bounded symmetrization of the Kullback\u2013Leibler divergence which does not require probability densities to have matching supports. In this paper, we introduce a vector-skew generalization of the scalar    \u03b1   -Jensen\u2013Bregman divergences and derive thereof the vector-skew    \u03b1   -Jensen\u2013Shannon divergences. We prove that the vector-skew    \u03b1   -Jensen\u2013Shannon divergences are f-divergences and study the properties of these novel divergences. Finally, we report an iterative algorithm to numerically compute the Jensen\u2013Shannon-type centroids for a set of probability densities belonging to a mixture family: This includes the case of the Jensen\u2013Shannon centroid of a set of categorical distributions or normalized histograms.<\/jats:p>","DOI":"10.3390\/e22020221","type":"journal-article","created":{"date-parts":[[2020,2,18]],"date-time":"2020-02-18T10:10:25Z","timestamp":1582020625000},"page":"221","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":143,"title":["On a Generalization of the Jensen\u2013Shannon Divergence and the Jensen\u2013Shannon Centroid"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5728-0726","authenticated-orcid":false,"given":"Frank","family":"Nielsen","sequence":"first","affiliation":[{"name":"Sony Computer Science Laboratories, Tokyo 141-0022, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,16]]},"reference":[{"key":"ref_1","unstructured":"Billingsley, P. (2008). Probability and Measure, John Wiley & Sons."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Deza, M.M., and Deza, E. (2009). Encyclopedia of Distances, Springer.","DOI":"10.1007\/978-3-642-00234-2"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1016\/j.sigpro.2012.09.003","article-title":"Divergence measures for statistical data processing\u2014An annotated bibliography","volume":"93","author":"Basseville","year":"2013","journal-title":"Signal Process."},{"key":"ref_4","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Nielsen, F. (2019). On the Jensen\u2013Shannon Symmetrization of Distances Relying on Abstract Means. Entropy, 21.","DOI":"10.3390\/e21050485"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sason, I. (May, January 26). Tight bounds for symmetric divergence measures and a new inequality relating f-divergences. Proceedings of the 2015 IEEE Information Theory Workshop (ITW), Jerusalem, Israel.","DOI":"10.1109\/ITW.2015.7133079"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1109\/TPAMI.1985.4767707","article-title":"Entropy and distance of random graphs with application to structural pattern recognition","volume":"7","author":"Wong","year":"1985","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1109\/TIT.2003.813506","article-title":"A new metric for probability distributions","volume":"49","author":"Endres","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","first-page":"415","article-title":"On powers of f-divergences defining a distance","volume":"26","author":"Kafka","year":"1991","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.exmath.2005.01.014","article-title":"Spirals in Hilbert space: With an application in information theory","volume":"23","author":"Fuglede","year":"2005","journal-title":"Expo. Math."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Acharyya, S., Banerjee, A., and Boley, D. (2013, January 2\u20134). Bregman divergences and triangle inequality. Proceedings of the 2013 SIAM International Conference on Data Mining, Austin, TX, USA.","DOI":"10.1137\/1.9781611972832.53"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2148","DOI":"10.1109\/TIT.2015.2401004","article-title":"Extrinsic Jensen\u2013Shannon divergence: Applications to variable-length coding","volume":"61","author":"Naghshvar","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bigi, B. (2003). Using Kullback-Leibler distance for text categorization. European Conference on Information Retrieval, Springer.","DOI":"10.1007\/3-540-36618-0_22"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"174111","DOI":"10.1063\/1.2121610","article-title":"Information entropy, information distances, and complexity in atoms","volume":"123","author":"Chatzisavvas","year":"2005","journal-title":"J. Chem. Phys."},{"key":"ref_16","unstructured":"Yurdakul, B. (2018). Statistical Properties of Population Stability Index. [Ph.D. Thesis, Western Michigan University]."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1098\/rspa.1946.0056","article-title":"An invariant form for the prior probability in estimation problems","volume":"186","author":"Jeffreys","year":"1946","journal-title":"Proc. R. Soc. Lond. A"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1109\/LSP.2013.2260538","article-title":"Jeffreys centroids: A closed-form expression for positive histograms and a guaranteed tight approximation for frequency histograms","volume":"20","author":"Nielsen","year":"2013","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Lee, L. (1999). Measures of Distributional Similarity. Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics, ACL \u201999, Association for Computational Linguistics.","DOI":"10.3115\/1034678.1034693"},{"key":"ref_20","unstructured":"Nielsen, F. (2010). A family of statistical symmetric divergences based on Jensen\u2019s inequality. arXiv."},{"key":"ref_21","unstructured":"Lee, L. (2001, January 4\u20137). On the effectiveness of the skew divergence for statistical language analysis. Proceedings of the 8th International Workshop on Artificial Intelligence and Statistics (AISTATS 2001), Key West, FL, USA."},{"key":"ref_22","first-page":"229","article-title":"Information-type measures of difference of probability distributions and indirect observation","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sason, I. (2018). On f-divergences: Integral representations, local behavior, and inequalities. Entropy, 20.","DOI":"10.3390\/e20050383"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Amari, S.I. (2016). Information Geometry and Its Applications, Springer.","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7616","DOI":"10.1109\/TIT.2014.2360184","article-title":"Information measures: The curious case of the binary alphabet","volume":"60","author":"Jiao","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_27","unstructured":"Yuille, A.L., and Rangarajan, A. (2002, January 9\u201314). The concave-convex procedure (CCCP). Proceedings of the Neural Information Processing Systems 2002, Vancouver, BC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2011). Skew Jensen-Bregman Voronoi diagrams. Transactions on Computational Science XIV, Springer.","DOI":"10.1007\/978-3-642-25249-5_4"},{"key":"ref_29","first-page":"1705","article-title":"Clustering with Bregman divergences","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2882","DOI":"10.1109\/TIT.2009.2018176","article-title":"Sided and symmetrized Bregman centroids","volume":"55","author":"Nielsen","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_31","unstructured":"Melbourne, J., Talukdar, S., Bhaban, S., Madiman, M., and Salapaka, M.V. (2020, February 16). On the Entropy of Mixture distributions. Available online: http:\/\/box5779.temp.domains\/~jamesmel\/publications\/."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2386","DOI":"10.1109\/TIT.2011.2110791","article-title":"Lower bounds for the minimax risk using f-divergences, and applications","volume":"57","author":"Guntuboyina","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"5973","DOI":"10.1109\/TIT.2016.2603151","article-title":"f-divergence Inequalities","volume":"62","author":"Sason","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Melbourne, J., Madiman, M., and Salapaka, M.V. (2019, January 24\u201327). Relationships between certain f-divergences. Proceedings of the 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, IL, USA .","DOI":"10.1109\/ALLERTON.2019.8919677"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sason, I. (2019). On Data-Processing and Majorization Inequalities for f-Divergences with Applications. Entropy, 21.","DOI":"10.3390\/e21101022"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","author":"Harremos","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Xu, P., Melbourne, J., and Madiman, M. (2017, January 25\u201330). Infinity-R\u00e9nyi entropy power inequalities. Proceedings of the 2017 IEEE International Symposium on Information Theory (ISIT), Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8007077"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2018, January 15\u201320). On the geometry of mixtures of prescribed distributions. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8461869"},{"key":"ref_39","first-page":"215","article-title":"Les \u00e9l\u00e9ments al\u00e9atoires de nature quelconque dans un espace distanci\u00e9","volume":"10","year":"1948","journal-title":"Ann. De L\u2019institut Henri Poincar\u00c9"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"5455","DOI":"10.1109\/TIT.2011.2159046","article-title":"The Burbea-Rao and Bhattacharyya centroids","volume":"57","author":"Nielsen","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_41","unstructured":"Lanckriet, G.R., and Sriperumbudur, B.K. (2009, January 7\u201310). On the convergence of the concave-convex procedure. Proceedings of the Advances in Neural Information Processing Systems 22 (NIPS 2009), Vancouver, BC, Canada."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Sun, K. (2016). Guaranteed bounds on information-theoretic measures of univariate mixtures using piecewise log-sum-exp inequalities. Entropy, 18.","DOI":"10.20944\/preprints201610.0086.v1"},{"key":"ref_43","unstructured":"Springer Verlag GmbH, European Mathematical Society (2019, December 19). Encyclopedia of Mathematics. Available online: https:\/\/www.encyclopediaofmath.org\/."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/BF00773592","article-title":"The singly truncated normal distribution: A non-steep exponential family","volume":"46","year":"1994","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2010, January 26\u201329). Entropies and cross-entropies of exponential families. Proceedings of the 2010 IEEE International Conference on Image Processing, Hong Kong, China.","DOI":"10.1109\/ICIP.2010.5652054"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Hadjeres, G. (2018). Monte Carlo information geometry: The dually flat case. arXiv.","DOI":"10.1007\/978-3-030-02520-5_5"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Schwander, O., and Nielsen, F. (2013). Learning mixtures by simplifying kernel density estimators. Matrix Information Geometry, Springer.","DOI":"10.1007\/978-3-642-30232-9_16"},{"key":"ref_48","unstructured":"Arthur, D., and Vassilvitskii, S. (2007, January 7\u20139). k-means++: The advantages of careful seeding. Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms (SODA\u201907), New Orleans, LA, USA."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3273","DOI":"10.3390\/e16063273","article-title":"On clustering histograms with k-means by using mixed \u03b1-divergences","volume":"16","author":"Nielsen","year":"2014","journal-title":"Entropy"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2015, January 19\u201324). Total Jensen divergences: Definition, properties and clustering. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brisbane, QLD, Australia.","DOI":"10.1109\/ICASSP.2015.7178324"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"162","DOI":"10.3390\/e3030162","article-title":"Basic concepts, identities and inequalities-the toolkit of information theory","volume":"3","year":"2001","journal-title":"Entropy"},{"key":"ref_52","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems 27 (NIPS 2014), Montreal, QC, Canada."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"100064","DOI":"10.1016\/j.rinam.2019.100064","article-title":"Some bounds for skewed \u03b1-Jensen-Shannon divergence","volume":"3","author":"Yamano","year":"2019","journal-title":"Results Appl. Math."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1017\/S1351324910000124","article-title":"Directional distributional similarity for lexical inference","volume":"16","author":"Kotlerman","year":"2010","journal-title":"Nat. Lang. Eng."},{"key":"ref_55","unstructured":"Johnson, D., and Sinanovic, S. (2001). Symmetrizing the Kullback-Leibler distance. IEEE Trans. Inf. Theory, 1\u20138."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/221\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:58:17Z","timestamp":1760173097000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/2\/221"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,16]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["e22020221"],"URL":"https:\/\/doi.org\/10.3390\/e22020221","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,16]]}}}