{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T20:54:05Z","timestamp":1785531245048,"version":"3.56.0"},"reference-count":35,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T00:00:00Z","timestamp":1711497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Sciences and Engineering Research Council (NSERC) of Canada"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>A unifying \u03b1-parametrized generator loss function is introduced for a dual-objective generative adversarial network (GAN) that uses a canonical (or classical) discriminator loss function such as the one in the original GAN (VanillaGAN) system. The generator loss function is based on a symmetric class probability estimation type function, L\u03b1, and the resulting GAN system is termed L\u03b1-GAN. Under an optimal discriminator, it is shown that the generator\u2019s optimization problem consists of minimizing a Jensen-f\u03b1-divergence, a natural generalization of the Jensen-Shannon divergence, where f\u03b1 is a convex function expressed in terms of the loss function L\u03b1. It is also demonstrated that this L\u03b1-GAN problem recovers as special cases a number of GAN problems in the literature, including VanillaGAN, least squares GAN (LSGAN), least kth-order GAN (LkGAN), and the recently introduced (\u03b1D,\u03b1G)-GAN with \u03b1D=1. Finally, experimental results are provided for three datasets\u2014MNIST, CIFAR-10, and Stacked MNIST\u2014to illustrate the performance of various examples of the L\u03b1-GAN system.<\/jats:p>","DOI":"10.3390\/e26040290","type":"journal-article","created":{"date-parts":[[2024,3,27]],"date-time":"2024-03-27T13:39:56Z","timestamp":1711546796000},"page":"290","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A Unifying Generator Loss Function for Generative Adversarial Networks"],"prefix":"10.3390","volume":"26","author":[{"given":"Justin","family":"Veiner","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7980-724X","authenticated-orcid":false,"given":"Fady","family":"Alajaji","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1495-3489","authenticated-orcid":false,"given":"Bahman","family":"Gharesifard","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, University of California, Los Angeles, CA 90095, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,27]]},"reference":[{"key":"ref_1","first-page":"2672","article-title":"Generative adversarial nets","volume":"Volume 27","author":"Ghahramani","year":"2014","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kwon, Y.H., and Park, M.G. (2019, January 16\u201320). Predicting future frames using retrospective cycle GAN. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00191"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"7474","DOI":"10.1109\/TPAMI.2021.3115428","article-title":"Exploiting deep generative prior for versatile image restoration and manipulation","volume":"44","author":"Pan","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","unstructured":"Jordon, J., Yoon, J., and Van Der Schaar, M. (May, January 30). PATE-GAN: Generating synthetic data with differential privacy guarantees. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_5","unstructured":"Radford, A., Metz, L., and Chintala, S. (2017, January 13\u201315). Unsupervised representation learning with deep convolutional generative adversarial networks. Proceedings of the 9th International Conference on Image and Graphics, Shanghai, China."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2473","DOI":"10.1162\/neco_a_01416","article-title":"Least kth-order and R\u00e9nyi generative adversarial networks","volume":"33","author":"Bhatia","year":"2021","journal-title":"Neural Comput."},{"key":"ref_7","unstructured":"Nowozin, S., Cseke, B., and Tomioka, R. (2016). Advances in Neural Information Processing Systems, MIT Press."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kurri, G.R., Sypherd, T., and Sankar, L. (2021, January 17\u201321). Realizing GANs via a tunable loss function. Proceedings of the IEEE Information Theory Workshop (ITW), Virtual.","DOI":"10.1109\/ITW48936.2021.9611499"},{"key":"ref_9","unstructured":"Kurri, G.R., Welfert, M., Sypherd, T., and Sankar, L. (July, January 26). \u03b1-GAN: Convergence and estimation guarantees. Proceedings of the IEEE International Symposium on Information Theory (ISIT), Espoo, Finland."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Welfert, M., Otstot, K., Kurri, G.R., and Sankar, L. (2023, January 25\u201330). (\u03b1D,\u03b1G)-GANs: Addressing GAN training instabilities via dual objectives. Proceedings of the IEEE International Symposium on Information Theory (ISIT), Taipei, Taiwan.","DOI":"10.1109\/ISIT54713.2023.10206844"},{"key":"ref_11","unstructured":"Welfert, M., Kurri, G.R., Otstot, K., and Sankar, L. (2023). Addressing GAN training instabilities via tunable classification losses. arXiv."},{"key":"ref_12","first-page":"85","article-title":"Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Bewis der Ergodizitat on Markhoffschen Ketten","volume":"8","author":"Csiszar","year":"1963","journal-title":"Publ. Math. Inst. Hung. Acad. Sci. Ser. A"},{"key":"ref_13","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Ser. (Methodol.)"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Nielsen, F. (2020). On a generalization of the Jensen\u2013Shannon divergence and the Jensen\u2013Shannon centroid. Entropy, 22.","DOI":"10.3390\/e22020221"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/LSP.2013.2288355","article-title":"On the chi square and higher-order chi distances for approximating f-divergences","volume":"21","author":"Nielsen","year":"2013","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0019-9958(71)90065-9","article-title":"Information-theoretical considerations on estimation problems","volume":"19","author":"Arimoto","year":"1971","journal-title":"Inf. Control."},{"key":"ref_20","first-page":"389","article-title":"On a class of perimeter-type distances of probability distributions","volume":"32","year":"1996","journal-title":"Kybernetika"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1515\/crll.1909.136.210","article-title":"Neue Begr\u00fcndung der Theorie quadratischer Formen von unendlichvielen Ver\u00e4nderlichen","volume":"1909","author":"Hellinger","year":"1909","journal-title":"J. Reine Angew. Math."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sason, I. (2018). On f-divergences: Integral representations, local behavior, and inequalities. Entropy, 20.","DOI":"10.3390\/e20050383"},{"key":"ref_23","first-page":"547","article-title":"On measures of entropy and information","volume":"Volume 4","year":"1961","journal-title":"Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","author":"Harremos","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_25","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., and Paul Smolley, S. (2017, January 22\u201329). Least squares generative adversarial networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.304"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST database of handwritten digit images for machine learning research","volume":"29","author":"Deng","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_28","unstructured":"Krizhevsky, A., and Hinton, G. (2024, February 22). Learning Multiple Layers of Features from Tiny Images 2009. Available online: https:\/\/www.cs.toronto.edu\/~kriz\/learning-features-2009-TR.pdf."},{"key":"ref_29","first-page":"1","article-title":"PacGAN: The power of two samples in generative adversarial networks","volume":"Volume 31","author":"Lin","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_30","unstructured":"Kingma, D., and Ba, J. (2014, January 14\u201316). Adam: A method for stochastic optimization. Proceedings of the International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_31","first-page":"1","article-title":"Improved training of Wasserstein GANs","volume":"Volume 30","author":"Gulrajani","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_32","first-page":"6626","article-title":"GANs trained by a two time-scale update rule converge to a local Nash equilibrium","volume":"Volume 30","author":"Heusel","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_33","unstructured":"Brock, A., Donahue, J., and Simonyan, K. (2018). Large scale GAN training for high fidelity natural image synthesis. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., and Aila, T. (2019, January 15\u201320). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00453"},{"key":"ref_35","unstructured":"Almahairi, A., Rajeshwar, S., Sordoni, A., Bachman, P., and Courville, A. (2018, January 10\u201315). Augmented CycleGAN: Learning many-to-many mappings from unpaired data. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/290\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:19:28Z","timestamp":1760105968000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/290"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,27]]},"references-count":35,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["e26040290"],"URL":"https:\/\/doi.org\/10.3390\/e26040290","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,27]]}}}