{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T08:10:19Z","timestamp":1759133419597,"version":"3.41.0"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T00:00:00Z","timestamp":1597190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>The emergence of deep generative models has recently enabled the automatic generation of massive amounts of graphical content, both in 2D and in 3D. Generative Adversarial Networks (GANs) and style control mechanisms, such as Adaptive Instance Normalization (AdaIN), have proved particularly effective in this context, culminating in the state-of-the-art StyleGAN architecture. While such models are able to learn diverse distributions, provided a sufficiently large training set, they are not well-suited for scenarios where the distribution of the training data exhibits a multi-modal behavior. In such cases, reshaping a uniform or normal distribution over the latent space into a complex multi-modal distribution in the data domain is challenging, and the generator might fail to sample the target distribution well. Furthermore, existing unsupervised generative models are not able to control the mode of the generated samples independently of the other visual attributes, despite the fact that they are typically disentangled in the training data.<\/jats:p>\n          <jats:p>\n            In this paper, we introduce uMM-GAN, a novel architecture designed to better model multi-modal distributions, in an unsupervised fashion. Building upon the StyleGAN architecture, our network learns multiple modes, in a\n            <jats:italic toggle=\"yes\">completely unsupervised manner<\/jats:italic>\n            , and combines them using a set of learned weights. We demonstrate that this approach is capable of effectively approximating a complex distribution as a superposition of multiple simple ones. We further show that uMM-GAN effectively disentangles between modes and style, thereby providing an independent degree of control over the generated content.\n          <\/jats:p>","DOI":"10.1145\/3386569.3392454","type":"journal-article","created":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T11:44:27Z","timestamp":1597232667000},"update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Unsupervised\n            <i>K<\/i>\n            -modal styled content generation"],"prefix":"10.1145","volume":"39","author":[{"given":"Omry","family":"Sendik","sequence":"first","affiliation":[{"name":"Tel Aviv University, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dani","family":"Lischinski","sequence":"additional","affiliation":[{"name":"The Hebrew University of Jerusalem, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Cohen-Or","sequence":"additional","affiliation":[{"name":"Tel Aviv University, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,12]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00453"},{"key":"e_1_2_2_2_1","volume-title":"Gaussian mixture generative adversarial networks for diverse datasets, and the unsupervised clustering of images. arXiv preprint arXiv:1808.10356","author":"Ben-Yosef Matan","year":"2018","unstructured":"Matan Ben-Yosef and Daphna Weinshall. 2018. Gaussian mixture generative adversarial networks for diverse datasets, and the unsupervised clustering of images. arXiv preprint arXiv:1808.10356 (2018)."},{"key":"e_1_2_2_3_1","volume-title":"Large Scale GAN Training for High Fidelity Natural Image Synthesis. ICLR","author":"Brock Andrew","year":"2019","unstructured":"Andrew Brock, Jeff Donahue, and Karen Simonyan. 2019. Large Scale GAN Training for High Fidelity Natural Image Synthesis. ICLR (2019)."},{"key":"e_1_2_2_4_1","volume-title":"Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders. CoRR abs\/1611.02648","author":"Dilokthanakul Nat","year":"2016","unstructured":"Nat Dilokthanakul, Pedro A. M. Mediano, Marta Garnelo, Matthew C. H. Lee, Hugh Salimbeni, Kai Arulkumaran, and Murray Shanahan. 2016. Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders. CoRR abs\/1611.02648 (2016). arXiv:1611.02648 http:\/\/arxiv.org\/abs\/1611.02648"},{"key":"e_1_2_2_5_1","volume-title":"Style generator inversion for image enhancement and animation. arXiv:1906.11880","author":"Gabbay Aviv","year":"2019","unstructured":"Aviv Gabbay and Yedid Hoshen. 2019. Style generator inversion for image enhancement and animation. arXiv:1906.11880 (2019)."},{"key":"e_1_2_2_6_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems. 2672--2680."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.525"},{"key":"e_1_2_2_8_1","unstructured":"Martin Heusel Hubert Ramsauer Thomas Unterthiner Bernhard Nessler and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems. 6626--6637."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.167"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_11"},{"key":"e_1_2_2_11_1","volume-title":"Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196","author":"Karras Tero","year":"2017","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"volume-title":"Advances in Neural Information Processing Systems.","author":"Khayatkhoei Mahyar","key":"e_1_2_2_13_1","unstructured":"Mahyar Khayatkhoei, Maneesh K. Singh, and Ahmed Elgammal. 2018. Disconnected Manifold Learning for Generative Adversarial Networks. In Advances in Neural Information Processing Systems. Vol. 31. 7354--7364."},{"key":"e_1_2_2_14_1","volume-title":"Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems. 10215--10224.","author":"Kingma Durk P","year":"2018","unstructured":"Durk P Kingma and Prafulla Dhariwal. 2018. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems. 10215--10224."},{"volume-title":"Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1312","author":"Diederik","key":"e_1_2_2_15_1","unstructured":"Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1312.6114"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.77"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01246-5_3"},{"key":"e_1_2_2_18_1","volume-title":"Maneesh Kumar Singh, and Ming-Hsuan Yang","author":"Lee Hsin-Ying","year":"2019","unstructured":"Hsin-Ying Lee, Hung-Yu Tseng, Qi Mao, Jia-Bin Huang, Yu-Ding Lu, Maneesh Kumar Singh, and Ming-Hsuan Yang. 2019. DRIT++: Diverse image-to-image translation via disentangled representations. arXiv preprint arXiv:1905.01270 (2019)."},{"key":"e_1_2_2_19_1","volume-title":"Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784","author":"Mirza Mehdi","year":"2014","unstructured":"Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014610"},{"key":"e_1_2_2_21_1","unstructured":"Augustus Odena Christopher Olah and Jonathon Shlens. 2016. Conditional Image Synthesis With Auxiliary Classifier GANs. (2016). arXiv:stat.ML\/1610.09585"},{"key":"e_1_2_2_22_1","volume-title":"MMGAN: Generative Adversarial Networks for Multi-Modal Distributions.","author":"Pandeva Teodora","year":"2019","unstructured":"Teodora Pandeva and Matthias Schubert. 2019. MMGAN: Generative Adversarial Networks for Multi-Modal Distributions. (2019). arXiv:cs.LG\/1911.06663"},{"key":"e_1_2_2_23_1","volume-title":"Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv.org (Nov","author":"Radford Alec","year":"2015","unstructured":"Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. arXiv.org (Nov. 2015). arXiv:cs.LG\/1511.06434v2"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_2_2_25_1","volume-title":"Style-Based GAN in PyTorch. https:\/\/github.com\/rosinality\/style-based-gan-pytorch\/. [Online","author":"Seonghyeon Kim","year":"2020","unstructured":"Kim Seonghyeon. 2020. Style-Based GAN in PyTorch. https:\/\/github.com\/rosinality\/style-based-gan-pytorch\/. [Online; accessed 9-April-2020]."},{"key":"e_1_2_2_26_1","first-page":"2269","article-title":"BourGAN: Generative networks with metric embeddings","volume":"32","author":"Xiao Chang","year":"2018","unstructured":"Chang Xiao, Peilin Zhong, and Changxi Zheng. 2018. BourGAN: Generative networks with metric embeddings. In Advances in Neural Information Processing Systems, Vol. 32. 2269--2280.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_27_1","volume-title":"LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365","author":"Yu Fisher","year":"2015","unstructured":"Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. 2015. LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365 (2015)."},{"key":"e_1_2_2_28_1","first-page":"465","article-title":"Toward multimodal image-to-image translation","volume":"31","author":"Zhu Jun-Yan","year":"2017","unstructured":"Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Darrell, Alexei A Efros, Oliver Wang, and Eli Shechtman. 2017. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems, Vol. 31. 465--476.","journal-title":"Advances in Neural Information Processing Systems"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392454","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3386569.3392454","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T05:40:37Z","timestamp":1750830037000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392454"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,12]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3386569.3392454"],"URL":"https:\/\/doi.org\/10.1145\/3386569.3392454","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"type":"print","value":"0730-0301"},{"type":"electronic","value":"1557-7368"}],"subject":[],"published":{"date-parts":[[2020,8,12]]},"assertion":[{"value":"2020-08-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}