{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T10:31:09Z","timestamp":1779186669577,"version":"3.51.4"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T00:00:00Z","timestamp":1763510400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>\n                    Generative models are becoming a promising tool in AI alongside discriminative learning. Several models have been proposed to learn in an unsupervised fashion the corresponding generative factors, namely the latent variables critical for capturing the full spectrum of data variability. Diffusion Models (DMs), Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs) are of particular interest due to their impressive ability to generate highly realistic data. Through a systematic empirical study, this paper delves into the intricate challenge of how DMs, GANs and VAEs internalize and replicate\n                    <jats:italic>rare<\/jats:italic>\n                    generative factors. Our findings reveal a pronounced tendency toward memorization of these factors. We study the reasons for this memorization and demonstrate that strategies such as spectral decoupling can mitigate this issue to a certain extent.\n                    <jats:xref>\n                      <jats:sup>1<\/jats:sup>\n                    <\/jats:xref>\n                  <\/jats:p>","DOI":"10.3389\/frai.2025.1697139","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T06:35:48Z","timestamp":1763534148000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Do generative models learn rare generative factors?"],"prefix":"10.3389","volume":"8","author":[{"given":"Fasih","family":"Haider","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edward","family":"Moroshko","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuyang","family":"Xue","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sotirios A.","family":"Tsaftaris","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,19]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4611613","article-title":"Beware of diffusion models for synthesizing medical images-a comparison with gans in terms of memorizing brain tumor images","author":"Akbar","year":"2023","journal-title":"arXiv preprint arXiv:2305.07644"},{"key":"B2","article-title":"Invariant risk minimization","author":"Arjovsky","year":"2019","journal-title":"arXiv preprint arXiv:1907.02893"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: a review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B4","doi-asserted-by":"publisher","first-page":"7327","DOI":"10.1109\/TPAMI.2021.3116668","article-title":"Deep generative modelling: a comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models","volume":"44","author":"Bond-Taylor","year":"2022","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B5","article-title":"Extracting training data from diffusion models","author":"Carlini","year":"2023","journal-title":"arXiv preprint arXiv:2301.13188"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3929\/ethx-b-000391770","article-title":"Morpho-MNIST: quantitative assessment and diagnostics for representation learning","volume":"20","author":"Castro","year":"2019","journal-title":"J. Mach. Learn. Res"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1166120","DOI":"10.3389\/fpubh.2023.1166120","article-title":"Chatgpt and the rise of large language models: the new AI-driven infodemic threat in public health","volume":"11","author":"De Angelis","year":"2023","journal-title":"Front. Public Health"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlp.2023.100024","article-title":"An evaluation on large language model outputs: discourse and memorization","author":"de Wynter","year":"2023","journal-title":"arXiv preprint arXiv:2304.08637"},{"key":"B9","article-title":"Ganalyzer: analysis and manipulation of gans latent space for controllable face synthesis","author":"Fard","year":"2023","journal-title":"arXiv preprint arXiv:2302.00908"},{"key":"B10","doi-asserted-by":"publisher","first-page":"102787","DOI":"10.1016\/j.trc.2020.102787","article-title":"Prediction of rare feature combinations in population synthesis: application of deep generative modelling","volume":"120","author":"Garrido","year":"2020","journal-title":"Transport. Res. Part C"},{"key":"B11","article-title":"\u201cGenerative adversarial nets,\u201d","author":"Goodfellow","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B12","article-title":"Towards a definition of disentangled representations","author":"Higgins","year":"2018","journal-title":"arXiv preprint arXiv:1812.02230"},{"key":"B13","first-page":"6840","article-title":"\u201cDenoising diffusion probabilistic models,\u201d","author":"Ho","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B14","article-title":"OPT-IML: scaling language model instruction meta learning through the lens of generalization","author":"Iyer","year":"2022","journal-title":"arXiv preprint arXiv:2212.12017"},{"key":"B15","doi-asserted-by":"publisher","first-page":"9090","DOI":"10.1109\/TPAMI.2022.3229593","article-title":"Survey: leakage and privacy at inference time","volume":"45","author":"Jegorova","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813","article-title":"\u201cAnalyzing and improving the image quality of stylegan,\u201d","author":"Karras","year":"2020","journal-title":"2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"B18","doi-asserted-by":"publisher","first-page":"102516","DOI":"10.1016\/j.media.2022.102516","article-title":"Learning disentangled representations in the imaging domain","volume":"80","author":"Liu","year":"2022","journal-title":"Med. Image Anal"},{"key":"B19","article-title":"\u201cC-disentanglement: discovering causally-independent generative factors under an inductive bias of confounder,\u201d","author":"Liu","year":"2023","journal-title":"ICML 2023 Workshop on Structured Probabilistic Inference &Generative Modeling"},{"key":"B20","author":"Maini","year":"2023"},{"key":"B21","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1212\/CON.0000000000000687","article-title":"Early-onset alzheimer disease and its variants","volume":"25","author":"Mendez","year":"2019","journal-title":"Continuum"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.301","article-title":"\u201cDisentangling generative factors in natural language with discrete variational autoencoders,\u201d","author":"Mercatali","year":"2021","journal-title":"Conference on Empirical Methods in Natural Language Processing"},{"key":"B23","article-title":"Chatgpt in the age of generative AI and large language models: a concise survey","author":"Mohamadi","year":"2023","journal-title":"arXiv preprint arXiv:2307.04251"},{"key":"B24","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/s12471-022-01723-6","article-title":"Screening, diagnosis and follow-up of brugada syndrome in children: a dutch expert consensus statement","volume":"31","author":"Peltenburg","year":"2022","journal-title":"Neth. Heart J"},{"key":"B25","first-page":"1256","article-title":"\u201cGradient starvation: a learning proclivity in neural networks,\u201d","author":"Pezeshki","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B26","author":"Scao","year":"2022"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2018.00071","article-title":"\u201cThe riemannian geometry of deep generative models,\u201d","author":"Shao","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops"},{"key":"B28","doi-asserted-by":"publisher","first-page":"281","DOI":"10.3390\/children11030281","article-title":"Brugada syndrome: focus for the general pediatrician","volume":"11","author":"Speranzon","year":"2024","journal-title":"Children"},{"key":"B29","article-title":"Galactica: a large language model for science","author":"Taylor","year":"2022","journal-title":"arXiv preprint arXiv:2211.09085"},{"key":"B30","year":"2022","journal-title":"Chatgpt: Optimizing Language Models for Dialogue"},{"key":"B31","article-title":"Llama: open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"arXiv preprint arXiv:2302.13971"},{"key":"B32","first-page":"10401","article-title":"\u201cOn disentangled representations learned from correlated data,\u201d","author":"Tr\u00e4uble","year":"2021","journal-title":"Proceedings of the 38th International Conference on Machine Learning"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00297","article-title":"\u201cL2m-gan: learning to manipulate latent space semantics for facial attribute editing,\u201d","author":"Yang","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299023","article-title":"\u201cA large-scale car dataset for fine-grained categorization and verification,\u201d","author":"Yang","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01102","article-title":"\u201cStyleswin: Transformer-based gan for high-resolution image generation,\u201d","author":"Zhang","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B36","article-title":"\u201cIdentifiability guarantees for causal disentanglement from soft interventions,\u201d","author":"Zhang","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B37","article-title":"Opt: Open pre-trained transformer language models","author":"Zhang","year":"2022","journal-title":"arXiv preprint arXiv:2205.01068"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1697139\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,21]],"date-time":"2025-11-21T13:05:18Z","timestamp":1763730318000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1697139\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,19]]},"references-count":37,"alternative-id":["10.3389\/frai.2025.1697139"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1697139","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,19]]},"article-number":"1697139"}}