{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T13:24:25Z","timestamp":1781616265219,"version":"3.54.5"},"reference-count":57,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2023,12,14]],"date-time":"2023-12-14T00:00:00Z","timestamp":1702512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Humans are able to quickly adapt to new situations, learn effectively with limited data, and create unique combinations of basic concepts. In contrast, generalizing out-of-distribution (OOD) data and achieving combinatorial generalizations are fundamental challenges for machine learning models. Moreover, obtaining high-quality labeled examples can be very time-consuming and expensive, particularly when specialized skills are required for labeling. To address these issues, we propose BtVAE, a method that utilizes conditional VAE models to achieve combinatorial generalization in certain scenarios and consequently to generate out-of-distribution (OOD) data in a semi-supervised manner. Unlike previous approaches that use new factors of variation during testing, our method uses only existing attributes from the training data but in ways that were not seen during training (e.g., small objects of a specific shape during training and large objects of the same shape during testing).<\/jats:p>","DOI":"10.3390\/e25121659","type":"journal-article","created":{"date-parts":[[2023,12,15]],"date-time":"2023-12-15T03:16:33Z","timestamp":1702610193000},"page":"1659","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Semi-Supervised Variational Autoencoders for Out-of-Distribution Generation"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2868-8380","authenticated-orcid":false,"given":"Frantzeska","family":"Lavda","sequence":"first","affiliation":[{"name":"Geneva School of Business Administration (DMML Group), University of Applied Sciences and Arts Western Switzerland (HES-SO), 1227 Geneva, Switzerland"},{"name":"Faculty of Science, Computer Science Department, University of Geneva, 1214 Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6282-0686","authenticated-orcid":false,"given":"Alexandros","family":"Kalousis","sequence":"additional","affiliation":[{"name":"Geneva School of Business Administration (DMML Group), University of Applied Sciences and Arts Western Switzerland (HES-SO), 1227 Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,14]]},"reference":[{"key":"ref_1","unstructured":"Von Humboldt, W., and Freiherr von Humboldt, W.F. (1999). On Language: On the Diversity of Human Language Construction and Its Influence on the Mental Development of the Human Species, Cambridge University Press."},{"key":"ref_2","unstructured":"Chomsky, N. (2014). Aspects of the Theory of Syntax, MIT Press."},{"key":"ref_3","unstructured":"Battaglia, P.W., Hamrick, J.B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., and Faulkner, R. (2018). Relational inductive biases, deep learning, and graph networks. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2017","journal-title":"Behav. Brain Sci."},{"key":"ref_5","unstructured":"Bengio, Y. (2013). Proceedings of the International Conference on Statistical Language and Speech Processing, Springer."},{"key":"ref_6","unstructured":"Odena, A. (2018, January 7). How Good is the Generator in Adversarial Training at Generating Hard Examples?. Proceedings of the NeurIPS 2018 Workshop on Security in Machine Learning, Montreal, QC, Canada."},{"key":"ref_7","unstructured":"Lee, K., Lee, K., Min, H., Zhang, J., Shin, J., and Lee, S.J. (2018, January 2\u20138). A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Arazo, E., Ortego, D., Albert, P., O\u2019Connor, N.E., and McGuinness, K. (2020, January 13\u201319). Pseudo-labeling and confirmation bias in deep semi-supervised learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/IJCNN48605.2020.9207304"},{"key":"ref_9","unstructured":"Berthelot, D., Carlini, N., Goodfellow, I., Papernot, N., Oliver, A., and Raffel, C.A. (2019, January 8\u201314). Mixmatch: A holistic approach to semi-supervised learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1979","DOI":"10.1109\/TPAMI.2018.2858821","article-title":"Virtual adversarial training: A regularization method for supervised and semi-supervised learning","volume":"41","author":"Miyato","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","unstructured":"Sajjadi, M., Javanmardi, M., and Tasdizen, T. (2016, January 5\u201313). Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_12","unstructured":"Tarvainen, A., and Valpola, H. (2017, January 4\u20139). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_13","unstructured":"Kingma, D.P., Mohamed, S., Jimenez Rezende, D., and Welling, M. (2014, January 9\u201314). Semi-supervised learning with deep generative models. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_14","unstructured":"Precup, D., and Teh, Y.W. (2017, January 6\u201311). Conditional Image Synthesis with Auxiliary Classifier GANs. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia. Proceedings of Machine Learning Research."},{"key":"ref_15","unstructured":"Locatello, F., Tschannen, M., Bauer, S., R\u00e4tsch, G., Sch\u00f6lkopf, B., and Bachem, O. (2020, January 26\u201330). Disentangling Factors of Variations Using Few Labels. Proceedings of the International Conference on Learning Representations, Virtual."},{"key":"ref_16","unstructured":"III, H.D., and Singh, A. (2020, January 13\u201318). Semi-Supervised StyleGAN for Disentanglement Learning. Proceedings of the 37th International Conference on Machine Learning, Virtual. PMLR; Proceedings of Machine Learning Research."},{"key":"ref_17","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_18","first-page":"6256","article-title":"Unsupervised Data Augmentation for Consistency Training","volume":"Volume 33","author":"Larochelle","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_19","unstructured":"Sricharan, K., and Srivastava, A. (2018, January 7). Building robust classifiers through generation of confident out of distribution examples. Proceedings of the Workshop on Bayesian Deep Learning, NeurIPS 2018, Montreal, QC, Canada."},{"key":"ref_20","unstructured":"Lee, K., Lee, H., Lee, K., and Shin, J. (May, January 30). Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_21","unstructured":"Vernekar, S., Gaurav, A., Abdelzad, V., Denouden, T., Salay, R., and Czarnecki, K. (2019). Out-of-distribution detection in classifiers via generation. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.neunet.2021.10.020","article-title":"Detecting out-of-distribution samples via variational auto-encoder with reliable uncertainty estimation","volume":"145","author":"Ran","year":"2022","journal-title":"Neural Netw."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Marek, P., Naik, V.I., Auvray, V., and Goyal, A. (2021). Oodgan: Generative adversarial network for out-of-domain data generation. arXiv.","DOI":"10.18653\/v1\/2021.naacl-industry.30"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., and Aila, T. (2019, January 15\u201320). A style-based generator architecture for generative adversarial networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00453"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18\u201324). High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_26","unstructured":"Wu, Z., Nitzan, Y., Shechtman, E., and Lischinski, D. (2021). Stylealign: Analysis and applications of aligned stylegan models. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1021\/acscentsci.7b00572","article-title":"Automatic chemical design using a data-driven continuous representation of molecules","volume":"4","author":"Wei","year":"2018","journal-title":"ACS Cent. Sci."},{"key":"ref_28","unstructured":"Kusner, M.J., Paige, B., and Hern\u00e1ndez-Lobato, J.M. (2017, January 6\u201311). Grammar variational autoencoder. Proceedings of the International Conference on Machine Learning PMLR, Sydney, Australia."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lee, K., Lee, Y., Ko, H.H., and Kang, M. (2022). A Study on the Channel Expansion VAE for Content-Based Image Retrieval. Appl. Sci., 12.","DOI":"10.3390\/app12189160"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rafiei, M., and Iosifidis, A. (2023). Class-Specific Variational Auto-Encoder for Content-Based Image Retrieval. arXiv.","DOI":"10.1109\/IJCNN54540.2023.10191068"},{"key":"ref_31","unstructured":"Rey, L.A.P., Holenderski, M.J., and Jarnikov, D.S. (2021, January 6\u201314). Content-based image retrieval from weakly-supervised disentangled representations. Proceedings of the NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, Virtual."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Shen, X., Su, H., Niu, S., and Demberg, V. (2018, January 2\u20137). Improving variational encoder-decoders in dialogue generation. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11960"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sun, B., Feng, S., Li, Y., Liu, J., and Li, K. (2021). Generating relevant and coherent dialogue responses using self-separated conditional variational autoencoders. arXiv.","DOI":"10.18653\/v1\/2021.acl-long.437"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mak, H.W.L., Han, R., and Yin, H.H. (2023). Application of variational autoEncoder (VAE) model and image processing approaches in game design. Sensors, 23.","DOI":"10.20944\/preprints202303.0023.v1"},{"key":"ref_35","unstructured":"Higgins, I., Sonnerat, N., Matthey, L., Pal, A., Burgess, C.P., Bo\u0161njak, M., Shanahan, M., Botvinick, M., Hassabis, D., and Lerchner, A. (May, January 30). SCAN: Learning Hierarchical Compositional Visual Concepts. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_36","unstructured":"Watters, N., Matthey, L., Burgess, C.P., and Lerchner, A. (2019). Spatial Broadcast Decoder: A Simple Architecture for Disentangled Representations in VAEs. arXiv."},{"key":"ref_37","unstructured":"Dittadi, A., Tr\u00e4uble, F., Locatello, F., W\u00fcthrich, M., Agrawal, V., Winther, O., Bauer, S., and Sch\u00f6lkopf, B. (2021, January 3\u20137). On the transfer of disentangled representations in realistic settings. Proceedings of the International Conference on Learning Representations 2021, Virtual."},{"key":"ref_38","unstructured":"Montero, M.L., Ludwig, C.J., Costa, R.P., Malhotra, G., and Bowers, J. (2021, January 3\u20137). The role of disentanglement in generalisation. Proceedings of the International Conference on Learning Representations 2021, Virtual."},{"key":"ref_39","unstructured":"Montero, M.L., Bowers, J., Costa, R.P., Ludwig, C.J., and Malhotra, G. (December, January 28). Lost in Latent Space: Examining failures of disentangled models at combinatorial generalisation. Proceedings of the Advances in Neural Information Processing Systems NeurIPS 2022, New Orleans, LA, USA."},{"key":"ref_40","unstructured":"Schott, L., von K\u00fcgelgen, J., Tr\u00e4uble, F., Gehler, P., Russell, C., Bethge, M., Sch\u00f6lkopf, B., Locatello, F., and Brendel, W. (2022, January 7\u201311). Visual Representation Learning Does Not Generalize Strongly Within the Same Domain. Proceedings of the 10th International Conference on Learning Representations (ICLR), Virtual."},{"key":"ref_41","first-page":"2060","article-title":"Learning disentangled semantic representation for domain adaptation","volume":"Volume 2019","author":"Cai","year":"2019","journal-title":"IJCAI: Proceedings of the Conference"},{"key":"ref_42","unstructured":"Zhu, X.J. (2005). Semi-Supervised Learning Literature Survey, University of Wisconsin-Madison, Department of Computer Sciences. Technical Report."},{"key":"ref_43","unstructured":"Lee, D.H. (2013, January 16\u201321). Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. Proceedings of the Workshop on Challenges in Representation Learning ICML, Atlanta, GA, USA."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1080\/01621459.1975.10479874","article-title":"Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis","volume":"70","author":"McLachlan","year":"1975","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_45","unstructured":"Bengio, Y., and LeCun, Y. (2014, January 14\u201316). Auto-Encoding Variational Bayes. Proceedings of the 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada. Conference Track Proceedings."},{"key":"ref_46","unstructured":"Rezende, D.J., Mohamed, S., and Wierstra, D. (2014, January 21\u201326). Stochastic backpropagation and variational inference in deep latent gaussian models. Proceedings of the International Conference on Machine Learning, Beijing, China."},{"key":"ref_47","unstructured":"Sohn, K., Lee, H., and Yan, X. (2015, January 7\u201312). Learning structured output representation using deep conditional generative models. Proceedings of the Advances in Neural Information Processing Systems 2015, Montreal, QC, Canada."},{"key":"ref_48","unstructured":"Cemgil, T., Ghaisas, S., Dvijotham, K.D., and Kohli, P. (2020, January 26\u201330). Adversarially robust representations with smooth encoders. Proceedings of the International Conference on Learning Representations 2020, Virtual."},{"key":"ref_49","first-page":"20685","article-title":"Likelihood regret: An out-of-distribution detection score for variational auto-encoder","volume":"33","author":"Xiao","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_50","unstructured":"Hendrycks, D., and Gimpel, K. (2016). A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. arXiv."},{"key":"ref_51","unstructured":"Matthey, L., Higgins, I., Hassabis, D., and Lerchner, A. (2023, June 01). dSprites: Disentanglement Testing Sprites Dataset. Available online: https:\/\/github.com\/deepmind\/dsprites-dataset\/."},{"key":"ref_52","unstructured":"Burgess, C., and Kim, H. (2023, June 01). 3D Shapes Dataset. Available online: https:\/\/github.com\/deepmind\/3dshapes-dataset\/."},{"key":"ref_53","unstructured":"LeCun, Y., Cortes, C., and Burges, C. (2023, June 01). MNIST Handwritten Digit Database. 2010. Volume 2. ATT Labs. Available online: http:\/\/yann.lecun.com\/exdb\/mnist."},{"key":"ref_54","unstructured":"Bengio, S., Wallach, H.M., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018, January 3\u20138). Learning Latent Subspaces in Variational Autoencoders. Proceedings of the Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montreal, BC, USA."},{"key":"ref_55","unstructured":"Guo, X., Du, Y., and Zhao, L. (2022, January 3\u20137). Property Controllable Variational Autoencoder via Invertible Mutual Dependence. Proceedings of the International Conference on Learning Representations 2021, Virtual."},{"key":"ref_56","unstructured":"Li, X., Lin, C., Li, R., Wang, C., and Guerin, F. (2020, January 13\u201318). Latent space factorisation and manipulation via matrix subspace projection. Proceedings of the International Conference on Machine Learning PMLR, Virtual."},{"key":"ref_57","unstructured":"Xu, Z., Niethamme, M., and Raffel, C. (2022, January 28). Compositional Generalization in Unsupervised Compositional Representation Learning: A Study on Disentanglement and Emergent Language. Proceedings of the Conference on Neural Information Processing Systems 2022, New Orleans, LA, USA."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/12\/1659\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:38:57Z","timestamp":1760132337000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/12\/1659"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,14]]},"references-count":57,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["e25121659"],"URL":"https:\/\/doi.org\/10.3390\/e25121659","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,14]]}}}