{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:11:57Z","timestamp":1760112717693,"version":"build-2065373602"},"reference-count":72,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T00:00:00Z","timestamp":1722988800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Center of Advanced Systems Understanding (CASUS)"},{"name":"Germany\u2019s Federal Ministry of Education and Research (BMBF)"},{"name":"Saxon Ministry for Science, Culture and Tourism (SMWK)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>We formulate a data-independent latent space regularization constraint for general unsupervised autoencoders. The regularization relies on sampling the autoencoder Jacobian at Legendre nodes, which are the centers of the Gauss\u2013Legendre quadrature. Revisiting this classic allows us to prove that regularized autoencoders ensure a one-to-one re-embedding of the initial data manifold into its latent representation. Demonstrations show that previously proposed regularization strategies, such as contractive autoencoding, cause topological defects even in simple examples, as do convolutional-based (variational) autoencoders. In contrast, topological preservation is ensured by standard multilayer perceptron neural networks when regularized using our approach. This observation extends from the classic FashionMNIST dataset to (low-resolution) MRI brain scans, suggesting that reliable low-dimensional representations of complex high-dimensional datasets can be achieved using this regularization technique.<\/jats:p>","DOI":"10.3390\/axioms13080535","type":"journal-article","created":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T11:33:52Z","timestamp":1723030432000},"page":"535","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Ensuring Topological Data-Structure Preservation under Autoencoder Compression Due to Latent Space Regularization in Gauss\u2013Legendre Nodes"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-6718-2665","authenticated-orcid":false,"given":"Chethan Krishnamurthy","family":"Ramanaik","sequence":"first","affiliation":[{"name":"Forschungsinstitut CODE, University of the Bundeswehr Munich, 85579 Neubiberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9992-6027","authenticated-orcid":false,"given":"Anna","family":"Willmann","sequence":"additional","affiliation":[{"name":"CASUS\u2014Center for Advanced Systems Understanding, Helmholtz-Zentrum Dresden-Rossendorf e.V. (HZDR), 01328 Dresden, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1990-2413","authenticated-orcid":false,"given":"Juan-Esteban","family":"Suarez Cardona","sequence":"additional","affiliation":[{"name":"CASUS\u2014Center for Advanced Systems Understanding, Helmholtz-Zentrum Dresden-Rossendorf e.V. (HZDR), 01328 Dresden, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pia","family":"Hanfeld","sequence":"additional","affiliation":[{"name":"CASUS\u2014Center for Advanced Systems Understanding, Helmholtz-Zentrum Dresden-Rossendorf e.V. (HZDR), 01328 Dresden, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6788-1047","authenticated-orcid":false,"given":"Nico","family":"Hoffmann","sequence":"additional","affiliation":[{"name":"SAXONY.ai, 01217 Dresden, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9214-8253","authenticated-orcid":false,"given":"Michael","family":"Hecht","sequence":"additional","affiliation":[{"name":"CASUS\u2014Center for Advanced Systems Understanding, Helmholtz-Zentrum Dresden-Rossendorf e.V. (HZDR), 01328 Dresden, Germany"},{"name":"Mathematical Institute, University of Wroc\u0142aw, 50-384 Wroc\u0142aw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"690","DOI":"10.1038\/nrm1979","article-title":"High-throughput fluorescence microscopy for systems biology","volume":"7","author":"Pepperkok","year":"2006","journal-title":"Nat. Rev. Mol. Cell Biol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1194","DOI":"10.1126\/science.1100709","article-title":"Multidimensional drug profiling by automated microscopy","volume":"306","author":"Perlman","year":"2004","journal-title":"Science"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1038\/nmeth.4549","article-title":"Machine learning in neuroscience","volume":"15","author":"Vogt","year":"2018","journal-title":"Nat. Methods"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.neuroimage.2017.08.019","article-title":"Ghosts in machine learning for cognitive neuroscience: Moving from data to theory","volume":"180","author":"Carlson","year":"2018","journal-title":"NeuroImage"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"101761","DOI":"10.1016\/j.media.2020.101761","article-title":"Deep white matter analysis (DeepWMA): Fast and consistent tractography segmentation","volume":"65","author":"Zhang","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1038\/s42254-021-00314-5","article-title":"Physics-informed machine learning","volume":"3","author":"Karniadakis","year":"2021","journal-title":"Nat. Rev. Phys."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"790","DOI":"10.1038\/s41567-019-0512-x","article-title":"Identifying topological order through unsupervised machine learning","volume":"15","author":"Scheurer","year":"2019","journal-title":"Nat. Phys."},{"key":"ref_8","unstructured":"Willmann, A., Stiller, P., Debus, A., Irman, A., Pausch, R., Chang, Y.Y., Bussmann, M., and Hoffmann, N. (2021). Data-Driven Shadowgraph Simulation of a 3D Object. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"995","DOI":"10.1038\/s41592-022-01541-z","article-title":"Self-supervised deep learning encodes high-resolution features of protein subcellular localization","volume":"19","author":"Kobayashi","year":"2022","journal-title":"Nat. Methods"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1038\/s41573-020-00117-w","article-title":"Image-based profiling for drug discovery: Due for a machine-learning upgrade?","volume":"20","author":"Chandrasekaran","year":"2021","journal-title":"Nat. Rev. Drug Discov."},{"key":"ref_11","first-page":"5079","article-title":"A high-throughput siRNA screen identifies genes that regulate mannose 6-phosphate receptor trafficking","volume":"127","author":"Anitei","year":"2014","journal-title":"J. Cell Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Nikitina, K., Segeletz, S., Kuhn, M., Kalaidzidis, Y., and Zerial, M. (2019, January 17\u201319). Basic Phenotypes of Endocytic System Recognized by Independent Phenotypes Analysis of a High-throughput Genomic Screen. Proceedings of the 2019 3rd International Conference on Computational Biology and Bioinformatics, Nagoya, Japan.","DOI":"10.1145\/3365966.3365972"},{"key":"ref_13","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, Munich, Germany, 5\u20139 October 2015, Springer. Proceedings, Part III 18."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1665","DOI":"10.18632\/aging.203916","article-title":"A tandem segmentation-classification approach for the localization of morphological predictors of C. elegans lifespan and motility","volume":"14","author":"Galimov","year":"2022","journal-title":"Aging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e00836-20","DOI":"10.1128\/mSphere.00836-20","article-title":"Mimicry embedding facilitates advanced neural network training for image-based pathogen detection","volume":"5","author":"Yakimovich","year":"2020","journal-title":"Msphere"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Fisch, D., Yakimovich, A., Clough, B., Mercer, J., and Frickel, E.M. (2020). Image-Based Quantitation of Host Cell\u2013Toxoplasma gondii Interplay Using HRMAn: A Host Response to Microbe Analysis Pipeline. Toxoplasma gondii: Methods and Protocols, Humana.","DOI":"10.1007\/978-1-4939-9857-9_21"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"102543","DOI":"10.1016\/j.isci.2021.102543","article-title":"Microscopy deep learning predicts virus infections and reveals mechanics of lytic-infected cells","volume":"24","author":"Andriasyan","year":"2021","journal-title":"Iscience"},{"key":"ref_18","unstructured":"S\u00e1nchez, J., Mardia, K., Kent, J., and Bibby, J. (1979). Multivariate Analysis, Academic Press."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Dunteman, G.H. (1989). Basic concepts of principal components analysis. Principal Components Analysis, SAGE Publications Ltd.","DOI":"10.4135\/9781412985475"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Krzanowski, W. (2000). Principles of Multivariate Analysis, OUP Oxford.","DOI":"10.1093\/oso\/9780198507086.001.0001"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rolinek, M., Zietlow, D., and Martius, G. (2019, January 15\u201320). Variational Autoencoders Pursue PCA Directions (by Accident). Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01269"},{"key":"ref_22","first-page":"3","article-title":"Deep learning in scientific computing: Understanding the instability mystery","volume":"54","author":"Antun","year":"2021","journal-title":"SIAM News"},{"key":"ref_23","unstructured":"Gottschling, N.M., Antun, V., Adcock, B., and Hansen, A.C. (2020). The troublesome kernel: Why deep learning for inverse problems is typically unstable. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"30088","DOI":"10.1073\/pnas.1907377117","article-title":"On instabilities of deep learning in image reconstruction and the potential costs of AI","volume":"117","author":"Antun","year":"2020","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_25","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). Robustness Verification of Tree-based Models. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Galhotra, S., Brun, Y., and Meliou, A. (2017, January 4\u20138). Fairness testing: Testing software for discrimination. Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Paderborn, Germany.","DOI":"10.1145\/3106237.3106277"},{"key":"ref_27","unstructured":"Mazzucato, D., and Urban, C. (2021). Reduced products of abstract domains for fairness certification of neural networks. Proceedings of the Static Analysis: 28th International Symposium, SAS 2021, Chicago, IL, USA, 17\u201319 October 2021, Springer. Proceedings 28."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lang, S. (1985). Differential Manifolds, Springer.","DOI":"10.1007\/978-1-4684-0265-0"},{"key":"ref_29","first-page":"xiv","article-title":"The implicit function theorem. Modern Birkh\u00e4user Classics","volume":"Volume 163","author":"Krantz","year":"2013","journal-title":"History, Theory, and Applications"},{"key":"ref_30","unstructured":"Stroud, A. (1971). Approximate Calculation of Multiple Integrals: Prentice-Hall Series in Automatic Computation, Prentice-Hall."},{"key":"ref_31","unstructured":"Stroud, A., and Secrest, D. (2011). Gaussian Quadrature Formulas, Prentice-Hall."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4837","DOI":"10.1090\/proc\/13623","article-title":"Multivariate polynomial approximation in the hypercube","volume":"145","author":"Trefethen","year":"2017","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Trefethen, L.N. (2019). Approximation Theory and Approximation Practice, SIAM.","DOI":"10.1137\/1.9781611975949"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Sobolev, S.L., and Vaskevich, V. (1997). The Theory of Cubature Formulas, Springer Science & Business Media.","DOI":"10.1007\/978-94-015-8913-0"},{"key":"ref_35","unstructured":"Veettil, S.K.T., Zheng, Y., Acosta, U.H., Wicaksono, D., and Hecht, M. (2022). Multivariate Polynomial Regression of Euclidean Degree Extends the Stability for Fast Approximations of Trefethen Functions. arXiv."},{"key":"ref_36","unstructured":"Suarez Cardona, J.E., Hofmann, P.A., and Hecht, M. (2023). Learning Partial Differential Equations by Spectral Approximates of General Sobolev Spaces. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Suarez Cardona, J.E., and Hecht, M. (2022). Replacing Automatic Differentiation by Sobolev Cubatures fastens Physics Informed Neural Nets and strengthens their Approximation Power. arXiv.","DOI":"10.1088\/2632-2153\/acf97a"},{"key":"ref_38","unstructured":"Hecht, M., Cheeseman, B.L., Hoffmann, K.B., and Sbalzarini, I.F. (2017). A Quadratic-Time Algorithm for General Multivariate Polynomial Interpolation. arXiv."},{"key":"ref_39","unstructured":"Hecht, M., Hoffmann, K.B., Cheeseman, B.L., and Sbalzarini, I.F. (2018). Multivariate Newton Interpolation. arXiv."},{"key":"ref_40","unstructured":"Hecht, M., Gonciarz, K., Michelfeit, J., Sivkin, V., and Sbalzarini, I.F. (2020). Multivariate Interpolation in Unisolvent Nodes\u2013Lifting the Curse of Dimensionality. arXiv."},{"key":"ref_41","first-page":"53","article-title":"Fast Interpolation and Fourier Transform in High-Dimensional Spaces","volume":"Volume 857","author":"Arai","year":"2018","journal-title":"Proceedings of the Intelligent Computing"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kumar, L.A., Jayashree, L.S., and Manimegalai, R. (2020). A Survey on Supervised and Unsupervised Learning Techniques. Proceedings of the International Conference on Artificial Intelligence, Smart Grid and Smart City Applications, Springer International Publishing.","DOI":"10.1007\/978-3-030-24051-6"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"341","DOI":"10.3390\/make1010020","article-title":"Recent Advances in Supervised Dimension Reduction: A Survey","volume":"1","author":"Chao","year":"2019","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_44","unstructured":"Mitchell, T.M. (1980). The Need for Biases in Learning Generalizations, Citeseer."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/BF00993472","article-title":"Evaluation and selection of biases in machine learning","volume":"20","author":"Gordon","year":"1995","journal-title":"Mach. Learn."},{"key":"ref_46","unstructured":"Wu, H., and Flierl, M. (2020, January 7\u201312). Vector quantization-based regularization for autoencoders. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_47","first-page":"6306","article-title":"Neural discrete representation learning","volume":"30","author":"Vinyals","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Rifai, S., Mesnil, G., Vincent, P., Muller, X., Bengio, Y., Dauphin, Y., and Glorot, X. (2011, January 5\u20139). Higher order contractive auto-encoder. Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Athens, Greece.","DOI":"10.1007\/978-3-642-23783-6_41"},{"key":"ref_49","unstructured":"Rifai, S., Vincent, P., Muller, X., Glorot, X., and Bengio, Y. (July, January 28). Contractive auto-encoders: Explicit invariance during feature extraction. Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML), Bellevue, WA, USA."},{"key":"ref_50","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_51","unstructured":"Burgess, C.P., Higgins, I., Pal, A., Matthey, L., Watters, N., Desjardins, G., and Lerchner, A. (2018). Understanding disentangling in \u03b2-VAE. arXiv."},{"key":"ref_52","unstructured":"Kumar, A., and Poole, B. (2020, January 12\u201318). On implicit regularization in \u03b2-VAEs. Proceedings of the 37th International Conference on Machine Learning (ICML 2020), Vienna, Austria."},{"key":"ref_53","first-page":"22708","article-title":"Local Disentanglement in Variational Auto-Encoders Using Jacobian L_1 Regularization","volume":"34","author":"Rhodes","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Gilbert, A.C., Zhang, Y., Lee, K., Zhang, Y., and Lee, H. (2017, January 19\u201325). Towards understanding the invertibility of convolutional neural networks. Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Australia.","DOI":"10.24963\/ijcai.2017\/236"},{"key":"ref_55","unstructured":"Anthony, M., and Bartlett, P.L. (2009). Neural Network Learning: Theoretical Foundations, Cambridge University Press."},{"key":"ref_56","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_57","unstructured":"Adams, R.A., and Fournier, J.J. (2003). Sobolev Spaces, Academic Press."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Brezis, H. (2011). Functional Analysis, Sobolev Spaces and Partial Differential Equations, Springer.","DOI":"10.1007\/978-0-387-70914-7"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Pedersen, M. (2018). Functional Analysis in Applied Mathematics and Engineering, CRC Press.","DOI":"10.1201\/9780203755426"},{"key":"ref_60","unstructured":"Gautschi, W. (2011). Numerical Analysis, Springer Science & Business Media."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Chen, W., Chern, S.S., and Lam, K.S. (1999). Lectures on Differential Geometry, World Scientific Publishing Company.","DOI":"10.1142\/3812"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Taubes, C.H. (2011). Differential Geometry: Bundles, Connections, Metrics and Curvature, OUP Oxford.","DOI":"10.1093\/acprof:oso\/9780199605880.001.0001"},{"key":"ref_63","unstructured":"Do Carmo, M.P. (2016). Differential Geometry of Curves and Surfaces: Revised and Updated, Courier Dover Publications. [2nd ed.]."},{"key":"ref_64","first-page":"633","article-title":"\u00dcber die analytische Darstellbarkeit sogenannter willk\u00fcrlicher Funktionen einer reellen Ver\u00e4nderlichen","volume":"2","author":"Weierstrass","year":"1885","journal-title":"Sitzungsberichte K. Preu\u00dfischen Akad. Wiss. Berl."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1090\/S0002-9939-1959-0113131-7","article-title":"The Stone-Weierstrass Theorem","volume":"10","year":"1959","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_66","first-page":"1","article-title":"Automatic differentiation in machine learning: A survey","volume":"18","author":"Baydin","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"ref_67","unstructured":"B\u00e9zout, E. (1779). Th\u00e9orie G\u00e9n\u00e9rale des \u00c9quations Alg\u00e9briques, de l\u2019imprimerie de Ph.-D. Pierres."},{"key":"ref_68","unstructured":"Fulton, W. (1974). Algebraic Curves (Mathematics Lecture Note Series), The Benjamin\/Cummings Publishing Co., Inc."},{"key":"ref_69","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_70","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. arXiv."},{"key":"ref_71","unstructured":"Moor, M., Horn, M., Rieck, B., and Borgwardt, K. (2020, January 13\u201318). Topological autoencoders. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"1498","DOI":"10.1162\/jocn.2007.19.9.1498","article-title":"Open Access Series of Imaging Studies (OASIS): Cross-sectional MRI data in young, middle aged, nondemented, and demented older adults","volume":"19","author":"Marcus","year":"2007","journal-title":"J. Cogn. Neurosci."}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/13\/8\/535\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:31:34Z","timestamp":1760110294000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/13\/8\/535"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,7]]},"references-count":72,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["axioms13080535"],"URL":"https:\/\/doi.org\/10.3390\/axioms13080535","relation":{},"ISSN":["2075-1680"],"issn-type":[{"type":"electronic","value":"2075-1680"}],"subject":[],"published":{"date-parts":[[2024,8,7]]}}}