{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T23:43:59Z","timestamp":1770507839189,"version":"3.49.0"},"reference-count":77,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T00:00:00Z","timestamp":1738886400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DMS-2113458"],"award-info":[{"award-number":["DMS-2113458"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DBI-2243257"],"award-info":[{"award-number":["DBI-2243257"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Data augmentation is a widely used technique and an essential ingredient in the recent advance in self-supervised representation learning. By preserving the similarity between augmented data, the resulting data representation can improve various downstream analyses and achieve state-of-the-art performance in many applications. Despite the empirical effectiveness, most existing methods lack theoretical understanding under a general nonlinear setting. To fill this gap, we develop a statistical framework on a low-dimensional product manifold to model the data augmentation transformation. Under this framework, we introduce a new representation learning method called augmentation invariant manifold learning and design a computationally efficient algorithm by reformulating it as a stochastic optimization problem. Compared with existing self-supervised methods, the new method simultaneously exploits the manifold\u2019s geometric structure and invariant property of augmented data and has an explicit theoretical guarantee. Our theoretical investigation characterizes the role of data augmentation in the proposed method and reveals why and how the data representation learned from augmented data can improve the k-nearest neighbour classifier in the downstream analysis, showing that a more complex data augmentation leads to more improvement in downstream analysis. Finally, numerical experiments on simulated and real data sets are presented to demonstrate the merit of the proposed method.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkaf003","type":"journal-article","created":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T11:33:52Z","timestamp":1738928032000},"page":"978-1000","source":"Crossref","is-referenced-by-count":0,"title":["Augmentation invariant manifold learning"],"prefix":"10.1093","volume":"87","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7440-9544","authenticated-orcid":false,"given":"Shulei","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Illinois at Urbana-Champaign , 605 E. Springfield Ave., Champaign, IL 61820 ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,2,7]]},"reference":[{"issue":"3","key":"2026020620014827500_qkaf003-B1","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1080\/00031305.1992.10475879","article-title":"An introduction to kernel and nearest-neighbor nonparametric regression","volume":"46","author":"Altman","year":"1992","journal-title":"The American Statistician"},{"key":"2026020620014827500_qkaf003-B2","doi-asserted-by":"publisher","author":"Arora","year":"2019","DOI":"10.48550\/arXiv.1902.09229"},{"issue":"2","key":"2026020620014827500_qkaf003-B3","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1214\/009053606000001217","article-title":"Fast learning rates for plug-in classifiers","volume":"35","author":"Audibert","year":"2007","journal-title":"Annals of Statistics"},{"key":"2026020620014827500_qkaf003-B4","first-page":"26671","article-title":"Contrastive and non-contrastive self-supervised learning recover global and local spectral embedding methods","volume":"35","author":"Balestriero","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"3","key":"2026020620014827500_qkaf003-B5","doi-asserted-by":"publisher","first-page":"516","DOI":"10.1016\/j.acha.2014.03.002","article-title":"The embedding dimension of laplacian eigenfunction maps","volume":"37","author":"Bates","year":"2014","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026020620014827500_qkaf003-B6","article-title":"Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate","volume":"31","author":"Belkin","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"6","key":"2026020620014827500_qkaf003-B7","doi-asserted-by":"publisher","first-page":"1373","DOI":"10.1162\/089976603321780317","article-title":"Laplacian eigenmaps for dimensionality reduction and data representation","volume":"15","author":"Belkin","year":"2003","journal-title":"Neural Computation"},{"issue":"8","key":"2026020620014827500_qkaf003-B8","doi-asserted-by":"publisher","first-page":"1289","DOI":"10.1016\/j.jcss.2007.08.006","article-title":"Towards a theoretical foundation for laplacian-based manifold methods","volume":"74","author":"Belkin","year":"2008","journal-title":"Journal of Computer and System Sciences"},{"issue":"8","key":"2026020620014827500_qkaf003-B9","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"1","key":"2026020620014827500_qkaf003-B10","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1016\/j.acha.2016.08.005","article-title":"Iterated diffusion maps for feature identification","volume":"45","author":"Berry","year":"2018","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026020620014827500_qkaf003-B11","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-25388-6","volume-title":"Lectures on the nearest neighbor method","author":"Biau","year":"2015"},{"key":"2026020620014827500_qkaf003-B12","first-page":"9912","article-title":"Unsupervised learning of visual features by contrasting cluster assignments","volume":"33","author":"Caron","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"114","key":"2026020620014827500_qkaf003-B13","first-page":"1","article-title":"Redunet: A white-box deep network from the principle of maximizing rate reduction","volume":"23","author":"Chan","year":"2022","journal-title":"Journal of Machine Learning Research"},{"key":"2026020620014827500_qkaf003-B14","first-page":"1","article-title":"A group-theoretic framework for data augmentation","volume":"21","author":"Chen","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2026020620014827500_qkaf003-B15","author":"Chen","year":"2020"},{"key":"2026020620014827500_qkaf003-B16","first-page":"15750","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Chen","year":"2021"},{"key":"2026020620014827500_qkaf003-B17","first-page":"181","volume-title":"German Medical Data Sciences: Shaping Change\u2013Creative Solutions for Innovative Medicine","author":"Chereda","year":"2019"},{"key":"2026020620014827500_qkaf003-B18","doi-asserted-by":"publisher","author":"Chrabaszcz","year":"2017","DOI":"10.48550\/arXiv.1707.08819"},{"key":"2026020620014827500_qkaf003-B19","doi-asserted-by":"publisher","author":"Chung","year":"2017","DOI":"10.48550\/arXiv.1711.08490"},{"key":"2026020620014827500_qkaf003-B20","author":"Coates","year":"2011"},{"issue":"1","key":"2026020620014827500_qkaf003-B21","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/j.acha.2006.04.006","article-title":"Diffusion maps","volume":"21","author":"Coifman","year":"2006","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026020620014827500_qkaf003-B22","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"2026020620014827500_qkaf003-B23","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng","year":"2009"},{"key":"2026020620014827500_qkaf003-B24","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin","year":"2019"},{"issue":"3","key":"2026020620014827500_qkaf003-B25","doi-asserted-by":"crossref","first-page":"1899","DOI":"10.1109\/TIT.2022.3216561","article-title":"Impact of signal-to-noise ratio and bandwidth on graph Laplacian spectrum from high-dimensional noisy point cloud","volume":"69","author":"Ding","year":"2022","journal-title":"IEEE Transactions on Information Theory"},{"issue":"10","key":"2026020620014827500_qkaf003-B26","doi-asserted-by":"publisher","first-page":"5591","DOI":"10.1073\/pnas.1031596100","article-title":"Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data","volume":"100","author":"Donoho","year":"2003","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2026020620014827500_qkaf003-B27","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1016\/j.acha.2021.06.002","article-title":"Spectral convergence of graph laplacian and heat kernel reconstruction in L\u221e from random samples","volume":"55","author":"Dunson","year":"2021","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026020620014827500_qkaf003-B28","first-page":"9588","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Dwibedi","year":"2021"},{"key":"2026020620014827500_qkaf003-B29","volume-title":"Discriminatory analysis: Nonparametric discrimination, consistency properties","author":"Fix","year":"1985"},{"issue":"4","key":"2026020620014827500_qkaf003-B30","doi-asserted-by":"publisher","first-page":"827","DOI":"10.1007\/s10208-019-09436-w","article-title":"Error estimates for spectral convergence of the graph laplacian on random geometric graphs toward the laplace\u2013beltrami operator","volume":"20","author":"Garc\u00eda Trillos","year":"2020","journal-title":"Foundations of Computational Mathematics"},{"key":"2026020620014827500_qkaf003-B31","volume-title":"International Conference on Learning Representations","author":"Gidaris","year":"2018"},{"key":"2026020620014827500_qkaf003-B32","volume-title":"Deep learning","author":"Goodfellow","year":"2016"},{"key":"2026020620014827500_qkaf003-B33","first-page":"21271","article-title":"Bootstrap your own latent\u2014A new approach to self-supervised learning","volume":"33","author":"Grill","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026020620014827500_qkaf003-B34","volume-title":"Statistical sensor fusion","author":"Gustafsson","year":"2010"},{"key":"2026020620014827500_qkaf003-B35","volume-title":"Feature extraction: Foundations and applications","author":"Guyon","year":"2008"},{"key":"2026020620014827500_qkaf003-B36","first-page":"5000","article-title":"Provable guarantees for self-supervised deep learning with spectral contrastive loss","volume":"34","author":"HaoChen","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026020620014827500_qkaf003-B38","first-page":"770","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"2026020620014827500_qkaf003-B37","first-page":"9729","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"He","year":"2020"},{"key":"2026020620014827500_qkaf003-B39","first-page":"3","article-title":"Autoencoders, minimum description length and helmholtz free energy","volume":"6","author":"Hinton","year":"1993","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026020620014827500_qkaf003-B40","volume-title":"International Conference on Learning Representations","author":"Hjelm","year":"2018"},{"key":"2026020620014827500_qkaf003-B41","first-page":"9598","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Hua","year":"2021"},{"issue":"11","key":"2026020620014827500_qkaf003-B42","doi-asserted-by":"publisher","first-page":"4037","DOI":"10.1109\/TPAMI.2020.2992393","article-title":"Self-supervised visual feature learning with deep neural networks: A survey","volume":"43","author":"Jing","year":"2020","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2026020620014827500_qkaf003-B43","volume-title":"Principal component analysis","author":"Jolliffe","year":"2002"},{"key":"2026020620014827500_qkaf003-B44","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/B978-0-12-816034-3.00008-0","volume-title":"Biomedical Information Technology","author":"Kim","year":"2020"},{"key":"2026020620014827500_qkaf003-B45","author":"Kingma","year":"2015"},{"issue":"9","key":"2026020620014827500_qkaf003-B46","doi-asserted-by":"publisher","first-page":"1449","DOI":"10.1109\/JPROC.2015.2460697","article-title":"Multimodal data fusion: An overview of methods, challenges, and prospects","volume":"103","author":"Lahat","year":"2015","journal-title":"Proceedings of the IEEE: Institute of Electrical and Electronics Engineers"},{"issue":"7553","key":"2026020620014827500_qkaf003-B47","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"11","key":"2026020620014827500_qkaf003-B48","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE: Institute of Electrical and Electronics Engineers"},{"issue":"3","key":"2026020620014827500_qkaf003-B49","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1016\/j.acha.2015.09.002","article-title":"Learning the geometry of common latent variables using alternating-diffusion","volume":"44","author":"Lederman","year":"2018","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026020620014827500_qkaf003-B50","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.inffus.2019.08.005","article-title":"Multi-view diffusion maps","volume":"55","author":"Lindenbaum","year":"2020","journal-title":"Information Fusion"},{"key":"2026020620014827500_qkaf003-B51","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/BF02547521","article-title":"\u00dcber die praktische aufl\u00f6sung von integralgleichungen mit anwendungen auf randwertaufgaben","volume":"54","author":"Nystr\u00f6m","year":"1930","journal-title":"Acta Mathematica"},{"key":"2026020620014827500_qkaf003-B52","first-page":"2701","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Pathak","year":"2017"},{"key":"2026020620014827500_qkaf003-B53","volume-title":"International Conference on Learning Representations","author":"Pope","year":"2021"},{"issue":"3","key":"2026020620014827500_qkaf003-B54","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1002\/cpa.v69.3","article-title":"Embeddings of riemannian manifolds with heat kernels and eigenfunctions","volume":"69","author":"Portegies","year":"2016","journal-title":"Communications on Pure and Applied Mathematics"},{"key":"2026020620014827500_qkaf003-B55","first-page":"8748","volume-title":"International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"2026020620014827500_qkaf003-B56","first-page":"3527","volume-title":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","author":"Rhee","year":"2018"},{"issue":"3","key":"2026020620014827500_qkaf003-B57","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A stochastic approximation method","volume":"22","author":"Robbins","year":"1951","journal-title":"The Annals of Mathematical Statistics"},{"issue":"5500","key":"2026020620014827500_qkaf003-B58","doi-asserted-by":"publisher","first-page":"2323","DOI":"10.1126\/science.290.5500.2323","article-title":"Nonlinear dimensionality reduction by locally linear embedding","volume":"290","author":"Roweis","year":"2000","journal-title":"Science"},{"issue":"1","key":"2026020620014827500_qkaf003-B59","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.acha.2019.01.001","article-title":"Multi-view kernel consensus for data analysis","volume":"49","author":"Salhov","year":"2020","journal-title":"Applied and Computational Harmonic Analysis"},{"issue":"5","key":"2026020620014827500_qkaf003-B60","doi-asserted-by":"publisher","first-page":"2733","DOI":"10.1214\/12-AOS1049","article-title":"Optimal weighted nearest neighbour classifiers","volume":"40","author":"Samworth","year":"2012","journal-title":"Annals of Statistics"},{"issue":"1","key":"2026020620014827500_qkaf003-B61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"Journal of Big Data"},{"issue":"8","key":"2026020620014827500_qkaf003-B62","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1002\/cpa.v65.8","article-title":"Vector diffusion maps and the connection laplacian","volume":"65","author":"Singer","year":"2012","journal-title":"Communications on Pure and Applied Mathematics"},{"issue":"3","key":"2026020620014827500_qkaf003-B63","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1016\/j.acha.2017.12.006","article-title":"Latent common manifold learning with alternating diffusion: Analysis and applications","volume":"47","author":"Talmon","year":"2019","journal-title":"Applied and Computational Harmonic Analysis"},{"issue":"5500","key":"2026020620014827500_qkaf003-B64","doi-asserted-by":"publisher","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","article-title":"A global geometric framework for nonlinear dimensionality reduction","volume":"290","author":"Tenenbaum","year":"2000","journal-title":"Science"},{"key":"2026020620014827500_qkaf003-B65","author":"Tian","year":"2020"},{"key":"2026020620014827500_qkaf003-B66","doi-asserted-by":"publisher","author":"Tian","year":"2020","DOI":"10.48550\/arXiv.2010.00578"},{"key":"2026020620014827500_qkaf003-B67","first-page":"1179","volume-title":"Algorithmic Learning Theory","author":"Tosh","year":"2021"},{"key":"2026020620014827500_qkaf003-B68","volume-title":"International Conference on Learning Representations","author":"Tsai","year":"2020"},{"issue":"544","key":"2026020620014827500_qkaf003-B69","doi-asserted-by":"publisher","first-page":"2454","DOI":"10.1080\/01621459.2022.2057317","article-title":"Self-supervised metric learning in multi-view data: A downstream task perspective","volume":"118","author":"Wang","year":"2023","journal-title":"Journal of the American Statistical Association"},{"key":"2026020620014827500_qkaf003-B70","doi-asserted-by":"publisher","author":"Wang","year":"2023","DOI":"10.48550\/arXiv.2310.19041"},{"key":"2026020620014827500_qkaf003-B71","doi-asserted-by":"publisher","author":"Wei","year":"2021","DOI":"10.48550\/arXiv.2010.03622"},{"issue":"1","key":"2026020620014827500_qkaf003-B72","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s11263-005-4939-z","article-title":"Unsupervised learning of image manifolds by semidefinite programming","volume":"70","author":"Weinberger","year":"2006","journal-title":"International Journal of Computer Vision"},{"key":"2026020620014827500_qkaf003-B73","author":"Wen","year":"2021"},{"key":"2026020620014827500_qkaf003-B74","first-page":"24794","article-title":"The mechanism of prediction head in non-contrastive self-supervised learning","volume":"35","author":"Wen","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026020620014827500_qkaf003-B75","doi-asserted-by":"crossref","DOI":"10.1017\/9781009004282","volume-title":"Optimization for data analysis","author":"Wright","year":"2022"},{"key":"2026020620014827500_qkaf003-B76","author":"Zbontar","year":"2021"},{"issue":"1","key":"2026020620014827500_qkaf003-B77","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1137\/S1064827502419154","article-title":"Principal manifolds and nonlinear dimensionality reduction via tangent space alignment","volume":"26","author":"Zhang","year":"2004","journal-title":"SIAM Journal on Scientific Computing"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/4\/978\/61792579\/qkaf003.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/4\/978\/61792579\/qkaf003.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T01:02:02Z","timestamp":1770426122000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/87\/4\/978\/8005183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,7]]},"references-count":77,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,2,7]]},"published-print":{"date-parts":[[2025,9,15]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkaf003","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,2,7]]}}}