{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T18:29:51Z","timestamp":1787336991933,"version":"build-2736575974"},"reference-count":44,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"DOI":"10.13039\/501100000324","name":"Gatsby Charitable Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000324","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000761","name":"Imperial College London","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000761","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["CNS-2334997"],"award-info":[{"award-number":["CNS-2334997"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Journal on Mathematics of Data Science"],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>Many recent theoretical works on meta-learning aim to achieve guarantees in leveraging similar representational structures from related tasks towards simplifying a target task. The main aim of theoretical guarantees on the subject is to establish the extent to which convergence rates\u2014in learning a common representation\u2014 may scale with the number [Formula: see text] of tasks (as well as the number of samples per task). First steps in this setting demonstrate this property when both the shared representation amongst tasks and task-specific regression functions are linear. This linear setting readily reveals the benefits of aggregating tasks, e.g., via averaging arguments. In practice, however, the representation is often highly nonlinear, introducing nontrivial biases in each task that cannot easily be averaged out as in the linear case. In the present work, we derive theoretical guarantees for meta-learning with nonlinear representations. In particular, assuming the shared nonlinearity maps to an infinite dimensional reproducing kernel Hilbert space, we show that additional biases can be mitigated with careful regularization that leverages the smoothness of task-specific regression functions, yielding improved rates that scale with the number of tasks as desired.<\/jats:p>","DOI":"10.1137\/24m1662977","type":"journal-article","created":{"date-parts":[[2025,10,3]],"date-time":"2025-10-03T08:00:28Z","timestamp":1759478428000},"page":"1594-1615","source":"Crossref","is-referenced-by-count":1,"title":["Nonlinear Meta-learning Can Guarantee Faster Rates"],"prefix":"10.1137","volume":"7","author":[{"given":"Dimitri","family":"Meunier","sequence":"first","affiliation":[{"name":"Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom. Equal contribution with Zhu Li."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhu","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Mathematics, Imperial College London, London, United Kingdom.\u00a0Equal contribution with Dimitri Meunier."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3169-7624","authenticated-orcid":true,"given":"Arthur","family":"Gretton","sequence":"additional","affiliation":[{"name":"Gatsby Computational Neuroscience Unit, University College London, London, United Kingdom. Equal contribution with Zhu Li."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samory","family":"Kpotufe","sequence":"additional","affiliation":[{"name":"Department of Statistics, Columbia University, New York, NY 10027 USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2025,10,3]]},"reference":[{"key":"ref1","unstructured":"M. Aliakbarpour, K. Bairaktari, G. Brown, A. Smith, N. Srebro, and J. Ullman, Metalearning with very few samples per task, in Conference on Learning Theory, PMLR, 2024, pp. 46\u201393."},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"A. Andoni, R. Panigrahy, G. Valiant, and L. Zhang, Learning sparse polynomial functions, in Proceedings of the Twenty-Fifth Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, 2014, pp. 500\u2013510, https:\/\/doi.org\/10.1137\/1.9781611973402.3.","DOI":"10.1137\/1.9781611973402.37"},{"key":"ref3","volume-title":"Advances in Neural Information Processing Systems 19","author":"Ben-David S.","year":"2006"},{"key":"ref4","volume-title":"Reproducing Kernel Hilbert Spaces in Probability and Statistics","author":"Berlinet A.","year":"2011"},{"key":"ref5","first-page":"46","volume":"22","author":"Blanchard G.","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref6","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1007\/s10208-017-9359-7","volume":"18","author":"Blanchard G.","year":"2018","journal-title":"Found. Comput. Math."},{"key":"ref7","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s10208-006-0196-8","volume":"7","author":"Caponnetto A.","year":"2007","journal-title":"Found. Comput. Math."},{"key":"ref8","unstructured":"S. Chen and R. Meka, Learning polynomials in few relevant dimensions, in Conference on Learning Theory, PMLR, 2020, pp. 1161\u20131227."},{"key":"ref9","unstructured":"G. Denevi, C. Ciliberto, R. Grazzi, and M. Pontil, Learning-to-learn stochastic gradient descent with biased regularization, in International Conference on Machine Learning, PMLR, 2019, pp. 1566\u20131575."},{"key":"ref10","unstructured":"S. S. Du, W. Hu, S. M. Kakade, J. D. Lee, and Q. Lei, Few-shot learning via learning the representation, provably, in International Conference on Learning Representations, 2021."},{"key":"ref11","unstructured":"C. Finn, P. Abbeel, and S. Levine, Model-agnostic meta-learning for fast adaptation of deep networks, in International Conference on Machine Learning, PMLR, 2017, pp. 1126\u20131135."},{"key":"ref12","unstructured":"C. Finn, A. Rajeswaran, S. Kakade, and S. Levine, Online meta-learning, in International Conference on Machine Learning, PMLR, 2019, pp. 1920\u20131930."},{"key":"ref13","first-page":"205","volume":"21","author":"Fischer S.","year":"2020","journal-title":"J. Mach. Learn. Res."},{"key":"ref14","doi-asserted-by":"crossref","first-page":"1871","DOI":"10.1214\/08-AOS637","volume":"37","author":"Fukumizu K.","year":"2009","journal-title":"Ann. Statist."},{"key":"ref15","unstructured":"T. Galanti, A. Gy\u00f6rgy, and M. Hutter, Generalization Bounds for Transfer Learning with Pretrained Classifiers, preprint, arXiv:2212.12532, 2022."},{"key":"ref16","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1214\/20-AOS1990","volume":"49","author":"Ghorbani B.","year":"2021","journal-title":"Ann. Statist."},{"key":"ref17","series-title":"Springer Ser. Statist. 297","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-5369-7","volume-title":"Smoothing spline ANOVA models","author":"Gu C.","year":"2013","edition":"2"},{"key":"ref18","unstructured":"M. Kanagawa, P. Hennig, D. Sejdinovic, and B. K. Sriperumbudur, Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences, preprint, arXiv:1807.02582, 2018."},{"key":"ref19","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s10208-018-09407-7","volume":"20","author":"Kanagawa M.","year":"2020","journal-title":"Found. Comput. Math."},{"key":"ref20","volume":"32","author":"Khodak M.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref21","unstructured":"W. Kong, R. Somani, Z. Song, S. Kakade, and S. Oh, Meta-learning for mixed linear regression, in International Conference on Machine Learning, PMLR, 2020, pp. 5394\u20135404."},{"key":"ref22","unstructured":"M. Konobeev, I. Kuzborskij, and C. Szepesv\u00e1ri, A distribution-dependent analysis of meta learning, in International Conference on Machine Learning, PMLR, 2021, pp. 5697\u20135706."},{"key":"ref23","first-page":"3182","volume":"39","author":"Li B.","year":"2011","journal-title":"Ann. Statist."},{"key":"ref24","first-page":"1272","volume":"37","author":"Li B.","year":"2009","journal-title":"Ann. Statist."},{"key":"ref25","first-page":"738","volume":"51","author":"Ma C.","year":"2023","journal-title":"Ann. Statist."},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"Y. Mansour, M. Mohri, and A. Rostamizadeh, Domain adaptation: Learning bounds and algorithms, in Conference on Learning Theory, PMLR, 2009.","DOI":"10.1007\/978-3-642-04414-4_4"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"A. Maurer, A chain rule for the expected suprema of Gaussian processes, in Proceedings of the 25th International Conference on Algorithmic Learning Theory, 2014.","DOI":"10.1007\/978-3-319-11662-4_18"},{"key":"ref28","first-page":"81","volume":"17","author":"Maurer A.","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref29","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.3390\/e23101257","volume":"23","author":"Meunier D.","year":"2021","journal-title":"Entropy"},{"key":"ref30","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.5802\/crmath.367","volume":"360","author":"Mourtada J.","year":"2022","journal-title":"C. R. Math. Acad. Sci. Paris"},{"key":"ref31","unstructured":"X. Niu, L. Su, J. Xu, and P. Yang, Collaborative Learning with Shared Linear Representations: Statistical Rates and Optimal Algorithms, preprint, arXiv:2409.04919, 2024."},{"key":"ref32","first-page":"2389","volume":"12","author":"Sriperumbudur B. K.","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref33","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-77242-4","volume-title":"Support Vector Machines","author":"Steinwart I.","year":"2008"},{"key":"ref34","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1007\/s00365-012-9153-3","volume":"35","author":"Steinwart I.","year":"2012","journal-title":"Constr. Approx."},{"key":"ref35","volume-title":"Matrix Perturbation Theory","author":"Stewart G. W.","year":"1990"},{"key":"ref36","first-page":"18487","volume":"34","author":"Thekumparampil K. K.","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref37","unstructured":"N. Tripuraneni, C. Jin, and M. Jordan, Provable meta-learning of linear representations, in International Conference on Machine Learning, PMLR, 2021, pp. 10434\u201310443."},{"key":"ref38","first-page":"7852","author":"Tripuraneni N.","year":"2020","journal-title":"Advances in Neural Information Processing Systems 33"},{"key":"ref39","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/BF01932678","volume":"12","author":"Wedin P.-\u00c5.","year":"1972","journal-title":"BIT"},{"key":"ref40","unstructured":"Q. Wu, F. Liang, and S. Mukherjee, Regularized Sliced Inverse Regression for Kernel Models, Technical report, Citeseer, 2007."},{"key":"ref41","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1016\/j.jmva.2008.01.006","volume":"99","author":"Yin X.","year":"2008","journal-title":"J. Multivariate Anal."},{"key":"ref42","unstructured":"O. K. Y\u00fcksel, E. Boursier, and N. Flammarion, First-order anil provably learns representations despite overparametrisation, in International Conference on Learning Representations, 2024."},{"key":"ref43","first-page":"3299","volume":"16","author":"Zhang Y.","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"ref44","doi-asserted-by":"crossref","unstructured":"R. Zippel, Probabilistic algorithms for sparse polynomials, in International Symposium on Symbolic and Algebraic Manipulation, Springer, 1979, pp. 216\u2013226.","DOI":"10.1007\/3-540-09519-5_73"}],"container-title":["SIAM Journal on Mathematics of Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/24M1662977","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T17:39:44Z","timestamp":1787333984000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/24M1662977"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,3]]},"references-count":44,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1137\/24M1662977"],"URL":"https:\/\/doi.org\/10.1137\/24m1662977","relation":{},"ISSN":["2577-0187"],"issn-type":[{"value":"2577-0187","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,3]]}}}