{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T14:17:35Z","timestamp":1776435455366,"version":"3.51.2"},"reference-count":26,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T00:00:00Z","timestamp":1758585600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T00:00:00Z","timestamp":1758585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Numer. Math."],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    We consider estimators obtained by iterates of the conjugate gradient (CG) algorithm applied to the normal equation of prototypical statistical inverse problems. Stopping the CG algorithm early induces regularisation, and optimal convergence rates of prediction and reconstruction error are established in wide generality for an ideal oracle stopping time. Based on this insight, a fully data-driven early stopping rule\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\tau $$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mi>\u03c4<\/mml:mi>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    is constructed, which also attains optimal rates, provided the error in estimating the noise level is not dominant. The error analysis of CG under statistical noise is subtle due to its nonlinear dependence on the observations. We provide an explicit error decomposition and identify two terms in the prediction error, which share important properties of classical bias and variance terms. Together with a continuous interpolation between CG iterates, this paves the way for a comprehensive error analysis of early stopping. In particular, a general oracle-type inequality is proved for the prediction error at\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\tau $$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mi>\u03c4<\/mml:mi>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    . For bounding the reconstruction error, a more refined probabilistic analysis, based on concentration of self-normalised Gaussian processes, is developed. The methodology also provides some new insights into early stopping for CG in deterministic inverse problems. A numerical study for standard examples shows good results in practice for early stopping at\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\tau $$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mi>\u03c4<\/mml:mi>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    .\n                  <\/jats:p>","DOI":"10.1007\/s00211-025-01469-4","type":"journal-article","created":{"date-parts":[[2025,9,23]],"date-time":"2025-09-23T08:34:34Z","timestamp":1758616474000},"page":"1739-1791","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Early stopping for conjugate gradients in statistical inverse problems"],"prefix":"10.1007","volume":"157","author":[{"given":"Laura","family":"Hucker","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Rei\u00df","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,9,23]]},"reference":[{"issue":"2","key":"1469_CR1","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.jat.2005.09.001","volume":"137","author":"AF Beardon","year":"2005","unstructured":"Beardon, A.F., Driver, K.A.: The zeros of linear combinations of orthogonal polynomials. J. Approx. Theory 137(2), 179\u2013186 (2005). https:\/\/doi.org\/10.1016\/j.jat.2005.09.001","journal-title":"J. Approx. Theory"},{"key":"1469_CR2","doi-asserted-by":"publisher","unstructured":"Bj\u00f6rck, \u00c5.: Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics, Philadelphia (1996). https:\/\/doi.org\/10.1137\/1.9781611971484","DOI":"10.1137\/1.9781611971484"},{"issue":"2","key":"1469_CR3","doi-asserted-by":"publisher","first-page":"3204","DOI":"10.1214\/18-ejs1482","volume":"12","author":"G Blanchard","year":"2018","unstructured":"Blanchard, G., Hoffmann, M., Rei\u00df, M.: Early stopping for statistical inverse problems via truncated SVD estimation. Electron. J. Stat. 12(2), 3204\u20133231 (2018). https:\/\/doi.org\/10.1214\/18-ejs1482","journal-title":"Electron. J. Stat."},{"issue":"3","key":"1469_CR4","doi-asserted-by":"publisher","first-page":"1043","DOI":"10.1137\/17m1154096","volume":"6","author":"G Blanchard","year":"2018","unstructured":"Blanchard, G., Hoffmann, M., Rei\u00df, M.: Optimal adaptation for early stopping in statistical inverse problems. SIAM\/ASA J. Uncertain. Quantif. 6(3), 1043\u20131075 (2018). https:\/\/doi.org\/10.1137\/17m1154096","journal-title":"SIAM\/ASA J. Uncertain. Quantif."},{"issue":"6","key":"1469_CR5","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1142\/s0219530516400017","volume":"14","author":"G Blanchard","year":"2016","unstructured":"Blanchard, G., Kr\u00e4mer, N.: Convergence rates of kernel conjugate gradient for random design regression. Anal. Appl. 14(6), 763\u2013794 (2016). https:\/\/doi.org\/10.1142\/s0219530516400017","journal-title":"Anal. Appl."},{"issue":"11","key":"1469_CR6","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/28\/11\/115011","volume":"28","author":"G Blanchard","year":"2012","unstructured":"Blanchard, G., Math\u00e9, P.: Discrepancy principle for statistical inverse problems with application to conjugate gradient iteration. Inverse Probl. 28(11), 115011 (2012). https:\/\/doi.org\/10.1088\/0266-5611\/28\/11\/115011","journal-title":"Inverse Probl."},{"key":"1469_CR7","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/978-3-642-19989-9_1","volume-title":"Inverse Problems and High-Dimensional Estimation. Lecture Notes in Statistics","author":"L Cavalier","year":"2011","unstructured":"Cavalier, L.: Inverse problems in statistics. In: Alquier, P., Gautier, E., Stoltz, G. (eds.) Inverse Problems and High-Dimensional Estimation. Lecture Notes in Statistics, vol. 203, pp. 3\u201396. Springer, Berlin, Heidelberg (2011). https:\/\/doi.org\/10.1007\/978-3-642-19989-9_1"},{"key":"1469_CR8","doi-asserted-by":"crossref","unstructured":"Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Mathematics and Its Applications, vol. 375. Kluwer Academic Publishers, Dordrecht (1996)","DOI":"10.1007\/978-94-009-1740-8"},{"key":"1469_CR9","unstructured":"Finocchio, G., Krivobokova, T.: An extended latent factor framework for ill-posed linear regression (2023). arxiv:2307.08377"},{"key":"1469_CR10","doi-asserted-by":"publisher","unstructured":"Hanke, M.: Conjugate gradient type methods for ill-posed problems. Pitman Research Notes in Mathematics Series, vol. 327. Chapman and Hall\/CRC, New York (1995). https:\/\/doi.org\/10.1201\/9781315140193","DOI":"10.1201\/9781315140193"},{"key":"1469_CR11","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1007\/s11075-007-9136-9","volume":"46","author":"PC Hansen","year":"2007","unstructured":"Hansen, P.C.: Regularization tools version 4.0 for Matlab 7.3. Numer. Algorithms 46, 189\u2013194 (2007). https:\/\/doi.org\/10.1007\/s11075-007-9136-9","journal-title":"Algorithms"},{"key":"1469_CR12","doi-asserted-by":"publisher","unstructured":"Hansen, P.C.: Discrete Inverse Problems: Insight and Algorithms. Society for Industrial and Applied Mathematics, Philadelphia (2010). https:\/\/doi.org\/10.1137\/1.9780898718836","DOI":"10.1137\/1.9780898718836"},{"issue":"6","key":"1469_CR13","doi-asserted-by":"publisher","first-page":"409","DOI":"10.6028\/jres.049.044","volume":"49","author":"MR Hestenes","year":"1952","unstructured":"Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409\u2013436 (1952). https:\/\/doi.org\/10.6028\/jres.049.044","journal-title":"J. Res. Natl. Bur. Stand."},{"key":"1469_CR14","unstructured":"Jahn, T.: Discretisation-adaptive regularisation of statistical inverse problems (2022). arxiv:2204.14037"},{"key":"1469_CR15","unstructured":"Johnstone, I.M.: Gaussian estimation: Sequence and wavelet models (2017). https:\/\/imjohnstone.su.domains\/\/GE_08_09_17.pdf, accessed 2023-01-16"},{"issue":"5","key":"1469_CR16","doi-asserted-by":"publisher","first-page":"1302","DOI":"10.1214\/aos\/1015957395","volume":"28","author":"B Laurent","year":"2000","unstructured":"Laurent, B., Massart, P.: Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28(5), 1302\u20131338 (2000). https:\/\/doi.org\/10.1214\/aos\/1015957395","journal-title":"Ann. Stat."},{"key":"1469_CR17","doi-asserted-by":"publisher","unstructured":"Louis, A.K.: Inverse und schlecht gestellte Probleme. B.\u00a0G. Teubner, Stuttgart (1989). https:\/\/doi.org\/10.1007\/978-3-322-84808-6","DOI":"10.1007\/978-3-322-84808-6"},{"key":"1469_CR18","doi-asserted-by":"crossref","unstructured":"Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics, vol. 1896. Springer, Berlin, Heidelberg (2007). https:\/\/doi.org\/10.1007\/978-3-540-48503-2","DOI":"10.1007\/978-3-540-48503-2"},{"issue":"1","key":"1469_CR19","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/24\/1\/015009","volume":"24","author":"P Math\u00e9","year":"2008","unstructured":"Math\u00e9, P., Hofmann, B.: How general are general source conditions? Inverse Probl. 24(1), 015009 (2008). https:\/\/doi.org\/10.1088\/0266-5611\/24\/1\/015009","journal-title":"Inverse Probl."},{"issue":"2","key":"1469_CR20","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/0041-5553(86)90002-9","volume":"26","author":"AS Nemirovskii","year":"1986","unstructured":"Nemirovskii, A.S.: The regularizing properties of the adjoint gradient method in ill-posed problems. USSR Comput. Math. Math. Phys. 26(2), 7\u201316 (1986). https:\/\/doi.org\/10.1016\/0041-5553(86)90002-9","journal-title":"USSR Comput. Math. Math. Phys."},{"key":"1469_CR21","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1007\/11752790_2","volume-title":"Subspace, Latent Structure and Future Selection. Lecture Notes in Computer Science","author":"R Rosipal","year":"2006","unstructured":"Rosipal, R., Kr\u00e4mer, N.: Overview and recent advances in partial least squares. In: Saunders, C., Grobelnik, M., Gunn, S., Shawe-Taylor, J. (eds.) Subspace, Latent Structure and Future Selection. Lecture Notes in Computer Science, vol. 3940, pp. 34\u201351. Springer, Berlin, Heidelberg (2006). https:\/\/doi.org\/10.1007\/11752790_2"},{"issue":"123","key":"1469_CR22","first-page":"1","volume":"18","author":"M Singer","year":"2017","unstructured":"Singer, M., Krivobokova, T., Munk, A.: Kernel partial least squares for stationary data. J. Mach. Learn. Res. 18(123), 1\u201341 (2017)","journal-title":"J. Mach. Learn. Res."},{"issue":"2","key":"1469_CR23","doi-asserted-by":"publisher","first-page":"3396","DOI":"10.1214\/20-ejs1747","volume":"14","author":"B Stankewitz","year":"2020","unstructured":"Stankewitz, B.: Smoothed residual stopping for statistical inverse problems via truncated SVD estimation. Electron. J. Stat. 14(2), 3396\u20133428 (2020). https:\/\/doi.org\/10.1214\/20-ejs1747","journal-title":"Electron. J. Stat."},{"key":"1469_CR24","doi-asserted-by":"publisher","DOI":"10.1090\/coll\/023","volume-title":"Orthogonal Polynomials, Colloquium Publications","author":"G Szeg\u0151","year":"1939","unstructured":"Szeg\u0151, G.: Orthogonal Polynomials, Colloquium Publications, vol. 23, 4th edn. American Mathematical Society, Providence, Rhode Island (1939). https:\/\/doi.org\/10.1090\/coll\/023","edition":"4"},{"issue":"9","key":"1469_CR25","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1016\/S0764-4442(00)00278-0","volume":"330","author":"A Tsybakov","year":"2000","unstructured":"Tsybakov, A.: On the best rate of adaptive estimation in some inverse problems. C.\u00a0R.\u00a0Acad.\u00a0Sci.\u00a0S\u00e9r.\u00a0I\u00a0Math. 330(9), 835\u2013840 (2000). https:\/\/doi.org\/10.1016\/S0764-4442(00)00278-0","journal-title":"C.\u00a0R.\u00a0Acad.\u00a0Sci.\u00a0S\u00e9r.\u00a0I\u00a0Math."},{"key":"1469_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/b13794","volume-title":"Introduction to Nonparametric Estimation","author":"AB Tsybakov","year":"2009","unstructured":"Tsybakov, A.B.: Introduction to Nonparametric Estimation. Springer Series in Statistics, Springer, New York (2009). https:\/\/doi.org\/10.1007\/b13794"}],"container-title":["Numerische Mathematik"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00211-025-01469-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00211-025-01469-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00211-025-01469-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T10:33:29Z","timestamp":1769510009000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00211-025-01469-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,23]]},"references-count":26,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,10]]}},"alternative-id":["1469"],"URL":"https:\/\/doi.org\/10.1007\/s00211-025-01469-4","relation":{},"ISSN":["0029-599X","0945-3245"],"issn-type":[{"value":"0029-599X","type":"print"},{"value":"0945-3245","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,23]]},"assertion":[{"value":"18 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 December 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2025","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2025","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 January 2026","order":6,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":7,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"This article was originally published under the subscription model but it is now published under an Open Access license.","order":8,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that there is no conflict of interest nor competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}