{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T16:49:41Z","timestamp":1781628581482,"version":"3.54.5"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2020,3,11]],"date-time":"2020-03-11T00:00:00Z","timestamp":1583884800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,3,11]],"date-time":"2020-03-11T00:00:00Z","timestamp":1583884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"European Research Council","award":["640156"],"award-info":[{"award-number":["640156"]}]},{"name":"European Research Council","award":["668998"],"award-info":[{"award-number":["668998"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Math Imaging Vis"],"published-print":{"date-parts":[[2020,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We investigate a well-known phenomenon of variational approaches in image processing, where typically the best image quality is achieved when the gradient flow process is stopped before converging to a stationary point. This paradox originates from a tradeoff between optimization and modeling errors of the underlying variational model and holds true even if deep learning methods are used to learn highly expressive regularizers from data. In this paper, we take advantage of this paradox and introduce an optimal stopping time into the gradient flow process, which in turn is learned from data by means of an optimal control approach. After a time discretization, we obtain variational networks, which can be interpreted as a particular type of recurrent neural networks. The learned variational networks achieve competitive results for image denoising and image deblurring on a standard benchmark data set. One of the key theoretical results is the development of first- and second-order conditions to verify optimal stopping time. A nonlinear spectral analysis of the gradient of the learned regularizer gives enlightening insights into the different regularization properties.<\/jats:p>","DOI":"10.1007\/s10851-019-00926-8","type":"journal-article","created":{"date-parts":[[2020,3,11]],"date-time":"2020-03-11T02:02:57Z","timestamp":1583892177000},"page":"396-416","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Variational Networks: An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration"],"prefix":"10.1007","volume":"62","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5936-048X","authenticated-orcid":false,"given":"Alexander","family":"Effland","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Erich","family":"Kobler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karl","family":"Kunisch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Pock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,3,11]]},"reference":[{"key":"926_CR1","volume-title":"Gradient Flows in Metric Spaces and in the Space of Probability Measures","author":"L Ambrosio","year":"2008","unstructured":"Ambrosio, L., Gigli, N., Savare, G.: Gradient Flows in Metric Spaces and in the Space of Probability Measures. Birkh\u00e4user, Basel (2008)"},{"key":"926_CR2","volume-title":"An Introduction to Numerical Analysis","author":"K Atkinson","year":"1989","unstructured":"Atkinson, K.: An Introduction to Numerical Analysis, 2nd edn. Wiley, Hoboken (1989)","edition":"2"},{"key":"926_CR3","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1137\/080716542","volume":"2","author":"A Beck","year":"2009","unstructured":"Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2, 183\u2013202 (2009)","journal-title":"SIAM J. Imag. Sci."},{"key":"926_CR4","doi-asserted-by":"crossref","unstructured":"Benning, M., Celledoni, E., Ehrhardt, M., Owren, B., Sch\u00f6nlieb, C.-B.: Deep learning as optimal control problems: models and numerical methods. arXiv:1904.05657 (2019)","DOI":"10.3934\/jcd.2019009"},{"issue":"5","key":"926_CR5","first-page":"381","volume":"4","author":"A Binder","year":"1996","unstructured":"Binder, A., Hanke, M., Scherzer, O.: On the Landweber iteration for nonlinear ill-posed problems. J. Inv. Ill Posed Probl. 4(5), 381\u2013390 (1996)","journal-title":"J. Inv. Ill Posed Probl."},{"key":"926_CR6","doi-asserted-by":"publisher","DOI":"10.1002\/9780470753767","volume-title":"Numerical Methods for Ordinary Differential Equations","author":"JC Butcher","year":"2008","unstructured":"Butcher, J.C.: Numerical Methods for Ordinary Differential Equations, 2nd edn. Wiley, Hoboken (2008)","edition":"2"},{"key":"926_CR7","doi-asserted-by":"crossref","unstructured":"Chang, B., Meng, L., Haber, E., Ruthotto, L., Begert, D., Holtham, E.: Reversible architectures for arbitrarily deep residual neural networks. In: AAAI Conference on Artificial Intelligence (2018)","DOI":"10.1609\/aaai.v32i1.11668"},{"key":"926_CR8","doi-asserted-by":"crossref","unstructured":"Chambolle, A., Caselles, V., Novaga, M., Cremers, D., Pock, T.: An introduction to total variation for image analysis. In: Theoretical Foundations and Numerical Methods for Sparse Recovery. Radon Series on Computational and Applied Mathematics, vol. 9, pp. 263\u2013340 (2009)","DOI":"10.1515\/9783110226157.263"},{"key":"926_CR9","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1017\/S096249291600009X","volume":"25","author":"A Chambolle","year":"2016","unstructured":"Chambolle, A., Pock, T.: An introduction to continuous optimization for imaging. Acta Numer. 25, 161\u2013319 (2016)","journal-title":"Acta Numer."},{"issue":"1","key":"926_CR10","doi-asserted-by":"publisher","first-page":"1060","DOI":"10.1109\/TIP.2014.2299065","volume":"99","author":"Y Chen","year":"2014","unstructured":"Chen, Y., Ranftl, R., Pock, T.: Insights into analysis operator learning: from patch-based sparse models to higher-order MRFs. IEEE Trans. Image Process. 99(1), 1060\u20131072 (2014)","journal-title":"IEEE Trans. Image Process."},{"issue":"6","key":"926_CR11","doi-asserted-by":"publisher","first-page":"1256","DOI":"10.1109\/TPAMI.2016.2596743","volume":"39","author":"Y Chen","year":"2017","unstructured":"Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256\u20131272 (2017)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"926_CR12","first-page":"1","volume":"5","author":"E Weinan","year":"2017","unstructured":"Weinan, E.: A proposal on machine learning via dynamical systems. Commun. Math. Stat. 5, 1\u201311 (2017)","journal-title":"Commun. Math. Stat."},{"key":"926_CR13","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1007\/s40687-018-0172-y","volume":"6","author":"E Weinan","year":"2019","unstructured":"Weinan, E., Han, J., Li, Q.: A mean-field optimal control formulation of deep learning. Res. Math. Sci. 6, 10 (2019)","journal-title":"Res. Math. Sci."},{"key":"926_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-1740-8","volume-title":"Regularization of inverse problems. Volume 375 of Mathematics and its Applications","author":"HW Engl","year":"1996","unstructured":"Engl, H.W., Hanke, M., Neubauer, A.: Regularization of inverse problems. Volume 375 of Mathematics and its Applications. Kluwer Academic Publishers Group, Dordrecht (1996)"},{"issue":"3","key":"926_CR15","doi-asserted-by":"publisher","first-page":"773","DOI":"10.1007\/s11075-018-0570-7","volume":"81","author":"S Gazzola","year":"2018","unstructured":"Gazzola, S., Hansen, P.C., Nagy, J.G.: IR Tools: a MATLAB package of iterative regularization methods and large-scale test problems. Numer. Algorithms 81(3), 773\u2013811 (2018)","journal-title":"Numer. Algorithms"},{"key":"926_CR16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-75847-3","volume-title":"Nonlinear Eigenproblems in Image Processing and Computer Vision","author":"G Gilboa","year":"2018","unstructured":"Gilboa, G.: Nonlinear Eigenproblems in Image Processing and Computer Vision. Springer, Berlin (2018)"},{"issue":"1","key":"926_CR17","doi-asserted-by":"publisher","first-page":"014004","DOI":"10.1088\/1361-6420\/aa9a90","volume":"34","author":"E Haber","year":"2017","unstructured":"Haber, E., Ruthotto, L.: Stable architectures for deep neural networks. Inverse Probl. 34(1), 014004 (2017)","journal-title":"Inverse Probl."},{"key":"926_CR18","volume-title":"Ordinary Differential Equations","author":"JK Hale","year":"1980","unstructured":"Hale, J.K.: Ordinary Differential Equations. Dover Publications, New York (1980)"},{"issue":"6","key":"926_CR19","doi-asserted-by":"publisher","first-page":"3055","DOI":"10.1002\/mrm.26977","volume":"79","author":"K Hammernik","year":"2018","unstructured":"Hammernik, K., Klatzer, T., Kobler, E., Recht, M.P., Sodickson, D.K., Pock, T., Knoll, F.: Learning a variational network for reconstruction of accelerated MRI data. Magn. Resonance Med 79(6), 3055\u20133071 (2018)","journal-title":"Magn. Resonance Med"},{"key":"926_CR20","doi-asserted-by":"crossref","unstructured":"Hansen, P. C.: Discrete inverse problems. In: Fundamentals of Algorithms, vol. 7. Society for Industrial and Applied Mathematics, Philadelphia, PA (2010)","DOI":"10.1137\/1.9780898718836"},{"key":"926_CR21","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"926_CR22","doi-asserted-by":"crossref","unstructured":"Ito, K., Kunisch, K.: Lagrange multiplier approach to variational problems and applications. In: Advances in Design and Control, vol. 15. Society for Industrial and Applied Mathematics, Philadelphia, PA (2008)","DOI":"10.1137\/1.9780898718614"},{"key":"926_CR23","doi-asserted-by":"publisher","DOI":"10.1515\/9783110208276","volume-title":"Iterative Regularization Methods for Nonlinear Ill-Posed Problems","author":"B Kaltenbacher","year":"2008","unstructured":"Kaltenbacher, B., Neubauer, A., Scherzer, O.: Iterative Regularization Methods for Nonlinear Ill-Posed Problems. Walter de Gruyter GmbH & Co. KG, Berlin (2008)"},{"key":"926_CR24","doi-asserted-by":"crossref","unstructured":"Kobler, E., Klatzer, T., Hammernik, K., Pock, T.: Variational networks: Connecting variational methods and deep learning. In: Pattern Recognition, pp. 281\u2013293. Springer, Berlin (2017)","DOI":"10.1007\/978-3-319-66709-6_23"},{"issue":"3","key":"926_CR25","doi-asserted-by":"publisher","first-page":"615","DOI":"10.2307\/2372313","volume":"73","author":"L Landweber","year":"1951","unstructured":"Landweber, L.: An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73(3), 615\u2013624 (1951)","journal-title":"Am. J. Math."},{"key":"926_CR26","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","volume":"521","author":"Y LeCun","year":"2015","unstructured":"LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436\u2013444 (2015)","journal-title":"Nature"},{"key":"926_CR27","first-page":"1","volume":"18","author":"Q Li","year":"2018","unstructured":"Li, Q., Chen, L., Tai, C., E, W.: Maximum principle based algorithms for deep learning. J. Mach. Learn. Res. 18, 1\u201329 (2018)","journal-title":"J. Mach. Learn. Res."},{"key":"926_CR28","unstructured":"Li, Q., Hao, S.: An optimal control approach to deep learning and applications to discrete-weight neural networks. arXiv:1803.01299 (2018)"},{"key":"926_CR29","unstructured":"Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International Conference on Computer Vision (2001)"},{"key":"926_CR30","unstructured":"Matet, S., Rosasco, L., Villa, S., Vu, B.\u00a0L.: Don\u2019t relax: early stopping for convex regularization. arXiv:1707.05422 (2017)"},{"issue":"7","key":"926_CR31","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1109\/34.56205","volume":"12","author":"P Perona","year":"1990","unstructured":"Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629\u2013639 (1990)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"4","key":"926_CR32","doi-asserted-by":"publisher","first-page":"1756","DOI":"10.1137\/16M1064064","volume":"9","author":"T Pock","year":"2016","unstructured":"Pock, T., Sabach, S.: Inertial proximal alternating linearized minimization (iPALM) for nonconvex and nonsmooth problems. SIAM J. Imag. Sci. 9(4), 1756\u20131787 (2016)","journal-title":"SIAM J. Imag. Sci."},{"key":"926_CR33","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/978-3-642-35289-8_5","volume-title":"Early Stopping\u2014But When? In Neural Networks: Tricks of the Trade","author":"L Prechelt","year":"2012","unstructured":"Prechelt, L.: Early Stopping\u2014But When? In Neural Networks: Tricks of the Trade, 2nd edn, pp. 53\u201367. Springer, Berlin (2012)","edition":"2"},{"key":"926_CR34","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-322-80234-7","volume-title":"Keine Probleme mit inversen Problemen","author":"A Rieder","year":"2003","unstructured":"Rieder, A.: Keine Probleme mit inversen Problemen. Friedr. Vieweg & Sohn, Braunschweig (2003)"},{"key":"926_CR35","first-page":"1630","volume":"28","author":"L Rosasco","year":"2015","unstructured":"Rosasco, L., Villa, S.: Learning with incremental iterative regularization. Adv. Neural Inf. Process. Syst. 28, 1630\u20131638 (2015)","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"2","key":"926_CR36","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1007\/s11263-008-0197-6","volume":"82","author":"S Roth","year":"2009","unstructured":"Roth, S., Black, M.J.: Fields of experts. Int. J. Comput. Vis. 82(2), 205\u2013229 (2009)","journal-title":"Int. J. Comput. Vis."},{"key":"926_CR37","doi-asserted-by":"crossref","unstructured":"Raskutti, G.,\u00a0Wainwright, M.\u00a0J., Yu, B.: Early stopping for non-parametric regression: an optimal data-dependent stopping rule. In: 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1318\u20131325 (2011)","DOI":"10.1109\/Allerton.2011.6120320"},{"key":"926_CR38","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","volume":"60","author":"L Rudin","year":"1992","unstructured":"Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D 60, 259\u2013268 (1992)","journal-title":"Phys. D"},{"key":"926_CR39","doi-asserted-by":"crossref","unstructured":"Schulter, S., Leistner, C., Bischof, H.: Fast and accurate image upscaling with super-resolution forests. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3791\u20133799 (2015)","DOI":"10.1109\/CVPR.2015.7299003"},{"key":"926_CR40","doi-asserted-by":"publisher","DOI":"10.1090\/gsm\/140","volume-title":"Ordinary Differential Equations and Dynamical Systems","author":"G Teschl","year":"2012","unstructured":"Teschl, G.: Ordinary Differential Equations and Dynamical Systems. American Mathematical Society, Providence (2012)"},{"key":"926_CR41","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-017-5167-4","volume-title":"Nonlinear Ill-Posed Problems","author":"AN Tikhonov","year":"1998","unstructured":"Tikhonov, A.N.: Nonlinear Ill-Posed Problems. Springer, Netherlands (1998)"},{"issue":"2","key":"926_CR42","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1007\/s00365-006-0663-2","volume":"26","author":"Y Yao","year":"2007","unstructured":"Yao, Y., Rosasco, L., Caponnetto, A.: On early stopping in gradient descent learning. Constr. Approx. 26(2), 289\u2013315 (2007)","journal-title":"Constr. Approx."},{"key":"926_CR43","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5020-3","volume-title":"Nonlinear Functional Analysis and its Applications III: Variational Methods and Optimization","author":"E Zeidler","year":"1985","unstructured":"Zeidler, E.: Nonlinear Functional Analysis and its Applications III: Variational Methods and Optimization. Springer, New York (1985)"},{"key":"926_CR44","doi-asserted-by":"publisher","unstructured":"Zhang,Y., Hofmann, B.: On the second order asymptotical regularization of linear ill-posed inverse problems. Appl. Anal. (2018). https:\/\/doi.org\/10.1080\/00036811.2018.1517412","DOI":"10.1080\/00036811.2018.1517412"},{"issue":"4","key":"926_CR45","doi-asserted-by":"publisher","first-page":"1538","DOI":"10.1214\/009053605000000255","volume":"33","author":"T Zhang","year":"2005","unstructured":"Zhang, T., Yu, B.: Boosting with early stopping: convergence and consistency. Ann. Stat. 33(4), 1538\u20131579 (2005)","journal-title":"Ann. Stat."}],"container-title":["Journal of Mathematical Imaging and Vision"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10851-019-00926-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10851-019-00926-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10851-019-00926-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T12:35:25Z","timestamp":1695904525000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10851-019-00926-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,11]]},"references-count":45,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,4]]}},"alternative-id":["926"],"URL":"https:\/\/doi.org\/10.1007\/s10851-019-00926-8","relation":{},"ISSN":["0924-9907","1573-7683"],"issn-type":[{"value":"0924-9907","type":"print"},{"value":"1573-7683","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,3,11]]},"assertion":[{"value":"17 July 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 October 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 March 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}