{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T14:38:57Z","timestamp":1787323137559,"version":"3.56.0"},"reference-count":35,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"name":"Hong Kong Research Grant Council","award":["14306423"],"award-info":[{"award-number":["14306423"]}]},{"DOI":"10.13039\/501100004853","name":"Chinese University of Hong Kong","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004853","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["390685813"],"award-info":[{"award-number":["390685813"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["390685689"],"award-info":[{"award-number":["390685689"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2024,12,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>In recent years, new regularization methods based on (deep) neural networks have shown very promising empirical performance for the numerical solution of ill-posed problems, e.g., in medical imaging and imaging science. Due to the nonlinearity of neural networks, these methods often lack satisfactory theoretical justification. In this work, we rigorously discuss the convergence of a successful unsupervised approach that utilizes untrained convolutional neural networks to represent solutions to linear ill-posed problems. Untrained neural networks are particularly appealing for many applications because they do not require paired training data. The regularization property of the approach relies solely on the architecture of the neural network instead. Due to the vast overparameterization of the employed neural network, suitable early stopping is essential for the success of the method. We establish that the classical discrepancy principle is an adequate method for early stopping of two-layer untrained convolutional neural networks learned by gradient descent, and furthermore, it yields an approximation with minimax optimal convergence rates. Numerical results are also presented to illustrate the theoretical findings.<\/jats:p>","DOI":"10.1137\/24m1636617","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T04:02:49Z","timestamp":1733803369000},"page":"2331-2361","source":"Crossref","is-referenced-by-count":9,"title":["Early Stopping of Untrained Convolutional Neural Networks"],"prefix":"10.1137","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7633-8265","authenticated-orcid":true,"given":"Tim","family":"Jahn","sequence":"first","affiliation":[{"name":"Institut f\u00fcr Mathematik, TU Berlin, Berlin, 10623 Germany."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3775-9155","authenticated-orcid":true,"given":"Bangti","family":"Jin","sequence":"additional","affiliation":[{"name":"Department of Mathematics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong, P.R. China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,12,10]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/ac9011"},{"key":"ref2","unstructured":"S. Arora, S. Du, W. Hu, Z. Li, and R. Wang, Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks, in Proceedings of the 36th International Conference on Machine Learning, PMLR 97, K. Chaudhuri and R. Salakhutdinov, eds. 2019, pp. 322\u2013332."},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/aba415"},{"key":"ref4","author":"Barbano R.","year":"2024","journal-title":"Trans. Mach. Learn. Res."},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1109\/TCI.2022.3233188"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/26\/10\/105004"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/acc2b6"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-024-01191-0"},{"key":"ref9","unstructured":"Y. Cao and Q. Gu, Generalization bounds of stochastic gradient descent for wide and deep neural networks, in NIPS\u201919: Proceedings of the 33rd International Conference on Neural Information Processing Systems, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, and E. B. Fox, eds. 2019, pp. 10836\u201310846."},{"key":"ref10","doi-asserted-by":"crossref","unstructured":"Z. Cheng, M. Gadelha, S. Maji, and D. Sheldon, A Bayesian perspective on the deep image prior, in 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5438\u20135446, https:\/\/doi.org\/10.1109\/CVPR.2019.00559.","DOI":"10.1109\/CVPR.2019.00559"},{"key":"ref11","unstructured":"L. Chizat, E. Oyallon, and F. Bach, On lazy training in differentiable programming, in NIPS\u201919: Proceedings of the 33rd International Conference on Neural Information Processing Systems, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, and E. B. Fox, eds. 2019, pp. 2937\u20132947."},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-019-00923-x"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-1740-8"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2018.2888491"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1002\/gamm.202470004"},{"key":"ref16","unstructured":"R. Heckel and P. Hand, Deep decoder: Concise image representations from untrained non-convolutional networks, in International Conference on Learning Representations, 2019, https:\/\/openreview.net\/forum?id=rylV-2C9KQ."},{"key":"ref17","unstructured":"R. Heckel and M. Soltanolkotabi, Compressive sensing with un-trained neural networks: Gradient descent finds a smooth approximation, in Proceedings of the 37th International Conference on Machine Learning, PMLR 119, 2020, pp. 4149\u20134158."},{"key":"ref18","unstructured":"R. Heckel and M. Soltanolkotabi, Denoising and regularization via exploiting the structural bias of convolutional generators, in International Conference on Learning Representations, 2020, https:\/\/openreview.net\/forum?id=HJeqhA4YDS."},{"key":"ref19","volume-title":"Inverse Problems: Tikhonov Theory and Algorithms","author":"Ito K.","year":"2015"},{"key":"ref20","unstructured":"A. Jacot, F. Gabriel, and C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in NIPS\u201918: Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 8580\u20138589."},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"Y. Jo, S. Y. Chun, and J. Choi, Rethinking deep image prior for denoising, in Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), 2021, pp. 5087\u20135096.","DOI":"10.1109\/ICCV48922.2021.00504"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1515\/9783110208276"},{"key":"ref23","unstructured":"D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, in 3rd International Conference for Learning Representations, San Diego, CA, 2015."},{"key":"ref24","unstructured":"J. Lee, L. Xiao, S. S. Schoenholz, Y. Bahri, R. Novak, J. Sohl-Dickstein, and J. Pennington, Wide neural networks of any depth evolve as linear models under gradient descent, in NIPS\u201919: Proceedings of the 33rd International Conference on Neural Information Processing Systems, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, and E. B. Fox, eds. 2019, pp. 8572\u20138583."},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/ab6d57"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/24\/1\/015009"},{"key":"ref27","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1007\/BF00966239","volume":"7","author":"Morozov V. A.","year":"1966","journal-title":"Soviet Math. Dokl."},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2022.3207451"},{"key":"ref29","first-page":"617","volume-title":"Medical Imaging with Deep Learning, PMLR 227","author":"Nittscher M.","year":"2024"},{"key":"ref30","volume-title":"ICML Workshop on Understanding and Improving Generalization in Deep Learning","author":"Oymak S.","year":"2019"},{"key":"ref31","doi-asserted-by":"crossref","unstructured":"O. Ronneberger, P. Fischer, and T. Brox, U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2015, pp. 234\u2013241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref32","first-page":"433","volume":"3","author":"Scarlett J.","year":"2023","journal-title":"IEEE J. Sel. Topics Inf. Theory"},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"D. Ulyanov, A. Vedaldi, and V. Lempitsky, Deep image prior, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9446\u20139454.","DOI":"10.1109\/CVPR.2018.00984"},{"key":"ref34","unstructured":"H. Wang, T. Li, Z. Zhuang, T. Chen, H. Liang, and J. Sun, Early stopping for deep image prior, Trans. Mach. Learn. Res. (2023), https:\/\/openreview.net\/forum?id=231ZzrLC8X."},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2021.3084288"}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/24M1636617","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:43:56Z","timestamp":1787319836000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/24M1636617"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,10]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12,31]]}},"alternative-id":["10.1137\/24M1636617"],"URL":"https:\/\/doi.org\/10.1137\/24m1636617","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,10]]}}}