{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T21:16:17Z","timestamp":1791234977839,"version":"4.1.0"},"reference-count":62,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"3","funder":[{"name":"Ollendorff Minerva Center"},{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["777826"],"award-info":[{"award-number":["777826"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003977","name":"Israel Science Foundation","doi-asserted-by":"publisher","award":["534\/19"],"award-info":[{"award-number":["534\/19"]}],"id":[{"id":"10.13039\/501100003977","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Imaging Sci."],"published-print":{"date-parts":[[2021,1]]},"abstract":"<jats:p>Neural networks have revolutionized the field of data science, yielding remarkable solutions in a data-driven manner. For instance, in the field of mathematical imaging, they have surpassed traditional methods based on convex regularization. However, a fundamental theory supporting the practical applications is still in the early stages of development. We take a fresh look at neural networks and examine them via nonlinear eigenvalue analysis. The field of nonlinear spectral theory is still emerging, providing insights about nonlinear operators and systems. In this paper we view a neural network as a complex nonlinear operator and attempt to find its nonlinear eigenvectors. We first discuss the existence of such eigenvectors and analyze the kernel of ReLU networks. Then we study a nonlinear power method for generic nonlinear operators. For proximal operators associated to absolutely one-homogeneous convex regularization functionals, we can prove convergence of the method to an eigenvector of the proximal operator. This motivates us to apply a nonlinear method to networks which are trained to act similarly as a proximal operator. In order to take the nonhomogeneity of neural networks into account we define a modified version of the power method. We perform extensive experiments for different proximal operators and on various shallow and deep neural networks designed for image denoising. Proximal eigenvectors will be used for geometric analysis of graphs, as clustering or the computation of distance functions. For simple neural nets, we observe the influence of training data on the eigenvectors. For state-of-the-art denoising networks, we show that eigenvectors can be interpreted as (un)stable modes of the network, when contaminated with noise or other degradations.<\/jats:p>","DOI":"10.1137\/20m1384154","type":"journal-article","created":{"date-parts":[[2021,8,4]],"date-time":"2021-08-04T14:55:21Z","timestamp":1628088921000},"page":"1114-1148","source":"Crossref","is-referenced-by-count":10,"title":["Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks"],"prefix":"10.1137","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6554-9892","authenticated-orcid":true,"given":"Leon","family":"Bungert","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ester","family":"Hait-Fraenkel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicolas","family":"Papadakis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guy","family":"Gilboa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2021,8,4]]},"reference":[{"key":"atypb1","doi-asserted-by":"publisher","DOI":"10.1137\/17M1139126"},{"key":"atypb2","unstructured":"A. I. Aviles-Rivero, N. Papadakis, R. Li, S. M. Alsaleh, R. T. Tan, and C.B. Schonlieb,\n                      Beyond Supervised Classification: Extreme Minimal Supervision with the Graph 1-Laplacian\n                      , preprint, arXiv:1906.08635, 2019."},{"key":"atypb3","doi-asserted-by":"crossref","unstructured":"H. H. Bauschke and P. L. Combettes,\n                      Convex Analysis and Monotone Operator Theory in Hilbert Spaces\n                      , CMS Books Math. Ouvages Math. SMC 408, Springer, New York, 2011.","DOI":"10.1007\/978-1-4419-9467-7"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1006\/jdeq.2001.4150"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.4310\/MAA.2013.v20.n4.a1"},{"key":"atypb6","first-page":"511","author":"Benning M.","year":"2017","journal-title":"New York"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1016\/0024-3795(74)90029-9"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1137\/090769521"},{"key":"atypb9","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/ab1d71"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1007\/s00028-019-00545-1"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.2140\/apde.2021.14.823"},{"key":"atypb12","first-page":"291","author":"Bungert L.","year":"2019","journal-title":"New York"},{"key":"atypb13","doi-asserted-by":"publisher","DOI":"10.2140\/paa.2020.2.703"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1137\/15M1054687"},{"key":"atypb15","first-page":"1","author":"Burger M.","year":"2013","journal-title":"New York"},{"key":"atypb16","doi-asserted-by":"publisher","DOI":"10.1016\/j.anihpc.2008.04.003"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-010-0251-1"},{"key":"atypb18","first-page":"383","author":"Chambolle A.","year":"2021","journal-title":"New York"},{"key":"atypb19","first-page":"6571","author":"Chen T. Q.","year":"2018","journal-title":"Proceedings of Advances in Neural Information Processing Systems"},{"key":"atypb20","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcp.2018.09.012"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2019.107281"},{"key":"atypb22","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2945113"},{"key":"atypb23","doi-asserted-by":"crossref","unstructured":"A. Effland, E. Kobler, K. Kunisch, and T. Pock,\n                      An Optimal Control Approach to Early Stopping Variational Methods for Image Restoration\n                      , preprint, arXiv:1907.08488, 2019.","DOI":"10.1007\/s10851-019-00926-8"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(01)00178-9"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2010.2066286"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/ab0cb2"},{"key":"atypb27","first-page":"740","author":"Garg R.","year":"2016","journal-title":"New York"},{"key":"atypb28","unstructured":"A. Gautier, M. Hein, and F. Tudisco,\n                      Computing the Norm of Nonnegative Matrices and the Log-Sobolev Constant of Markov Chains\n                      , preprint, arXiv:2002.02447, 2020."},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1137\/18M1165049"},{"key":"atypb30","doi-asserted-by":"publisher","DOI":"10.1137\/18M1165037"},{"key":"atypb31","first-page":"36","author":"Gilboa G.","year":"2013","journal-title":"New York"},{"key":"atypb32","doi-asserted-by":"publisher","DOI":"10.1137\/130930704"},{"key":"atypb33","doi-asserted-by":"crossref","unstructured":"G. Gilboa,\n                      Nonlinear Eigenproblems in Image Processing and Computer Vision\n                      , Springer, New York, 2018.","DOI":"10.1007\/978-3-319-75847-3"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2011.2107908"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6420\/aa9a90"},{"key":"atypb36","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2872630"},{"key":"atypb37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2021.103041"},{"key":"atypb38","first-page":"847","author":"Hein M.","year":"2010","journal-title":"Proceedings of Advances in Neural Information Processing Systems"},{"key":"atypb39","unstructured":"R. A. Horn and C. R. Johnson,\n                      Matrix Analysis\n                      , Cambridge University Press, Cambridge, 2012."},{"key":"atypb40","unstructured":"O. Katzir,\n                      On the Scale-Space of Filters and Their Applications\n                      , Master's thesis, Technion, Israel Institute of Technology, Haifa, Israel, 2017."},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1524\/anly.2006.26.4.545"},{"key":"atypb42","doi-asserted-by":"crossref","unstructured":"E. Kobler, A. Effland, K. Kunisch, and T. Pock,\n                      Total Deep Variation for Linear Inverse Problems\n                      , preprint, arXiv:2001.05005, 2020.","DOI":"10.1109\/CVPR42600.2020.00757"},{"key":"atypb43","first-page":"5162","author":"Liu F.","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2015.0203"},{"key":"atypb45","first-page":"1781","author":"Meinhardt T.","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"atypb46","unstructured":"Y. Meyer,\n                      Oscillating Patterns in Image Processing and in Some Nonlinear Evolution Equations: The 15th Dean Jacquelines B. Lewis Memorial Lectures\n                      , Univ. Lecture Ser. 22, AMS, Providence, RI."},{"key":"atypb47","first-page":"152","volume":"9","author":"Mises R.","year":"1929","journal-title":"Math. Mech."},{"key":"atypb48","doi-asserted-by":"crossref","unstructured":"M. Moeller, J. Diebold, G. Gilboa, and D. Cremers,\n                      Learning nonlinear spectral filters for color image reconstruction\n                      , in Proceedings of the IEEE International Conference on Computer Vision (ICCV'15), 2015, pp. 289-297.","DOI":"10.1109\/ICCV.2015.41"},{"key":"atypb49","first-page":"3256","author":"Moeller M.","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"atypb50","first-page":"4293","author":"Nam H.","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb51","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-017-0577-6"},{"key":"atypb52","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2018.2820224"},{"key":"atypb53","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(92)90242-F"},{"key":"atypb54","unstructured":"E. K. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin,\n                      Plug-and-Play Methods Provably Converge with Properly Trained Denoisers\n                      , preprint, arXiv:1905.05406, 2019."},{"key":"atypb55","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2014.09.003"},{"key":"atypb56","unstructured":"A. Szlam and X. Bresson,\n                      Total variation and Cheeger cuts\n                      , in Proceedings of the International Conference on Machine Learning (ICML'10), 2010, pp. 1039-1046."},{"key":"atypb57","unstructured":"M. Thorpe and Y. van Gennip,\n                      Deep Limits of Residual Neural Networks\n                      , preprint, arXiv:1810.11741, 2018."},{"key":"atypb58","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2020.3006390"},{"key":"atypb59","doi-asserted-by":"publisher","DOI":"10.1137\/16M1074503"},{"key":"atypb60","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2662206"},{"key":"atypb61","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2839891"},{"key":"atypb62","unstructured":"Z. Zhang,\n                      Polyhedron Under Linear Transformations\n                      , preprint, arXiv:1201.6584, 2012."}],"container-title":["SIAM Journal on Imaging Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/20M1384154","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T13:52:56Z","timestamp":1787320376000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/20M1384154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1]]},"references-count":62,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,1]]}},"alternative-id":["10.1137\/20M1384154"],"URL":"https:\/\/doi.org\/10.1137\/20m1384154","relation":{},"ISSN":["1936-4954"],"issn-type":[{"value":"1936-4954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1]]}}}