{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T08:21:09Z","timestamp":1792743669793,"version":"4.5.0"},"reference-count":56,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"2","funder":[{"DOI":"10.13039\/501100004795","name":"Institut Universitaire de France","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004795","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-1715671"],"award-info":[{"award-number":["CCF-1715671"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Journal on Mathematics of Data Science"],"published-print":{"date-parts":[[2020,1]]},"abstract":"<jats:p>Obtaining sharp Lipschitz constants for feed-forward neural networks is essential to assess their robustness in the face of perturbations of their inputs. We derive such constants in the context of a general layered network model involving compositions of nonexpansive averaged operators and affine operators. By exploiting this architecture, our analysis finely captures the interactions between the layers, yielding tighter Lipschitz constants than those resulting from the product of individual bounds for groups of layers. The proposed framework is shown to cover in particular many practical instances encountered in feed-forward neural networks. Our Lipschitz constant estimates are further improved in the case of structures employing scalar nonlinear functions, which include standard convolutional networks as special cases.<\/jats:p>","DOI":"10.1137\/19m1272780","type":"journal-article","created":{"date-parts":[[2020,6,29]],"date-time":"2020-06-29T14:53:49Z","timestamp":1593442429000},"page":"529-557","source":"Crossref","is-referenced-by-count":67,"title":["Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators"],"prefix":"10.1137","volume":"2","author":[{"given":"Patrick L.","family":"Combettes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean-Christophe","family":"Pesquet","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2020,6,29]]},"reference":[{"key":"atypb1","doi-asserted-by":"crossref","first-page":"14410","DOI":"10.1109\/ACCESS.2018.2807385","volume":"6","author":"Akhtar N.","year":"2018","journal-title":"IEEE Access"},{"key":"atypb2","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1007\/s00521-012-1315-5","volume":"24","author":"Aladag C. H.","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"atypb3","first-page":"274","author":"Athalye A.","year":"2018","journal-title":"NY"},{"key":"atypb4","first-page":"1","volume":"4","author":"Baillon J.-B.","year":"1978","journal-title":"Houston J. Math."},{"key":"atypb5","unstructured":"R. Balan, M. Singh, and D. Zou,\n                      Lipschitz Properties for Deep Convolutional Networks\n                      , preprint.https:\/\/arxiv.org\/abs\/1701.05217.pdf(2017)."},{"key":"atypb6","first-page":"6240","volume":"30","author":"Bartlett P. L.","year":"2017","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb7","unstructured":"M. Basirat and P. M. Roth,\n                      The Quest for the Golden Activation Function\n                      , preprint,https:\/\/arxiv.org\/pdf\/1808.00783(2018)."},{"key":"atypb8","doi-asserted-by":"crossref","unstructured":"H. H. Bauschke and P. L. Combettes,\n                      Convex Analysis and Monotone Operator Theory in Hilbert Spaces,\n                      2nd ed., Springer, New York, 2019.","DOI":"10.1007\/978-3-319-48311-5_31"},{"key":"atypb9","doi-asserted-by":"crossref","first-page":"1597","DOI":"10.1109\/TSP.2015.2502551","volume":"64","author":"Bayram I.","year":"2016","journal-title":"IEEE Trans. Signal Process."},{"key":"atypb10","doi-asserted-by":"crossref","first-page":"034005","DOI":"10.1088\/1361-6420\/ab460a","volume":"36","author":"Bertocchi C.","year":"2020","journal-title":"Inverse Problems"},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1137\/18M118709X"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1137\/15M1045223"},{"key":"atypb13","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1007\/s10884-015-9438-x","volume":"29","author":"Bo\u0163 R. I.","year":"2017","journal-title":"J. Dynam. Differential Equations"},{"key":"atypb14","first-page":"111","author":"Boureau Y.-L.","year":"2010","journal-title":"WI"},{"key":"atypb15","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/s11856-018-1723-z","volume":"227","author":"Bravo M.","year":"2018","journal-title":"Israel J. Math."},{"key":"atypb16","first-page":"143","author":"Bricen\u0342o-Arias L. M.","year":"2013","journal-title":"New York"},{"key":"atypb17","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1109\/TPAMI.2012.230","volume":"35","author":"Bruna J.","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"atypb18","first-page":"3","author":"Carlini N.","year":"2017","journal-title":"New York"},{"key":"atypb19","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TNNLS.2014.2310059","volume":"29","author":"Chorowski J.","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"atypb20","unstructured":"D.A. Clevert, T. Unterthiner, and S. Hochreiter,\n                      Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)\n                      , preprint,https:\/\/arxiv.org\/abs\/1511.07289(2015)."},{"key":"atypb21","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1080\/02331930412331327157","volume":"53","author":"Combettes P. L.","year":"2004","journal-title":"Optimization"},{"key":"atypb22","doi-asserted-by":"publisher","DOI":"10.1137\/17M112806X"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1137\/060669498"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1137\/050626090"},{"key":"atypb26","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1007\/s10957-012-0245-9","volume":"158","author":"Condat L.","year":"2013","journal-title":"J. Optim. Theory Appl."},{"key":"atypb27","doi-asserted-by":"crossref","unstructured":"F. Facchinei and J.S. Pang,\n                      Finite-Dimensional Variational Inequalities and Complementarity Problems\n                      , Springer, New York, 2003.","DOI":"10.1007\/b97543"},{"key":"atypb28","first-page":"12","author":"Geman S.","year":"1985","journal-title":"Washington, DC"},{"key":"atypb29","first-page":"315","volume":"15","author":"Glorot X.","year":"2011","journal-title":"JMLR W&CP"},{"key":"atypb30","unstructured":"I. J. Goodfellow, J. Shlens, and C. Szegedy,\n                      Explaining and Harnessing Adversarial Examples\n                      , preprint,https:\/\/arxiv.org\/abs\/1412.6572(2014)."},{"key":"atypb31","first-page":"1319","author":"Goodfellow I. J.","year":"2013","journal-title":"NY"},{"key":"atypb32","unstructured":"G. Hardy, J. E. Littlewood, and G. P\u00f3lya,\n                      Inequalities,\n                      2nd ed. Cambridge University Press, Cambridge, 1952."},{"key":"atypb33","first-page":"2266","volume":"30","author":"Hein M.","year":"2017","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb35","first-page":"1962","author":"Kreuk F.","year":"2018","journal-title":"Proceedings of the IEEE International Conference on Acoustic, Speech Signal Process., IEEE, Piscataway, NJ"},{"key":"atypb36","unstructured":"A. Krizhevsky,\n                      Convolutional Deep Belief Networks on CIFAR\n                      -10, Technical report, University of Toronto, Toronto, 2010,https:\/\/www.cs.toronto.edu\/~kriz\/conv-cifar10-aug2010.pdf"},{"key":"atypb37","first-page":"464","volume":"51","author":"Lee C.-Y.","year":"2016","journal-title":"JMLR W&CP"},{"key":"atypb38","unstructured":"A. L. Maas, A. Y. Hannun, and A. Y. Ng,\n                      Rectifier nonlinearities improve neural network acoustic models\n                      , in Proceedings of the 30th International Conference on Machine Learning, Curran Associates, Red Hook, NY, 2013."},{"key":"atypb39","unstructured":"A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu,\n                      Towards deep learning models resistant to adversarial attacks\n                      , preprint,https:\/\/arxiv.org\/abs\/1706.06083, 2017"},{"key":"atypb40","unstructured":"H. N. Mhaskar and C. A. Micchelli,\n                      How to choose an activation function\n                      , in Advances in Neural Information Processing Systems, Morgan Kaufmann, San Mateo, CA, 1994, pp. 319-326."},{"key":"atypb41","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1007\/s10957-017-1113-4","volume":"176","author":"Moorsi W. M.","year":"2018","journal-title":"J. Optim. Theory Appl."},{"key":"atypb42","doi-asserted-by":"crossref","first-page":"1023","DOI":"10.1143\/JPSJ.64.1023","volume":"64","author":"Nakagawa M.","year":"1995","journal-title":"J. Phys. Soc. Japan"},{"key":"atypb43","doi-asserted-by":"crossref","unstructured":"N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami,\n                      Distillation as a defense to adversarial perturbations against deep neural networks\n                      , in IEEE Symposium on Security Privacy, IEEE Computer Society, Los Alamitos, CA, 2016, pp. 582-597.","DOI":"10.1109\/SP.2016.41"},{"key":"atypb44","unstructured":"A. Raghunathan, J. Steinhardt, and P. Liang,\n                      Certified Defenses Against Adversarial Examples\n                      , preprint,https:\/\/arxiv.org\/pdf\/1801.09344.pdf(2018)."},{"key":"atypb45","unstructured":"P. Ramachandran, B. Zoph, and Q. V. Le,\n                      Searching for Activation Functions\n                      , preprint,https:\/\/arxiv.org\/pdf\/1710.05941.pdf(2018)."},{"key":"atypb46","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1037\/h0042519","volume":"65","author":"Rosenblatt F.","year":"1958","journal-title":"Psych. Rev."},{"key":"atypb47","doi-asserted-by":"crossref","unstructured":"W. Ruan, X. Huang, and M. Kwiatkowska,\n                      Reachability analysis of deep neural networks with provable guarantees\n                      , in Proceedings of the 27th International Joint Conference on Artificial Intelligence, AAAI Press, Palo Alto, CA, 2018, pp. 2651-2659.","DOI":"10.24963\/ijcai.2018\/368"},{"key":"atypb48","first-page":"3856","volume":"30","author":"Sabour S.","year":"2017","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb49","first-page":"3839","volume":"31","author":"Scaman K.","year":"2018","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb50","doi-asserted-by":"crossref","first-page":"4265","DOI":"10.1109\/TSP.2017.2708039","volume":"65","author":"Sokoli\u0107 J.","year":"2017","journal-title":"IEEE Trans. Signal Process."},{"key":"atypb51","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1109\/TCI.2019.2893568","volume":"5","author":"Sun Y.","year":"2019","journal-title":"IEEE Trans. Comput. Imaging"},{"key":"atypb52","unstructured":"C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus,\n                      Intriguing Properties of Neural Networks\n                      , preprint,https:\/\/arxiv.org\/pdf\/1312.6199.pdf(2013)."},{"key":"atypb53","first-page":"6541","volume":"31","author":"Tsuzuku Y.","year":"2018","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"atypb54","first-page":"5286","volume":"80","author":"Wong E.","year":"2018","journal-title":"NY"},{"key":"atypb55","doi-asserted-by":"crossref","first-page":"044003","DOI":"10.1088\/1361-6420\/33\/4\/044003","volume":"33","author":"Yamagishi M.","year":"2017","journal-title":"Inverse Problems"},{"key":"atypb56","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1109\/TCNS.2018.2813928","volume":"6","author":"Yi P.","year":"2019","journal-title":"IEEE Trans. Control Netw. Syst."},{"key":"atypb57","first-page":"1990","author":"Zeidler E.","year":"1985","journal-title":"New York"},{"key":"atypb58","unstructured":"Q. Zhao and L. D. Griffin,\n                      Suppressing the Unusual: Towards Robust CNNs Using Symmetric Activation Functions\n                      , preprint,https:\/\/arxiv.org\/abs\/1603.05145.pdf(2016)."}],"container-title":["SIAM Journal on Mathematics of Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/19M1272780","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T17:55:58Z","timestamp":1787334958000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/19M1272780"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1]]},"references-count":56,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2020,1]]}},"alternative-id":["10.1137\/19M1272780"],"URL":"https:\/\/doi.org\/10.1137\/19m1272780","relation":{},"ISSN":["2577-0187"],"issn-type":[{"value":"2577-0187","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1]]}}}