{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T09:33:22Z","timestamp":1790674402022,"version":"4.1.0"},"reference-count":59,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"3","funder":[{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Journal on Mathematics of Data Science"],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>We introduce ABBA networks, a novel class of (almost) nonnegative neural networks, which are shown to possess a series of appealing properties. In particular, we demonstrate that these networks are universal approximators while enjoying the advantages of nonnegative weighted networks. We derive tight Lipschitz bounds in both the fully connected and convolutional cases. We propose a strategy for designing ABBA nets that are robust against adversarial attacks, by finely controlling the Lipschitz constant of the network during the training phase. We show that our method outperforms other state-of-the-art defenses against adversarial white-box attackers. Experiments are performed on image classification tasks on four benchmark datasets.<\/jats:p>","DOI":"10.1137\/23m1589591","type":"journal-article","created":{"date-parts":[[2024,7,15]],"date-time":"2024-07-15T04:03:24Z","timestamp":1721016204000},"page":"649-678","source":"Crossref","is-referenced-by-count":4,"title":["ABBA Neural Networks: Coping with Positivity, Expressivity, and Robustness"],"prefix":"10.1137","volume":"6","author":[{"given":"Ana","family":"Neac\u015fu","sequence":"first","affiliation":[{"name":"Speech and Dialogue Laboratory, University Politehnica of Bucharest, Romania."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5943-8061","authenticated-orcid":true,"given":"Jean-Christophe","family":"Pesquet","sequence":"additional","affiliation":[{"name":"Centre de Vision Num\u00e9rique, Inria, CentraleSup\u00e9lec, Gif-sur-Yvette, France."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vlad","family":"Vasilescu","sequence":"additional","affiliation":[{"name":"Speech and Dialogue Laboratory, University Politehnica of Bucharest, Romania."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Corneliu","family":"Burileanu","sequence":"additional","affiliation":[{"name":"Speech and Dialogue Laboratory, University Politehnica of Bucharest, Romania."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2024,7,15]]},"reference":[{"key":"ref1","unstructured":"B. Amos, L. Xu, and J. Z. Kolter, Input convex neural networks, in Proceedings Int. Conference Machine Learning, PMLR 70, 2017, pp. 146\u2013155."},{"key":"ref2","unstructured":"C. Anil, J. Lucas, and R. Grosse, Sorting out Lipschitz function approximation, in Proceedings Int. Conference Machine Learning, PMLR, 2019, pp. 291\u2013301."},{"key":"ref3","doi-asserted-by":"crossref","first-page":"2132","DOI":"10.3390\/electronics10172132","volume":"10","author":"Apostolidis K. D.","year":"2021","journal-title":"Electronics"},{"key":"ref4","doi-asserted-by":"crossref","first-page":"3969","DOI":"10.1109\/TNNLS.2017.2747861","volume":"29","author":"Ayinde B. O.","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref5","doi-asserted-by":"crossref","unstructured":"L. B\u00e9thune, T. Boissin, M. Serrurier, F. Mamalet, C. Friedrich, and A. Gonzalez Sanz, Pay attention to your loss: Understanding misconceptions about Lipschitz neural networks, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 35, 2022, pp. 20077\u201320091.","DOI":"10.52202\/068431-1460"},{"key":"ref6","unstructured":"J. Bromley, I. Guyon, Y. LeCun, E. S\u00e4ckinger, and R. Shah, Signature verification using a \u201csiamese\u201d time delay neural network, in Proc. Ann. Conf. Neur. Inform. Proc. Syst. Vol. 6, Morgan Kaufmann, 1993, pp. 737\u2013744."},{"key":"ref7","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/TNNLS.2014.2310059","volume":"26","author":"Chorowski J.","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref8","unstructured":"M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier, Parseval networks: Improving robustness to adversarial examples, in Proc. Int. Conf. Mach. Learn., 2017, pp. 854\u2013863."},{"key":"ref9","unstructured":"J. Cohen, E. Rosenfeld, and Z. Kolter, Certified adversarial robustness via randomized smoothing, in Proc. Int. Conf. Machine Learn., 2019, pp. 1310\u20131320."},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1137\/19M1272780"},{"key":"ref11","unstructured":"F. Croce, M. Andriushchenko, and M. Hein, Provable robustness of ReLU networks via maximization of linear regions, in Proc. Int. Conf. Art. Intell. and Statistics, 2019, pp. 2057\u20132066."},{"key":"ref12","unstructured":"I. Diakonikolas, D. M. Kane, V. Kontonis, and N. Zarifis, Algorithms and SQ lower bounds for PAC learning one-hidden-layer ReLU networks, in Proc. Conf. Learn. Theory, PMLR, 2020, pp. 1514\u20131539."},{"key":"ref13","first-page":"819","author":"Gamarnik D.","year":"2021","journal-title":"Proc. IEEE Int. Symp. Inform. Theory"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"M. Gleize, E. Shnarch, L. Choshen, L. Dankin, G. Moshkowich, R. Aharonov, and N. Slonim, Are you convinced? Choosing the more convincing evidence with a Siamese network, in Proc. Ann. Meeting of the Assoc. for Comput. Linguistics, Association for Computational Linguistics, 2019, pp. 967\u2013976.","DOI":"10.18653\/v1\/P19-1093"},{"key":"ref15","unstructured":"I. Goodfellow, J. Shlens, and C. Szegedy, Explaining and harnessing adversarial examples, in Proc. Int. Conf. Learn. Represent., 2015."},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"S. Gowal, K. Dvijotham, R. Stanforth, T. A. Mann, and P. Kohli, A dual approach to verify and train deep networks, in Proc. Int. Conf. Art. Intell., 2019, pp. 6156\u20136160.","DOI":"10.24963\/ijcai.2019\/854"},{"key":"ref17","unstructured":"C. Guo, B. Karrer, K. Chaudhuri, and L. van der Maaten, Bounding training data reconstruction in private (deep) learning, in Proc. Int. Conf. Machine Learn., 2022, pp. 8056\u20138071."},{"key":"ref18","doi-asserted-by":"crossref","DOI":"10.3389\/frsip.2022.794469","volume":"2","author":"Gupta K.","year":"2022","journal-title":"Front. Signal Process."},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"A. He, C. Luo, X. Tian, and W. Zeng, A twofold siamese network for real-time object tracking, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2018, pp. 4834\u20134843.","DOI":"10.1109\/CVPR.2018.00508"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"G. Katz, C. Barrett, D. L. Dill, K. Julian, and M. J. Kochenderfer, Reluplex: An efficient SMT solver for verifying deep neural networks, in Proc. Int. Conf. Comp. Aided Verif., 2017, pp. 97\u2013117.","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"ref21","unstructured":"G. Khromov and S. P. Singh, Some intriguing aspects about Lipschitz continuity of neural networks, in The Twelfth International Conference on Learning Representations, 2023."},{"key":"ref22","unstructured":"P. Kidger and T. Lyons, Universal approximation with deep narrow networks, in Proc. Conf. Learn. Theory, 2020, pp. 2306\u20132327."},{"key":"ref23","unstructured":"G. Koch, R. Zemel, and R. Salakhutdinov, Siamese neural networks for one-shot image recognition, in Proc. Int. Conf. Machine Learn., Vol. 2, 2015."},{"key":"ref24","unstructured":"A. Kurakin, I. Goodfellow, and S. Bengio, Adversarial machine learning at scale, in Proc. Int. Conf. Learn. Represent., 2016."},{"key":"ref25","doi-asserted-by":"crossref","unstructured":"M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana, Certified robustness to adversarial examples with differential privacy, in Proc. IEEE Symp. Security and Privacy, 2019, pp. 656\u2013672.","DOI":"10.1109\/SP.2019.00044"},{"key":"ref26","doi-asserted-by":"crossref","first-page":"788","DOI":"10.1038\/44565","volume":"401","author":"Lee D. D.","year":"1999","journal-title":"Nature"},{"key":"ref27","unstructured":"K. Leino, Z. Wang, and M. Fredrikson, Globally-robust neural networks, in Proc. Int. Conf. Mach. Learn., 2021, pp. 6212\u20136222."},{"key":"ref28","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/S0893-6080(05)80131-5","volume":"6","author":"Leshno M.","year":"1993","journal-title":"Neural Networks"},{"key":"ref29","unstructured":"B. Li, C. Chen, W. Wang, and L. Carin, Certified adversarial robustness with additive noise, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 32, Curran Associates, 2019, pp. 9464\u20139474."},{"key":"ref30","doi-asserted-by":"crossref","first-page":"2935","DOI":"10.1109\/TPAMI.2017.2773081","volume":"40","author":"Li Z.","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Machine Intell."},{"key":"ref31","first-page":"3730","volume":"4","author":"Liu Z.","year":"2015","journal-title":"Proc. IEEE Int. Conf. Comput. Vis."},{"key":"ref32","unstructured":"A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, Towards deep learning models resistant to adversarial attacks, in Proc. Int. Conf. Learn. Represent., 2018."},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"S.M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, DeepFool: A simple and accurate method to fool deep neural networks, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2016, pp. 2574\u20132582.","DOI":"10.1109\/CVPR.2016.282"},{"key":"ref34","unstructured":"L. Moroney, Rock, Paper, Scissors Dataset, 2019, https:\/\/www.tensorflow.org\/datasets\/catalog\/rock_paper_scissors."},{"key":"ref35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3635159","volume":"15","author":"Neac\u015fu A.","year":"2024","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.1137\/22M1504573"},{"key":"ref37","unstructured":"S. Parkinson, G. Ongie, and R. Willett, Linear Neural Network Layers Promote Learning Single- and Multiple-Index Models, preprint, arXiv:2305.15598, 2023."},{"key":"ref38","first-page":"121","volume-title":"IEEE Control Systems Letters","volume":"6","author":"Pauli P.","year":"2021"},{"key":"ref39","unstructured":"M. Pintor, F. Roli, W. Brendel, and B. Biggio, Fast minimum-norm adversarial attacks through adaptive norm constraints, in Proc. Ann. Conf. Neur. Inform. Proc. Syst. Vol. 34, 2021, pp. 20052\u201320062."},{"key":"ref40","unstructured":"A. Raghunathan, J. Steinhardt, and P. Liang, Certified defenses against adversarial examples, in Proc. Int. Conf. Learn. Represent., 2018."},{"key":"ref41","first-page":"10900","volume-title":"Proc. Ann. Conf. Neur. Inform. Proc. Syst.","volume":"31","author":"Raghunathan A.","year":"2018"},{"key":"ref42","doi-asserted-by":"crossref","unstructured":"J. Rony, L. G. Hafemann, L. S. Oliveira, I. B. Ayed, R. Sabourin, and E. Granger, Decoupling direction and norm for efficient gradient-based \\(\\ell_2\\) adversarial attacks and defenses, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2019, pp. 4322\u20134330.","DOI":"10.1109\/CVPR.2019.00445"},{"key":"ref43","unstructured":"S. Sabour, N. Frosst, and G. E. Hinton, Dynamic routing between capsules, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 30, Curran Associates, 2017, pp. 3856\u20133866."},{"key":"ref44","unstructured":"H. Salman, J. Li, I. Razenshteyn, P. Zhang, H. Zhang, S. Bubeck, and G. Yang, Provably robust deep learning via adversarially trained smoothed classifiers, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 32, Curran Associates, 2019, pp. 11292\u201311303."},{"key":"ref45","doi-asserted-by":"crossref","unstructured":"M. Serrurier, F. Mamalet, A. Gonz\u00e1lez-Sanz, T. Boissin, J.M. Loubes, and E. Del Barrio, Achieving robustness in classification using optimal transport with hinge regularization, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2021, pp. 505\u2013514.","DOI":"10.1109\/CVPR46437.2021.00057"},{"key":"ref46","unstructured":"A. Shafahi, P. Saadatpanah, C. Zhu, A. Ghiasi, C. Studer, D. Jacobs, and T. Goldstein, Adversarially robust transfer learning, Proc. Int. Conf. Learn. Represent., 2020."},{"key":"ref47","unstructured":"S. H. Silva and P. Najafirad, Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey, preprint, arXiv:2007.00753, 2020."},{"key":"ref48","unstructured":"A. Sinha, H. Namkoong, and J. Duchi, Certifying some distributional robustness with principled adversarial training, in Proc. Int. Conf. Learn. Represent., 2018."},{"key":"ref49","unstructured":"C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, Intriguing properties of neural networks, in Proc. Int. Conf. Learn. Represent., 2014."},{"key":"ref50","doi-asserted-by":"crossref","unstructured":"Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, DeepFace: Closing the gap to human-level performance in face verification, in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2014, pp. 1701\u20131708.","DOI":"10.1109\/CVPR.2014.220"},{"key":"ref51","unstructured":"D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry, There Is No Free Lunch in Adversarial Robustness (But There Are Unexpected Benefits), preprint, arXiv:1805.12152, 2018."},{"key":"ref52","volume-title":"Multirate Systems and Filter Banks","author":"Vaidyanathan P.","year":"1993"},{"key":"ref53","unstructured":"A. Virmaux and K. Scaman, Lipschitz regularity of deep neural networks: Analysis and efficient estimation, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 31, Curran Associates, 2018, pp. 3839\u20133848."},{"key":"ref54","doi-asserted-by":"crossref","unstructured":"Q. Wang, M. A. Powell, A. Geisa, E. Bridgeford, C. E. Priebe, and J. T. Vogelstein, Why do networks have inhibitory\/negative connections? in Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2023, pp. 22551\u201322559.","DOI":"10.1109\/ICCV51070.2023.02061"},{"key":"ref55","unstructured":"Y. Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu, Improving adversarial robustness requires revisiting misclassified examples, in Proc. Int. Conf. Learn. Represent., 2020."},{"key":"ref56","unstructured":"C. Wei and J. Z. Kolter, Certified robustness for deep equilibrium models via interval bound propagation, in Proc. Int. Conf. Learn. Represent., 2021."},{"key":"ref57","unstructured":"R. Zhai, C. Dan, D. He, H. Zhang, B. Gong, P. Ravikumar, C.J. Hsieh, and L. Wang, MACER: Attack-free and scalable robust training via maximizing certified radius, in Proc. Int. Conf. Learn. Represent., 2020."},{"key":"ref58","doi-asserted-by":"crossref","unstructured":"B. Zhang, D. Jiang, D. He, and L. Wang, Rethinking Lipschitz neural networks and certified robustness: A Boolean function perspective, in Proc. Ann. Conf. Neur. Inform. Proc. Syst., Vol. 35, Curran Associates, 2022, pp. 19398\u201319413.","DOI":"10.52202\/068431-1410"},{"key":"ref59","unstructured":"H. Zhang, Y. Yu, J. Jiao, E. Xing, L. El Ghaoui, and M. Jordan, Theoretically principled trade-off between robustness and accuracy, in Proc. Int. Conf. Machine Learn., PMLR, 2019, pp. 7472\u20137482."}],"container-title":["SIAM Journal on Mathematics of Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/23M1589591","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T17:43:34Z","timestamp":1787334214000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/23M1589591"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,15]]},"references-count":59,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1137\/23M1589591"],"URL":"https:\/\/doi.org\/10.1137\/23m1589591","relation":{},"ISSN":["2577-0187"],"issn-type":[{"value":"2577-0187","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,15]]}}}