{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T11:51:45Z","timestamp":1780487505482,"version":"3.54.1"},"reference-count":48,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,5,5]],"date-time":"2025-05-05T00:00:00Z","timestamp":1746403200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,5]],"date-time":"2025-05-05T00:00:00Z","timestamp":1746403200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Centre for Research & Technology Hellas"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Adversarial attacks pose a threat to neural networks, requiring robust methods to mitigate them. Adversarial Training has emerged as a promising approach; however, its practical application in real-world deep learning systems is hindered by the trade-offs between efficiency and robustness, as optimizing for one aspect may come at cost of the other. This paper presents a comprehensive investigation into the impact of different Adversarial Training approaches and model types on the robustness of adversarially trained models, while considering the dynamic trade-offs involved. Leveraging our previously published method, Delayed Adversarial Training with Non-Sequential Adversarial Epochs \u2013 DATNS, we conduct extended empirical analyses through new experiments to effectively balance these trade-offs and navigate the interplay between efficiency and robustness, as well as catastrophic forgetting and interpretability. By providing our insights on the discussed trade-offs this research aims to enable the development of more efficient, robust, and interpretable models against adversarial attacks.<\/jats:p>","DOI":"10.1007\/s11063-025-11751-z","type":"journal-article","created":{"date-parts":[[2025,5,5]],"date-time":"2025-05-05T06:58:53Z","timestamp":1746428333000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Dynamic Trade-Offs in Adversarial Training: Exploring Efficiency, Robustness, Forgetting, and Interpretability"],"prefix":"10.1007","volume":"57","author":[{"given":"Efi","family":"Kafali","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Theodoros","family":"Semertzidis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Petros","family":"Daras","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,5,5]]},"reference":[{"key":"11751_CR1","unstructured":"Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. arxiv 2013. arXiv preprint arXiv:1312.6199"},{"key":"11751_CR2","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572"},{"key":"11751_CR3","doi-asserted-by":"crossref","unstructured":"Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 427\u2013436","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"11751_CR4","doi-asserted-by":"crossref","unstructured":"Moosavi-Dezfooli S-M, Fawzi A, Frossard P (2016) Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2574\u20132582","DOI":"10.1109\/CVPR.2016.282"},{"key":"11751_CR5","unstructured":"Shafahi A, Najibi M, Ghiasi MA, Xu Z, Dickerson J, Studer C, Davis LS, Taylor G, Goldstein T (2019) Adversarial training for free! In: Advances in Neural Information Processing Systems, pp 3358\u20133369"},{"key":"11751_CR6","unstructured":"Wang Y, Ma X, Bailey J, Yi J, Zhou B, Gu Q (2019) On the convergence and robustness of adversarial training. In: ICML 1:2"},{"key":"11751_CR7","unstructured":"Duesterwald E, Murthi A, Venkataraman G, Sinn M, Vijaykeerthy D (2019) Exploring the hyperparameter landscape of adversarial robustness. arXiv preprint arXiv:1905.03837"},{"key":"11751_CR8","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083"},{"key":"11751_CR9","doi-asserted-by":"crossref","unstructured":"Zhou X, Tsang IW, Yin J (2022) Latent boundary-guided adversarial training. arXiv preprint arXiv:2206.03717","DOI":"10.1007\/s10994-022-06203-x"},{"key":"11751_CR10","doi-asserted-by":"crossref","unstructured":"Kafali E, Semertzidis T, Daras P (2021) Delayed adversarial training with non-sequential adversarial epochs. In: 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), pp 1363\u20131367 . IEEE","DOI":"10.1109\/ICTAI52525.2021.00217"},{"issue":"7","key":"11751_CR11","doi-asserted-by":"publisher","first-page":"9721","DOI":"10.1007\/s11063-023-11223-2","volume":"55","author":"A Boufssasse","year":"2023","unstructured":"Boufssasse A, Hssayni Eh, Joudar N-E, Ettaouil M (2023) A multi-objective optimization model for redundancy reduction in convolutional neural networks. Neural Process Lett. 55(7):9721\u20139741","journal-title":"Neural Process Lett."},{"issue":"6","key":"11751_CR12","doi-asserted-by":"publisher","first-page":"2056","DOI":"10.1111\/coin.12556","volume":"38","author":"EH Hssayni","year":"2022","unstructured":"Hssayni EH, Joudar N-E, Ettaouil M (2022) A deep learning framework for time series classification using normal cloud representation and convolutional neural network optimization. Comput Intell 38(6):2056\u20132074","journal-title":"Comput Intell"},{"key":"11751_CR13","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.109567","volume":"253","author":"N-E Joudar","year":"2022","unstructured":"Joudar N-E, Ettaouil M et al (2022) An adaptive drop method for deep neural networks regularization: estimation of dropconnect hyperparameter using generalization gap. Knowl-Based Syst 253:109567","journal-title":"Knowl-Based Syst"},{"key":"11751_CR14","unstructured":"Silva SH, Najafirad P (2020) Opportunities and challenges in deep learning adversarial robustness: a survey. arXiv preprint arXiv:2007.00753"},{"key":"11751_CR15","unstructured":"Tram\u00e8r F, Kurakin A, Papernot N, Goodfellow I, Boneh D, McDaniel P (2017) Ensemble adversarial training: attacks and defenses. arXiv preprint arXiv:1705.07204"},{"key":"11751_CR16","doi-asserted-by":"crossref","unstructured":"Liu X, Cheng M, Zhang H, Hsieh C-J (2018) Towards robust neural networks via random self-ensemble. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 369\u2013385","DOI":"10.1007\/978-3-030-01234-2_23"},{"key":"11751_CR17","unstructured":"Sen S, Ravindran B, Raghunathan A (2020) Empir: ensembles of mixed precision deep networks for increased robustness against adversarial attacks. arXiv preprint arXiv:2004.10162"},{"key":"11751_CR18","unstructured":"Wong E, Rice L, Kolter JZ (2020) Fast is better than free: revisiting adversarial training. arXiv preprint arXiv:2001.03994"},{"key":"11751_CR19","unstructured":"Andriushchenko M, Flammarion N (2020) Understanding and improving fast adversarial training. arXiv preprint arXiv:2007.02617"},{"key":"11751_CR20","unstructured":"Croce F, Hein M (2020) Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. arXiv preprint arXiv:2003.01690"},{"key":"11751_CR21","unstructured":"Shaham U, Yamada Y, Negahban S (2015) Understanding adversarial training: increasing local stability of neural nets through robust optimization. arXiv preprint arXiv:1511.05432"},{"key":"11751_CR22","doi-asserted-by":"publisher","first-page":"155161","DOI":"10.1109\/ACCESS.2021.3127960","volume":"9","author":"N Akhtar","year":"2021","unstructured":"Akhtar N, Mian A, Kardan N, Shah M (2021) Advances in adversarial attacks and defenses in computer vision: a survey. IEEE Access 9:155161\u2013155196. https:\/\/doi.org\/10.1109\/ACCESS.2021.3127960","journal-title":"IEEE Access"},{"key":"11751_CR23","unstructured":"Gao R, Cai T, Li H, Hsieh C-J, Wang L, Lee JD (2019) Convergence of adversarial training in overparametrized neural networks. In: Advances in Neural Information Processing Systems, pp 13029\u201313040"},{"key":"11751_CR24","first-page":"7054","volume":"34","author":"B Wu","year":"2021","unstructured":"Wu B, Chen J, Cai D, He X, Gu Q (2021) Do wider neural networks really help adversarial robustness? Adv Neural Inform Process Syst 34:7054\u20137067","journal-title":"Adv Neural Inform Process Syst"},{"key":"11751_CR25","unstructured":"Zhang H, Yu Y, Jiao J, Xing E, El\u00a0Ghaoui L, Jordan M (2019) Theoretically principled trade-off between robustness and accuracy. In: International Conference on Machine Learning, pp 7472\u20137482 . PMLR"},{"key":"11751_CR26","doi-asserted-by":"crossref","unstructured":"Zheng H, Zhang Z, Gu J, Lee H, Prakash A (2020) Efficient adversarial training with transferable adversarial examples. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp 1181\u20131190","DOI":"10.1109\/CVPR42600.2020.00126"},{"key":"11751_CR27","doi-asserted-by":"crossref","unstructured":"Cai Q-Z, Du M, Liu C, Song D (2018) Curriculum adversarial training. arXiv preprint arXiv:1805.04807","DOI":"10.24963\/ijcai.2018\/520"},{"key":"11751_CR28","doi-asserted-by":"crossref","unstructured":"Mendon\u00e7a MO, Maroto J, Frossard P, Diniz PS (2022) Adversarial training with informed data selection. In: 2022 30th European Signal Processing Conference (EUSIPCO), pp 608\u2013612 . IEEE","DOI":"10.23919\/EUSIPCO55093.2022.9909845"},{"key":"11751_CR29","doi-asserted-by":"crossref","unstructured":"Gupta S, Dube P, Verma A (2020) Improving the affordability of robustness training for dnns. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 780\u2013781","DOI":"10.1109\/CVPRW50498.2020.00398"},{"issue":"2","key":"11751_CR30","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1007\/s10994-022-06283-9","volume":"112","author":"H Hihn","year":"2023","unstructured":"Hihn H, Braun DA (2023) Hierarchically structured task-agnostic continual learning. Mach Learn 112(2):655\u2013686","journal-title":"Mach Learn"},{"key":"11751_CR31","unstructured":"Lee S-W, Kim J-H, Jun J, Ha J-W, Zhang B-T (2017) Overcoming catastrophic forgetting by incremental moment matching. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/f708f064faaf32a43e4d3c784e6af9ea-Paper.pdf"},{"issue":"6","key":"11751_CR32","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1016\/j.tics.2017.04.001","volume":"21","author":"ME Hasselmo","year":"2017","unstructured":"Hasselmo ME (2017) Avoiding catastrophic forgetting. Trends Cognit Sci 21(6):407\u2013408. https:\/\/doi.org\/10.1016\/j.tics.2017.04.001","journal-title":"Avoiding catastrophic forgetting. Trends Cognit Sci"},{"key":"11751_CR33","doi-asserted-by":"publisher","first-page":"88279","DOI":"10.1109\/ACCESS.2021.3090672","volume":"9","author":"J Hurtado","year":"2021","unstructured":"Hurtado J, Lobel H, Soto A (2021) Overcoming catastrophic forgetting using sparse coding and meta learning. IEEE Access 9:88279\u201388290. https:\/\/doi.org\/10.1109\/ACCESS.2021.3090672","journal-title":"IEEE Access"},{"key":"11751_CR34","first-page":"8119","volume":"35","author":"H Kim","year":"2021","unstructured":"Kim H, Lee W, Lee J (2021) Understanding catastrophic overfitting in single-step adversarial training. Proc AAAI Conf Artif Intell 35:8119\u20138127","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"11751_CR35","doi-asserted-by":"crossref","unstructured":"Li T, Wu Y, Chen S, Fang K, Huang X (2022) Subspace adversarial training. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp 13409\u201313418","DOI":"10.1109\/CVPR52688.2022.01305"},{"key":"11751_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.neunet.2022.02.015","volume":"150","author":"S Chen","year":"2022","unstructured":"Chen S, Shen H, Wang R, Wang X (2022) Towards improving fast adversarial training in multi-exit network. Neural Netw 150:1\u201311","journal-title":"Neural Netw"},{"key":"11751_CR37","unstructured":"Wen S, Itti L (2018) Overcoming catastrophic forgetting problem by weight consolidation and long-term memory. arXiv preprint arXiv:1805.07441"},{"key":"11751_CR38","first-page":"4356","volume":"34","author":"X Dong","year":"2021","unstructured":"Dong X, Luu AT, Lin M, Yan S, Zhang H (2021) How should pre-trained language models be fine-tuned towards adversarial robustness? Adv Neural Inform Process Syst 34:4356\u20134369","journal-title":"Adv Neural Inform Process Syst"},{"issue":"11","key":"11751_CR39","doi-asserted-by":"publisher","first-page":"2266","DOI":"10.1162\/neco_a_01232","volume":"31","author":"X Yao","year":"2019","unstructured":"Yao X, Huang T, Wu C, Zhang R-X, Sun L (2019) Adversarial feature alignment: avoid catastrophic forgetting in incremental task lifelong learning. Neural Comput 31(11):2266\u20132291","journal-title":"Neural Comput"},{"key":"11751_CR40","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2024.110281","volume":"149","author":"K Fang","year":"2024","unstructured":"Fang K, Tao Q, Wu Y, Li T, Cai J, Cai F, Huang X, Yang J (2024) Towards robust neural networks via orthogonal diversity. Pattern Recognit 149:110281","journal-title":"Pattern Recognit"},{"key":"11751_CR41","unstructured":"Tsipras D, Santurkar S, Engstrom L, Turner A, Madry A (2019) Robustness may be at odds with accuracy. In: International Conference on Learning Representations"},{"key":"11751_CR42","unstructured":"Etmann C, Lunz S, Maass P, Schoenlieb C (2019) On the connection between adversarial robustness and saliency map interpretability. In: International Conference on Machine Learning, pp 1823\u20131832 . PMLR"},{"key":"11751_CR43","unstructured":"Kim B, Seo J, Jeon T (2019) Bridging adversarial robustness and gradient interpretability. arXiv:1903.11626"},{"issue":"4","key":"11751_CR44","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1109\/MSP.2022.3142719","volume":"39","author":"IE Nielsen","year":"2022","unstructured":"Nielsen IE, Dera D, Rasool G, Ramachandran RP, Bouaynaya NC (2022) Robust explainability: a tutorial on gradient-based attribution methods for deep neural networks. IEEE Signal Process Mag 39(4):73\u201384","journal-title":"IEEE Signal Process Mag"},{"key":"11751_CR45","unstructured":"Zhang T, Zhu Z (2019) Interpreting adversarially trained convolutional neural networks. In: International Conference on Machine Learning, pp 7502\u20137511 . PMLR"},{"key":"11751_CR46","doi-asserted-by":"crossref","unstructured":"Gavrikov P, Keuper J, Keuper M (2023) An extended study of human-like behavior under adversarial training. arXiv preprint arXiv:2303.12669","DOI":"10.1109\/CVPRW59228.2023.00233"},{"key":"11751_CR47","doi-asserted-by":"crossref","unstructured":"Mao C, Gupta A, Nitin V, Ray B, Song S, Yang J, Vondrick C (2020) Multitask learning strengthens adversarial robustness. In: European Conference on Computer Vision, pp 158\u2013174 . Springer","DOI":"10.1007\/978-3-030-58536-5_10"},{"key":"11751_CR48","first-page":"25179","volume":"34","author":"Z Deng","year":"2021","unstructured":"Deng Z, Zhang L, Vodrahalli K, Kawaguchi K, Zou JY (2021) Adversarial training helps transfer learning via better representations. Adv Neural Inform Process Syst 34:25179\u201325191","journal-title":"Adv Neural Inform Process Syst"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-025-11751-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-025-11751-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-025-11751-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T17:03:35Z","timestamp":1750957415000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-025-11751-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,5]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["11751"],"URL":"https:\/\/doi.org\/10.1007\/s11063-025-11751-z","relation":{},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,5]]},"assertion":[{"value":"6 March 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 May 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Efi Kafali, Theodoros Semertzidis and Petros Daras have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interests"}},{"value":"Not applicable","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"All authors consent for the publication of this article.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"47"}}