{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T05:38:39Z","timestamp":1783575519038,"version":"3.55.0"},"reference-count":35,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T00:00:00Z","timestamp":1728864000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Outstanding Member of Youth Innovation Promotion Association CAS","award":["Y2022052"],"award-info":[{"award-number":["Y2022052"]}]},{"name":"Outstanding Member of Youth Innovation Promotion Association CAS","award":["62131019"],"award-info":[{"award-number":["62131019"]}]},{"name":"Key Projects of National Natural Science Foundation of China","award":["Y2022052"],"award-info":[{"award-number":["Y2022052"]}]},{"name":"Key Projects of National Natural Science Foundation of China","award":["62131019"],"award-info":[{"award-number":["62131019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Deep neural network models are vulnerable to attacks from adversarial methods, such as gradient attacks. Evening small perturbations can cause significant differences in their predictions. Adversarial training (AT) aims to improve the model\u2019s adversarial robustness against gradient attacks by generating adversarial samples and optimizing the adversarial training objective function of the model. Existing methods mainly focus on improving robust accuracy, balancing natural and robust accuracy and suppressing robust overfitting. They rarely consider the AT problem from the characteristics of deep neural networks themselves, such as the stability properties under certain conditions. From a mathematical perspective, deep neural networks with stable training processes may have a better ability to suppress overfitting, as their training process is smoother and avoids sudden drops in performance. We provide a proof of the existence of Ulam stability for deep neural networks. Ulam stability not only determines the existence of the solution for an operator inequality, but it also provides an error bound between the exact and approximate solutions. The feature subspace of a deep neural network with Ulam stability can be accurately characterized and constrained by a function with special properties and a controlled error boundary constant. This restricted feature subspace leads to a more stable training process. Based on these properties, we propose an adversarial training framework called Ulam stability adversarial training (US-AT). This framework can incorporate different Ulam stability conditions and benchmark AT models, optimize the construction of the optimal feature subspace, and consistently improve the model\u2019s robustness and training stability. US-AT is simple and easy to use, and it can be easily integrated with existing multi-class AT models, such as GradAlign and TRADES. Experimental results show that US-AT methods can consistently improve the robust accuracy and training stability of benchmark models.<\/jats:p>","DOI":"10.3390\/sym16101363","type":"journal-article","created":{"date-parts":[[2024,10,14]],"date-time":"2024-10-14T05:47:58Z","timestamp":1728884878000},"page":"1363","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Enhancing Adversarial Robustness through Stable Adversarial Training"],"prefix":"10.3390","volume":"16","author":[{"given":"Kun","family":"Yan","sequence":"first","affiliation":[{"name":"Beijing Institute of Technology, Beijing 100081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Luyi","family":"Yang","sequence":"additional","affiliation":[{"name":"Foreign Language Department, Information and Engineering University, Zhengzhou 450000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhanpeng","family":"Yang","sequence":"additional","affiliation":[{"name":"The Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjuan","family":"Ren","sequence":"additional","affiliation":[{"name":"The Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,14]]},"reference":[{"key":"ref_1","unstructured":"Lei, H., Tsai, Y.Y., Chen, P.Y., and Ho, T.Y. (2023, January 17\u201324). Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada."},{"key":"ref_2","unstructured":"Goodfellow, I.J., Shlens, J., and Szegedy, C. (2015). Explaining and Harnessing Adversarial Examples. arXiv."},{"key":"ref_3","unstructured":"Kurakin, A., Goodfellow, I.J., and Bengio, S. (2016). Adversarial machine Learning at Scale. arXiv."},{"key":"ref_4","unstructured":"Maini, P., Wong, E., and Kolter, J.Z. (2020, January 12\u201318). Adversarial Robustness Against the Union of Multiple Perturbation Models. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_5","unstructured":"Mao, C.Z., Zhong, Z.Y., Yang, J.F., Vondrick, C., and Ray, B. (2019, January 8\u201314). Metric Learning for Adversarial Robustness. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_6","unstructured":"Andriushchenko, M., and Flammarion, N. (2020, January 6\u201312). Understanding and Improving Fast Adversarial Training. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Herrmann, C., Sargent, K., Jiang, L., Zabih, R., Chang, H.W., Liu, C., Krishnan, D., and Sun, D.Q. (2022, January 18\u201324). Pyramid Adversarial Training Improves ViT Performance. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01306"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, Y.X., and Xu, C. (2023, January 17\u201324). Trade-off between Robustness and Accuracy of Vision Transformers. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00730"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sun, L.T., Ke, D., Wang, X., Huang, Z.T., and Huang, K.Z. (2022). Robustness of Deep Learning-Based Specific Emitter Identification under Adversarial Attacks. Remote Sens., 14.","DOI":"10.3390\/rs14194996"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"580","DOI":"10.1007\/s12559-022-10062-y","article-title":"Minimum Power Adversarial Attacks in Communication Signal Modulation Classification with Deep Learning","volume":"15","author":"Ke","year":"2023","journal-title":"Cogn. Comput."},{"key":"ref_11","unstructured":"Tack, J., Yu, S., Jeong, J., Kim, M., Hwang, S.J., and Shin, J. (March, January 22). Consistency Regularization for Adversarial Robustness. Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), Virtual Event."},{"key":"ref_12","unstructured":"Zhang, S.D., Gao, H.C., Zhang, T.W., Zhou, Y.Y., and Wu, Z.H. (2022). Alleviating Robust Overfitting of Adversarial Training With Consistency Regularization. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1049\/rsn2.12410","article-title":"Adversarial discriminative domain adaptation for modulation classification based on Ulam stability","volume":"17","author":"Ren","year":"2023","journal-title":"Iet Radar, Sonar Navig."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.ins.2023.01.096","article-title":"A two-branch symmetric domain adaptation neural network based on Ulam stability theory","volume":"628","author":"Ren","year":"2023","journal-title":"Inf. Sci."},{"key":"ref_15","unstructured":"Zhang, H.Y., Yu, Y.D., Jiao, J.T., Xing, E.P., Ghaoui, L.E., and Jordan, M.I. (2019). Theoretically Principled Trade-off between Robustness and Accuracy. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Huang, S.H., Lu, Z.C., Deb, K., and Boddeti, V.N. (2023, January 17\u201324). Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspective. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00793"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bai, T., Luo, J.Q., Zhao, J., Wen, B.H., and Wang, Q. (2021, January 19\u201326). Recent Advances in Adversarial Training for Adversarial Robustness. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, QC, Canada.","DOI":"10.24963\/ijcai.2021\/591"},{"key":"ref_18","unstructured":"Wong, E., Rice, L., and Kolter, J.Z. (2020). Fast is better than free: Revisiting adversarial training. arXiv."},{"key":"ref_19","unstructured":"Shafahi, A., Najibi, M., Ghiasi, A., Xu, Z., Dickerson, J., Studer, C., Davis, L.S., Taylor, G., and Goldstein, T. (2019). Adversarial Training for Free!. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, T., Wu, Y., Chen, S., Fang, K., and Huang, X. (2022, January 18\u201324). Subspace Adversarial Training. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01305"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.1109\/TIFS.2022.3184262","article-title":"Mutual Adversarial Training: Learning Together is Better Than Going Alone","volume":"17","author":"Liu","year":"2022","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gong, H.H., Dong, M.J., Ma, S.Q., Camtepe, S., Nepal, S., and Xu, C. (2024, January 16\u201324). Random Entangled Tokens for Adversarially Robust Vision Transformer. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02318"},{"key":"ref_23","unstructured":"Rice, L., Wong, E., and Kolter, Z. (2020, January 12\u201318). Overfitting in Adversarially Robust Deep Learning. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yu, C.J., Han, B., Shen, L., Yu, J., Gong, C., Gong, M.M., and Liu, T.L. (2022, January 23\u201325). Understanding Robust Overfitting of Adversarial Training and Beyond. Proceedings of the International Conference on Machine Learning, Hangzhou, China.","DOI":"10.24963\/ijcai.2022\/512"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Tang, L.Y., and Zhang, L. (2024, January 16\u201324). Robust Overfitting Does Matter: Test-Time Adversarial Purification with FGSM. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02298"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhou, Y.H., and Hua, Z.Y. (2024, January 16\u201324). Defense without Forgetting: Continual Adversarial Defense with Anisotropic & Isotropic Pseudo Replay. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02290"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, L., Guan, H.Y., Qiu, J.N., and Spratling, M. (2024, January 16\u201324). One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02304"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wang, Y.T., Fu, H.Y., Zou, W., and Jia, J.Y. (2024, January 16\u201324). MMCert: Provable Defense against Adversarial Attacks to Multi-modal Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.02328"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3137","DOI":"10.1090\/S0002-9939-98-04680-2","article-title":"Hyers-Ulam-Rassias Stability of Jensen Equation and Its Applications","volume":"126","author":"Jung","year":"1998","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1073\/pnas.27.4.222","article-title":"On The Stability of Linear Functional Equation","volume":"27","author":"Hyers","year":"1941","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1639","DOI":"10.1007\/s00025-015-0448-0","article-title":"A Fixed Point Approach to the Stability of Functional Equations on Noncommutative Spaces","volume":"72","author":"Xu","year":"2017","journal-title":"Results Math."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1090\/S0002-9904-1945-08337-2","article-title":"On Approximate Isometries","volume":"51","author":"Hyers","year":"1945","journal-title":"Bull. Amer. Math. Soc."},{"key":"ref_33","unstructured":"Krizhevsky, A., and Hinton, G. (2024, September 03). Learning Multiple Layers of Features from Tiny Images. Available online: https:\/\/www.semanticscholar.org\/paper\/Learning-Multiple-Layers-of-Features-from-Tiny-Krizhevsky\/5d90f06bb70a0a3dced62413346235c02b1aa086."},{"key":"ref_34","unstructured":"O\u2019Shea, T.J. (2024, September 03). Deepsig. Available online: http:\/\/www.deepsig.io\/."},{"key":"ref_35","unstructured":"Sriramanan, G., Addepalli, S., Baburaj, A., and Babu, R.V. (2019, January 8\u201314). Guided Adversarial Attack for Evaluating and Enhancing Adversarial Defenses. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canad."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/10\/1363\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:12:57Z","timestamp":1760112777000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/16\/10\/1363"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,14]]},"references-count":35,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["sym16101363"],"URL":"https:\/\/doi.org\/10.3390\/sym16101363","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,14]]}}}