{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T19:45:51Z","timestamp":1769283951397,"version":"3.49.0"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T00:00:00Z","timestamp":1689033600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T00:00:00Z","timestamp":1689033600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005721","name":"Universit\u00e4t Bielefeld","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The widely employed tiny neural networks (TNNs) in mobile devices are vulnerable to adversarial attacks. However, more advanced research on the robustness of TNNs is highly in demand. This work focuses on improving the robustness of TNNs without sacrificing the model\u2019s accuracy. To find the optimal trade-off networks in terms of the adversarial accuracy, clean accuracy, and model size, we present TAM-NAS, a tiny adversarial multi-objective one-shot network architecture search method. First, we build a novel search space comprised of new tiny blocks and channels to establish a balance between the model size and adversarial performance. Then, we demonstrate how the supernet facilitates the acquisition of the optimal subnet under white-box adversarial attacks, provided that the supernet significantly impacts the subnet\u2019s performance. Concretely, we investigate a new adversarial training paradigm by evaluating the adversarial transferability, the width of the supernet, and the distinction between training subnets from scratch and fine-tuning. Finally, we undertake statistical analysis for the layer-wise combination of specific blocks and channels on the first non-dominated front, which can be utilized as a design guideline for the design of TNNs.<\/jats:p>","DOI":"10.1007\/s40747-023-01139-8","type":"journal-article","created":{"date-parts":[[2023,7,11]],"date-time":"2023-07-11T07:02:06Z","timestamp":1689058926000},"page":"6117-6138","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Tiny adversarial multi-objective one-shot neural architecture search"],"prefix":"10.1007","volume":"9","author":[{"given":"Guoyang","family":"Xie","sequence":"first","affiliation":[]},{"given":"Jinbao","family":"Wang","sequence":"additional","affiliation":[]},{"given":"Guo","family":"Yu","sequence":"additional","affiliation":[]},{"given":"Jiayi","family":"Lyu","sequence":"additional","affiliation":[]},{"given":"Feng","family":"Zheng","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1100-0631","authenticated-orcid":false,"given":"Yaochu","family":"Jin","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,7,11]]},"reference":[{"key":"1139_CR1","unstructured":"Christian S, Zaremba W, Sutskever I, Joan B, Dumitru E, Ian G, Rob F (2014) Intriguing properties of neural networks"},{"key":"1139_CR2","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples"},{"key":"1139_CR3","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks"},{"key":"1139_CR4","unstructured":"Cubuk ED, Zoph B, Schoenholz SS, Le Quoc V (2018) Intriguing properties of adversarial examples"},{"key":"1139_CR5","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"1139_CR6","doi-asserted-by":"crossref","unstructured":"Su D, Zhang H, Chen H, Yi J, Chen P-Y, Gao Y (2018) Is robustness the cost of accuracy? A comprehensive study on the robustness of 18 deep image classification models. In: Proceedings of the European conference on computer vision (ECCV), pp 631\u2013648","DOI":"10.1007\/978-3-030-01258-8_39"},{"key":"1139_CR7","unstructured":"Xie C, Yuille A (2020) Intriguing properties of adversarial training"},{"key":"1139_CR8","unstructured":"Huang H, Wang Y, Erfani SM, Quanquan G, James B, Xingjun M (2021) Exploring architectural ingredients of adversarially robust deep neural networks. In: Neural information processing systems"},{"key":"1139_CR9","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/j.neucom.2021.04.111","volume":"453","author":"J Liu","year":"2021","unstructured":"Liu J, Jin Y (2021) Multi-objective search of robust neural architectures against multiple types of adversarial attacks. Neurocomputing 453:73\u201384","journal-title":"Neurocomputing"},{"key":"1139_CR10","unstructured":"Gro\u00dfhans M, Sawade C, Br\u00fcckner M, Scheffer T (2013) Bayesian games for adversarial regression problems. In: International conference on machine learning, pp 55\u201363"},{"key":"1139_CR11","unstructured":"Tsipras D, Santurkar S, Engstrom L, Turner A, Madry A (2019) Robustness may be at odds with accuracy"},{"key":"1139_CR12","doi-asserted-by":"crossref","unstructured":"Jin Y, Sendhoff B (2008) Pareto-based multiobjective machine learning: an overview and case studies. In: IEEE transactions on systems, man, and cybernetics, Part C (applications and reviews) 38(3):397\u2013415","DOI":"10.1109\/TSMCC.2008.919172"},{"key":"1139_CR13","doi-asserted-by":"crossref","unstructured":"Lu Z, Whalen I, Boddeti VN, Dhebar YD, Deb K, Goodman ED, Banzhaf W (2018) Nsga-net: neural architecture search using multi-objective genetic algorithm. In: Proceedings of the genetic and evolutionary computation conference","DOI":"10.1145\/3321707.3321729"},{"key":"1139_CR14","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1109\/TEVC.2020.3024708","volume":"25","author":"Z Lu","year":"2019","unstructured":"Lu Z, Whalen I, Dhebar YD, Deb K, Goodman ED, Wolfgang B, Vishnu NB (2019) Multiobjective evolutionary design of deep convolutional neural networks for image classification. IEEE Trans Evol Comput 25:277\u2013291","journal-title":"IEEE Trans Evol Comput"},{"key":"1139_CR15","doi-asserted-by":"crossref","unstructured":"Ma N, Zhang X, Zheng H-T, Sun J (2018) Shufflenet v2: practical guidelines for efficient cnn architecture design. In: Proceedings of the European conference on computer vision (ECCV), pp 116\u2013131","DOI":"10.1007\/978-3-030-01264-9_8"},{"key":"1139_CR16","doi-asserted-by":"crossref","unstructured":"Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251\u20131258","DOI":"10.1109\/CVPR.2017.195"},{"key":"1139_CR17","doi-asserted-by":"crossref","unstructured":"Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132\u20137141","DOI":"10.1109\/CVPR.2018.00745"},{"key":"1139_CR18","doi-asserted-by":"crossref","unstructured":"Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794\u20137803","DOI":"10.1109\/CVPR.2018.00813"},{"key":"1139_CR19","doi-asserted-by":"publisher","DOI":"10.2307\/j.ctvjsf522","volume-title":"Game theory","author":"RB Myerson","year":"2013","unstructured":"Myerson RB (2013) Game theory. Harvard University Press, Cambridge"},{"issue":"1","key":"1139_CR20","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1137\/070699652","volume":"39","author":"C Daskalakis","year":"2009","unstructured":"Daskalakis C, Goldberg PW, Papadimitriou CH (2009) The complexity of computing a nash equilibrium. SIAM J Comput 39(1):195\u2013259","journal-title":"SIAM J Comput"},{"key":"1139_CR21","unstructured":"Zhang D, Zhang T, Lu Y, Zhu Z, Dong B (2019) You only propagate once: accelerating adversarial training via maximal principle. 32"},{"key":"1139_CR22","unstructured":"Kannan H, Kurakin A, Goodfellow I (2018) Adversarial logit pairing. arXiv preprint arXiv:1803.06373"},{"key":"1139_CR23","unstructured":"Zhang H, Yu Y, Jiao J, Xing EP, Ghaoui LE, Jordan MI (2019) Theoretically principled trade-off between robustness and accuracy"},{"key":"1139_CR24","doi-asserted-by":"crossref","unstructured":"Xie L, Yuille A (2017) Genetic cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1379\u20131388","DOI":"10.1109\/ICCV.2017.154"},{"key":"1139_CR25","first-page":"4780","volume":"33","author":"E Real","year":"2019","unstructured":"Real E, Aggarwal A, Huang Y, Le QV (2019) Regularized evolution for image classifier architecture search. Proc AAAI Conf Artif Intell 33:4780\u20134789","journal-title":"Proc AAAI Conf Artif Intell"},{"key":"1139_CR26","doi-asserted-by":"crossref","unstructured":"Zoph B, Vasudevan V, Shlens J, Le Quoc V (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8697\u20138710","DOI":"10.1109\/CVPR.2018.00907"},{"key":"1139_CR27","unstructured":"Pham H, Guan MY, Zoph B, Le QV, Dean J (2018) Efficient neural architecture search via parameter sharing. 80:4092\u20134101"},{"key":"1139_CR28","unstructured":"Liu H, Simonyan K, Yiming Y (2019) Darts: differentiable architecture search"},{"key":"1139_CR29","doi-asserted-by":"crossref","unstructured":"Dong X, Yang Y (2019) One-shot neural architecture search via self-evaluated template network. In: Proceedings of the IEEE international conference on computer vision, pp 3681\u20133690","DOI":"10.1109\/ICCV.2019.00378"},{"key":"1139_CR30","doi-asserted-by":"crossref","unstructured":"Guo M, Yang Y, Xu R, Liu Z, Lin D (2020) When nas meets robustness: In search of robust architectures against adversarial attacks. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 631\u2013640","DOI":"10.1109\/CVPR42600.2020.00071"},{"key":"1139_CR31","unstructured":"Bender G, Kindermans P-J, Zoph B, Vasudevan V, Le Q (2018) Understanding and simplifying one-shot architecture search. In: International conference on machine learning, pp 550\u2013559"},{"issue":"2","key":"1139_CR32","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb K, Pratap A, Agarwal S, Meyarivan TAMT (2002) A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans Evol Comput 6(2):182\u2013197","journal-title":"IEEE Trans Evol Comput"},{"key":"1139_CR33","doi-asserted-by":"crossref","unstructured":"Huang S, Lu Z, Deb K, Boddeti VN (2022) Revisiting residual networks for adversarial robustness: an architectural perspective. arXiv preprint arXiv:2212.11005","DOI":"10.1109\/CVPR52729.2023.00793"},{"key":"1139_CR34","unstructured":"Krizhevsky A, Hinton G et\u00a0al (2009) Learning multiple layers of features from tiny images. Master\u2019s thesis, Department of Computer Science, University of Toronto"},{"key":"1139_CR35","unstructured":"Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. NIPS workshop on deep learning and unsupervised feature learning"},{"issue":"3","key":"1139_CR36","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Hao S, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis (IJCV) 115(3):211\u2013252","journal-title":"Int J Comput Vis (IJCV)"},{"key":"1139_CR37","doi-asserted-by":"crossref","unstructured":"Buades A, Coll B, Morel J-M (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR\u201905), vol 2, pp 60\u201365. IEEE","DOI":"10.1109\/CVPR.2005.38"},{"key":"1139_CR38","doi-asserted-by":"crossref","unstructured":"Guo Z, Zhang X, Haoyuan M, Heng W, Liu Z, Wei Y, Sun J (2020) Single path one-shot neural architecture search with uniform sampling. 12361:544\u2013560","DOI":"10.1007\/978-3-030-58517-4_32"},{"key":"1139_CR39","doi-asserted-by":"crossref","unstructured":"Kopp RE (1962) Pontryagin maximum principle. In: Mathematics in science and engineering, vol 5, pp 255\u2013279. Elsevier","DOI":"10.1016\/S0076-5392(08)62095-0"},{"key":"1139_CR40","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks, pp 116\u2013131","DOI":"10.1007\/978-3-319-46493-0_38"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01139-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01139-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01139-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T23:15:16Z","timestamp":1729725316000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01139-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,11]]},"references-count":40,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["1139"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01139-8","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,11]]},"assertion":[{"value":"5 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 April 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}