{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:12:47Z","timestamp":1750219967251,"version":"3.41.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2023,6,7]],"date-time":"2023-06-07T00:00:00Z","timestamp":1686096000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022ZD0210500"],"award-info":[{"award-number":["2022ZD0210500"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61972067\/U21A20491\/UI908214"],"award-info":[{"award-number":["61972067\/U21A20491\/UI908214"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Distinguished Young Scholars Funding of Dalian","award":["2022RJ01"],"award-info":[{"award-number":["2022RJ01"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Deep neural networks (DNNs) are widely used for computer vision tasks. However, it has been shown that deep models are vulnerable to adversarial attacks\u2014that is, their performances drop when imperceptible perturbations are made to the original inputs, which may further degrade the following visual tasks or introduce new problems such as data and privacy security. Hence, metrics for evaluating the robustness of deep models against adversarial attacks are desired. However, previous metrics are mainly proposed for evaluating the adversarial robustness of shallow networks on the small-scale datasets. Although the Cross Lipschitz Extreme Value for nEtwork Robustness (CLEVER) metric has been proposed for large-scale datasets (e.g., the ImageNet dataset), it is computationally expensive and its performance relies on a tractable number of samples. In this article, we propose the Adversarial Converging Time Score (ACTS), an attack-dependent metric that quantifies the adversarial robustness of a DNN on a specific input. Our key observation is that local neighborhoods on a DNN\u2019s output surface would have different shapes given different inputs. Hence, given different inputs, it requires different time for converging to an adversarial sample. Based on this geometry meaning, the ACTS measures the converging time as an adversarial robustness metric. We validate the effectiveness and generalization of the proposed ACTS metric against different adversarial attacks on the large-scale ImageNet dataset using state-of-the-art deep networks. Extensive experiments show that our ACTS metric is an efficient and effective adversarial metric over the previous CLEVER metric.<\/jats:p>","DOI":"10.1145\/3587936","type":"journal-article","created":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T11:43:40Z","timestamp":1678880620000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3369-6772","authenticated-orcid":false,"given":"Yang","family":"Wang","sequence":"first","affiliation":[{"name":"Dalian University of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9189-9506","authenticated-orcid":false,"given":"Bo","family":"Dong","sequence":"additional","affiliation":[{"name":"Princeton University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5855-3810","authenticated-orcid":false,"given":"Ke","family":"Xu","sequence":"additional","affiliation":[{"name":"City University of Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8519-4750","authenticated-orcid":false,"given":"Haiyin","family":"Piao","sequence":"additional","affiliation":[{"name":"Northwestern Polytechnical University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8716-5793","authenticated-orcid":false,"given":"Yufei","family":"Ding","sequence":"additional","affiliation":[{"name":"University of California, Santa Barbara"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3121-1823","authenticated-orcid":false,"given":"Baocai","family":"Yin","sequence":"additional","affiliation":[{"name":"Dalian University of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8046-722X","authenticated-orcid":false,"given":"Xin","family":"Yang","sequence":"additional","affiliation":[{"name":"Dalian University of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,6,7]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_1_2_2","DOI":"10.1109\/ACCESS.2018.2807385"},{"key":"e_1_3_1_3_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Athalye Anish","year":"2018","unstructured":"Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_4_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Athalye Anish","year":"2018","unstructured":"Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing robust adversarial examples. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_5_2","volume-title":"Advances in Neural Information Processing Systems","author":"Bastani Osbert","year":"2016","unstructured":"Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. 2016. Measuring neural net robustness with constraints. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_1_6_2","article-title":"Dimensionality reduction as a defense against evasion attacks on machine learning classifiers","author":"Bhagoji Arjun Nitin","year":"2017","unstructured":"Arjun Nitin Bhagoji, Daniel Cullina, and Prateek Mittal. 2017. Dimensionality reduction as a defense against evasion attacks on machine learning classifiers. arXiv:1704.02654 (2017).","journal-title":"arXiv:1704.02654"},{"key":"e_1_3_1_7_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Brendel Wieland","year":"2018","unstructured":"Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2018. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. In Proceedings of the International Conference on Learning Representations."},{"doi-asserted-by":"publisher","key":"e_1_3_1_8_2","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_1_9_2","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Chen Pin Yu","year":"2017","unstructured":"Pin Yu Chen, Yash Sharma, Huan Zhang, Jinfeng Yi, and Cho Jui Hsieh. 2017. EAD: Elastic-net attacks to deep neural networks via adversarial examples. In Proceedings of the AAAI Conference on Artificial Intelligence."},{"doi-asserted-by":"publisher","key":"e_1_3_1_10_2","DOI":"10.1145\/3128572.3140448"},{"key":"e_1_3_1_11_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Cheng Minhao","year":"2018","unstructured":"Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. 2018. Query-efficient hard-label black-box attack: An optimization-based approach. In Proceedings of the International Conference on Learning Representations."},{"doi-asserted-by":"publisher","key":"e_1_3_1_12_2","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_13_2","volume-title":"Advances in Neural Information Processing Systems","author":"Ding Jianchuan","year":"2022","unstructured":"Jianchuan Ding, Bo Dong, Felix Heide, Yufei Ding, Yunduo Zhou, Baocai Yin, and Xin Yang. 2022. Biologically inspired dynamic thresholds for spiking neural networks. In Advances in Neural Information Processing Systems."},{"doi-asserted-by":"publisher","key":"e_1_3_1_14_2","DOI":"10.1109\/CVPR42600.2020.00040"},{"doi-asserted-by":"publisher","key":"e_1_3_1_15_2","DOI":"10.1109\/CVPR.2018.00957"},{"doi-asserted-by":"publisher","key":"e_1_3_1_16_2","DOI":"10.1109\/CVPR.2019.00444"},{"doi-asserted-by":"publisher","key":"e_1_3_1_17_2","DOI":"10.1109\/CVPR.2019.00790"},{"key":"e_1_3_1_18_2","article-title":"A rotation and a translation suffice: Fooling CNNs with simple transformations","author":"Engstrom Logan","year":"2017","unstructured":"Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. 2017. A rotation and a translation suffice: Fooling CNNs with simple transformations. arXiv:1712.02779 (2017).","journal-title":"arXiv:1712.02779"},{"doi-asserted-by":"publisher","key":"e_1_3_1_19_2","DOI":"10.1145\/3524619"},{"doi-asserted-by":"publisher","key":"e_1_3_1_20_2","DOI":"10.1109\/SP.2018.00058"},{"doi-asserted-by":"publisher","key":"e_1_3_1_21_2","DOI":"10.1609\/aaai.v33i01.3301541"},{"key":"e_1_3_1_22_2","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow Ian J.","year":"2014","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv:1412.6572 (2014).","journal-title":"arXiv:1412.6572"},{"doi-asserted-by":"publisher","key":"e_1_3_1_23_2","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_24_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Ilyas Andrew","year":"2018","unstructured":"Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_25_2","article-title":"Adversarial logit pairing","author":"Kannan Harini","year":"2018","unstructured":"Harini Kannan, Alexey Kurakin, and Ian J. Goodfellow. 2018. Adversarial logit pairing. arXiv:1803.06373 (2018).","journal-title":"arXiv:1803.06373"},{"doi-asserted-by":"publisher","key":"e_1_3_1_26_2","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"e_1_3_1_27_2","article-title":"Adversarial examples in the physical world","author":"Kurakin Alexey","year":"2016","unstructured":"Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2016. Adversarial examples in the physical world. arXiv:1607.02533 (2016).","journal-title":"arXiv:1607.02533"},{"key":"e_1_3_1_28_2","article-title":"Adversarial attacks and defences competition","author":"Kurakin Alexey","year":"2018","unstructured":"Alexey Kurakin, Ian J. Goodfellow, Samy Bengio, Yinpeng Dong, Fangzhou Liao, Ming Liang, Tianyu Pang, et\u00a0al. 2018. Adversarial attacks and defences competition. arXiv:1804.00097 (2018).","journal-title":"arXiv:1804.00097"},{"doi-asserted-by":"publisher","key":"e_1_3_1_29_2","DOI":"10.1109\/TCYB.2019.2914099"},{"issue":"3","key":"e_1_3_1_30_2","article-title":"Learning to fool the speaker recognition","volume":"17","author":"Li Jiguo","year":"2021","unstructured":"Jiguo Li, Xinfeng Zhang, Jizheng Xu, Siwei Ma, and Wen Gao. 2021. Learning to fool the speaker recognition. ACM Transactions on Multimedia Computing 17, 3s (2021), Article 109, 21 pages.","journal-title":"ACM Transactions on Multimedia Computing"},{"key":"e_1_3_1_31_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Li Yandong","year":"2019","unstructured":"Yandong Li, Lijun Li, Liqiang Wang, Tong Zhang, and Boqing Gong. 2019. NATTACK: Learning the distributions of adversarial examples for an improved black-box attack on deep neural networks. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_32_2","article-title":"Detecting adversarial examples in deep networks with adaptive noise reduction","author":"Liang Bin","year":"2017","unstructured":"Bin Liang, Hongcheng Li, Miaoqiang Su, Xirong Li, Wenchang Shi, and Xiaofeng Wang. 2017. Detecting adversarial examples in deep networks with adaptive noise reduction. arXiv:1705.08378 (2017).","journal-title":"arXiv:1705.08378"},{"doi-asserted-by":"publisher","key":"e_1_3_1_33_2","DOI":"10.1007\/978-3-030-01234-2_23"},{"key":"e_1_3_1_34_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Liu Xuanqing","year":"2018","unstructured":"Xuanqing Liu, Yao Li, Chongruo Wu, and Cho-Jui Hsieh. 2018. Adv-BNN: Improved adversarial defense through robust Bayesian neural network. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_1_35_2","article-title":"Towards deep learning models resistant to adversarial attacks","author":"Madry Aleksander","year":"2017","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv:1706.06083 (2017).","journal-title":"arXiv:1706.06083"},{"key":"e_1_3_1_36_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Moosavi-Dezfooli Seyed-Mohsen","year":"2018","unstructured":"Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, Pascal Frossard, and Stefano Soatto. 2018. Robustness of classifiers to universal perturbations: A geometric perspective. In Proceedings of the International Conference on Learning Representations."},{"doi-asserted-by":"publisher","key":"e_1_3_1_37_2","DOI":"10.1109\/CVPR.2016.282"},{"key":"e_1_3_1_38_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Pang Tianyu","year":"2019","unstructured":"Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. 2019. Improving adversarial robustness via promoting ensemble diversity. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_39_2","article-title":"Extending defensive distillation","author":"Papernot Nicolas","year":"2017","unstructured":"Nicolas Papernot and Patrick McDaniel. 2017. Extending defensive distillation. arXiv:1705.05264 (2017).","journal-title":"arXiv:1705.05264"},{"doi-asserted-by":"publisher","key":"e_1_3_1_40_2","DOI":"10.1145\/3052973.3053009"},{"doi-asserted-by":"publisher","key":"e_1_3_1_41_2","DOI":"10.1109\/EuroSP.2016.36"},{"doi-asserted-by":"publisher","key":"e_1_3_1_42_2","DOI":"10.1109\/SP.2016.41"},{"unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. In Proceedings of the 31st Conference on Neural Information Processing Systems (NeurIPS\u201917) .","key":"e_1_3_1_43_2"},{"key":"e_1_3_1_44_2","volume-title":"Advances in Neural Information Processing Systems","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, et\u00a0al. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_1_45_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014).","journal-title":"arXiv:1409.1556"},{"unstructured":"Aman Sinha Hongseok Namkoong Riccardo Volpi and John Duchi. 2017. Certifying some distributional robustness with principled adversarial training. In Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917) .","key":"e_1_3_1_46_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_47_2","DOI":"10.1007\/978-3-030-01258-8_39"},{"doi-asserted-by":"publisher","key":"e_1_3_1_48_2","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_3_1_49_2","article-title":"Intriguing properties of neural networks","author":"Szegedy Christian","year":"2013","unstructured":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv:1312.6199 (2013).","journal-title":"arXiv:1312.6199"},{"issue":"2","key":"e_1_3_1_50_2","article-title":"An image privacy protection algorithm based on adversarial perturbation generative networks","volume":"17","author":"Tong Chao","year":"2021","unstructured":"Chao Tong, Mengze Zhang, Chao Lang, and Zhigao Zheng. 2021. An image privacy protection algorithm based on adversarial perturbation generative networks. ACM Transactions on Multimedia Computing 17, 2 (2021), Article 43, 14 pages.","journal-title":"ACM Transactions on Multimedia Computing"},{"key":"e_1_3_1_51_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Weng Tsui Wei","year":"2018","unstructured":"Tsui Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho Jui Hsieh, Duane Boning, Inderjit S. Dhillon, and Luca Daniel. 2018. Towards fast computation of certified robustness for ReLU networks. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_52_2","article-title":"Evaluating the robustness of neural networks: An extreme value theory approach","author":"Weng Tsui-Wei","year":"2018","unstructured":"Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. 2018. Evaluating the robustness of neural networks: An extreme value theory approach. arXiv:1801.10578 (2018).","journal-title":"arXiv:1801.10578"},{"unstructured":"Wikipedia. 2018. Jacobian Matrix and Determinant. Retrieved February 26 2018 from https:\/\/www.en.wikipedia.org\/wiki\/Jacobian_matrix_and_determinant.","key":"e_1_3_1_53_2"},{"key":"e_1_3_1_54_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations","author":"Wong Eric","year":"2020","unstructured":"Eric Wong, Leslie Rice, and J. Zico Kolter. 2020. Fast is better than free: Revisiting adversarial training. In Proceedings of the 8th International Conference on Learning Representations."},{"doi-asserted-by":"publisher","key":"e_1_3_1_55_2","DOI":"10.1109\/CVPR.2019.00284"},{"doi-asserted-by":"publisher","key":"e_1_3_1_56_2","DOI":"10.1109\/TCYB.2018.2882908"},{"key":"e_1_3_1_57_2","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang Huan","year":"2018","unstructured":"Huan Zhang, Tsui-Wei Weng, Pin-Yu Chen, Cho-Jui Hsieh, and Luca Daniel. 2018. Efficient neural network robustness certification with general activation functions. In Advances in Neural Information Processing Systems."},{"key":"e_1_3_1_58_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhang Hongyang","year":"2019","unstructured":"Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan. 2019. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the International Conference on Machine Learning."},{"doi-asserted-by":"publisher","key":"e_1_3_1_59_2","DOI":"10.1109\/ICCV48922.2021.01280"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587936","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3587936","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:14Z","timestamp":1750182554000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587936"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,7]]},"references-count":58,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3587936"],"URL":"https:\/\/doi.org\/10.1145\/3587936","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2023,6,7]]},"assertion":[{"value":"2022-09-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}