{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T12:28:24Z","timestamp":1762432104034,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":89,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,11]],"date-time":"2021-07-11T00:00:00Z","timestamp":1625961600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100007162","name":"Guangdong Science and Technology Department","doi-asserted-by":"publisher","award":["2018B010107004"],"award-info":[{"award-number":["2018B010107004"]}],"id":[{"id":"10.13039\/501100007162","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62072309"],"award-info":[{"award-number":["62072309"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001459","name":"Ministry of Education - Singapore","doi-asserted-by":"publisher","award":["19-C220-SMU-002"],"award-info":[{"award-number":["19-C220-SMU-002"]}],"id":[{"id":"10.13039\/501100001459","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,11]]},"DOI":"10.1145\/3460319.3464822","type":"proceedings-article","created":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T22:18:43Z","timestamp":1625782723000},"page":"42-55","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":36,"title":["Attack as defense: characterizing adversarial examples using robustness"],"prefix":"10.1145","author":[{"given":"Zhe","family":"Zhao","sequence":"first","affiliation":[{"name":"ShanghaiTech University, China"}]},{"given":"Guangke","family":"Chen","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, China"}]},{"given":"Jingyi","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8011-5868","authenticated-orcid":false,"given":"Yiwei","family":"Yang","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, China"}]},{"given":"Fu","family":"Song","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3545-1392","authenticated-orcid":false,"given":"Jun","family":"Sun","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore"}]}],"member":"320","published-online":{"date-parts":[[2021,7,11]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Apollo. 2018. An open reliable and secure software platform for autonomous driving systems. http:\/\/apollo.auto  Apollo. 2018. An open reliable and secure software platform for autonomous driving systems. http:\/\/apollo.auto"},{"volume-title":"Proceedings of the 35th International Conference on Machine Learning. 274\u2013283","author":"Athalye Anish","key":"e_1_3_2_1_2_1","unstructured":"Anish Athalye , Nicholas Carlini , and David A. Wagner . 2018. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples . In Proceedings of the 35th International Conference on Machine Learning. 274\u2013283 . Anish Athalye, Nicholas Carlini, and David A. Wagner. 2018. Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples. In Proceedings of the 35th International Conference on Machine Learning. 274\u2013283."},{"key":"e_1_3_2_1_3_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning. 284\u2013293","author":"Athalye Anish","year":"2018","unstructured":"Anish Athalye , Logan Engstrom , Andrew Ilyas , and Kevin Kwok . 2018 . Synthesizing Robust Adversarial Examples . In Proceedings of the 35th International Conference on Machine Learning. 284\u2013293 . Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing Robust Adversarial Examples. In Proceedings of the 35th International Conference on Machine Learning. 284\u2013293."},{"key":"e_1_3_2_1_4_1","unstructured":"Osbert Bastani Yani Ioannou Leonidas Lampropoulos Dimitrios Vytiniotis Aditya V. Nori and Antonio Criminisi. 2016. Measuring Neural Net Robustness with Constraints. In NIPS. 2613\u20132621.  Osbert Bastani Yani Ioannou Leonidas Lampropoulos Dimitrios Vytiniotis Aditya V. Nori and Antonio Criminisi. 2016. Measuring Neural Net Robustness with Constraints. In NIPS. 2613\u20132621."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1964.tb00553.x"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Brendel Wieland","year":"2018","unstructured":"Wieland Brendel , Jonas Rauber , and Matthias Bethge . 2018 . Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models . In Proceedings of International Conference on Learning Representations. Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2018. Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Buckman Jacob","year":"2018","unstructured":"Jacob Buckman , Aurko Roy , Colin Raffel , and Ian Goodfellow . 2018 . Thermometer encoding: One hot way to resist adversarial examples . In Proceedings of International Conference on Learning Representations. Jacob Buckman, Aurko Roy, Colin Raffel, and Ian Goodfellow. 2018. Thermometer encoding: One hot way to resist adversarial examples. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_8_1","volume-title":"\u201cefficient defenses against adversarial attacks\" are not robust to adversarial examples. CoRR, abs\/1711.08478","author":"Carlini Nicholas","year":"2017","unstructured":"Nicholas Carlini and David Wagner . 2017. Magnet and \u201cefficient defenses against adversarial attacks\" are not robust to adversarial examples. CoRR, abs\/1711.08478 ( 2017 ). Nicholas Carlini and David Wagner. 2017. Magnet and \u201cefficient defenses against adversarial attacks\" are not robust to adversarial examples. CoRR, abs\/1711.08478 (2017)."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140444"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_1_11_1","volume-title":"Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems. CoRR, abs\/1911.01840","author":"Chen Guangke","year":"2019","unstructured":"Guangke Chen , Sen Chen , Lingling Fan , Xiaoning Du , Zhe Zhao , Fu Song , and Yang Liu . 2019. Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems. CoRR, abs\/1911.01840 ( 2019 ). Guangke Chen, Sen Chen, Lingling Fan, Xiaoning Du, Zhe Zhao, Fu Song, and Yang Liu. 2019. Who is Real Bob? Adversarial Attacks on Speaker Recognition Systems. CoRR, abs\/1911.01840 (2019)."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11302"},{"key":"e_1_3_2_1_13_1","volume-title":"Hendrik Paul Lopuha\u00e4, and Ludolf Erwin Meester","author":"Dekking Frederik Michel","year":"2005","unstructured":"Frederik Michel Dekking , Cornelis Kraaikamp , Hendrik Paul Lopuha\u00e4, and Ludolf Erwin Meester . 2005 . A Modern Introduction to Probability and Statistics: Understanding why and how. Springer Science & Business Media . Frederik Michel Dekking, Cornelis Kraaikamp, Hendrik Paul Lopuha\u00e4, and Ludolf Erwin Meester. 2005. A Modern Introduction to Probability and Statistics: Understanding why and how. Springer Science & Business Media."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICECCS51672.2020.00016"},{"key":"e_1_3_2_1_15_1","volume-title":"Things You May Not Know About Adversarial Example: A Black-box Adversarial Image Attack. CoRR, abs\/1905.07672","author":"Duan Yuchao","year":"2019","unstructured":"Yuchao Duan , Zhe Zhao , Lei Bu , and Fu Song . 2019. Things You May Not Know About Adversarial Example: A Black-box Adversarial Image Attack. CoRR, abs\/1905.07672 ( 2019 ). Yuchao Duan, Zhe Zhao, Lei Bu, and Fu Song. 2019. Things You May Not Know About Adversarial Example: A Black-box Adversarial Image Attack. CoRR, abs\/1905.07672 (2019)."},{"key":"e_1_3_2_1_16_1","volume-title":"A Dual Approach to Scalable Verification of Deep Networks. CoRR, abs\/1803.06567","author":"Dvijotham Krishnamurthy","year":"2018","unstructured":"Krishnamurthy Dvijotham , Robert Stanforth , Sven Gowal , Timothy A. Mann , and Pushmeet Kohli . 2018. A Dual Approach to Scalable Verification of Deep Networks. CoRR, abs\/1803.06567 ( 2018 ). Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy A. Mann, and Pushmeet Kohli. 2018. A Dual Approach to Scalable Verification of Deep Networks. CoRR, abs\/1803.06567 (2018)."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-68167-2_19"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-53288-8_3"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00175"},{"key":"e_1_3_2_1_20_1","unstructured":"Reuben Feinman Ryan R Curtin Saurabh Shintre and Andrew B Gardner. 2017. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410.  Reuben Feinman Ryan R Curtin Saurabh Shintre and Andrew B Gardner. 2017. Detecting adversarial samples from artifacts. arXiv preprint arXiv:1703.00410."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00058"},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Goodfellow Ian","year":"2015","unstructured":"Ian Goodfellow , Jonathon Shlens , and Christian Szegedy . 2015 . Explaining and harnessing adversarial examples . In Proceedings of International Conference on Learning Representations. Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01090-4_1"},{"key":"e_1_3_2_1_24_1","unstructured":"Chuan Guo Mayank Rana Moustapha Cisse and Laurens Van Der Maaten. 2017. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117.  Chuan Guo Mayank Rana Moustapha Cisse and Laurens Van Der Maaten. 2017. Countering adversarial images using input transformations. arXiv preprint arXiv:1711.00117."},{"key":"e_1_3_2_1_25_1","volume-title":"Proceedings of the 11th USENIX Workshop on Offensive Technologies.","author":"He Warren","year":"2017","unstructured":"Warren He , James Wei , Xinyun Chen , Nicholas Carlini , and Dawn Song . 2017 . Adversarial Example Defense: Ensembles of Weak Defenses are not Strong . In Proceedings of the 11th USENIX Workshop on Offensive Technologies. Warren He, James Wei, Xinyun Chen, Nicholas Carlini, and Dawn Song. 2017. Adversarial Example Defense: Ensembles of Weak Defenses are not Strong. In Proceedings of the 11th USENIX Workshop on Offensive Technologies."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63387-9_1"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning. 2142\u20132151","author":"Ilyas Andrew","year":"2018","unstructured":"Andrew Ilyas , Logan Engstrom , Anish Athalye , and Jessy Lin . 2018 . Black-box Adversarial Attacks with Limited Queries and Information . In Proceedings of the 35th International Conference on Machine Learning. 2142\u20132151 . Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box Adversarial Attacks with Limited Queries and Information. In Proceedings of the 35th International Conference on Machine Learning. 2142\u20132151."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.3301962"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_1_31_1","unstructured":"Alex Krizhevsky.. 2009. Learning multiple layers of features from tiny images.  Alex Krizhevsky.. 2009. Learning multiple layers of features from tiny images."},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Kurakin Alexey","year":"2017","unstructured":"Alexey Kurakin , Ian Goodfellow , and Samy Bengio . 2017 . Adversarial examples in the physical world . In Proceedings of International Conference on Learning Representations. Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_33_1","unstructured":"Madry Lab. 2020. MNIST and CIFAR10 Adversarial Examples Challenges. https:\/\/github.com\/MadryLab  Madry Lab. 2020. MNIST and CIFAR10 Adversarial Examples Challenges. https:\/\/github.com\/MadryLab"},{"key":"e_1_3_2_1_34_1","volume-title":"Marx","author":"Larsen Richard J.","year":"2011","unstructured":"Richard J. Larsen and Morris L . Marx . 2011 . An Introduction to Mathematical Statistics and Its Applications. Prentice Hall . Richard J. Larsen and Morris L. Marx. 2011. An Introduction to Mathematical Statistics and Its Applications. Prentice Hall."},{"key":"e_1_3_2_1_35_1","unstructured":"Yann LeCun Corinna Cortes and Christopher JC Burges. 1998. The mnist database of handwritten digits.  Yann LeCun Corinna Cortes and Christopher JC Burges. 1998. The mnist database of handwritten digits."},{"key":"e_1_3_2_1_36_1","unstructured":"Kimin Lee Kibok Lee Honglak Lee and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems. 7167\u20137177.  Kimin Lee Kibok Lee Honglak Lee and Jinwoo Shin. 2018. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems. 7167\u20137177."},{"key":"e_1_3_2_1_37_1","volume-title":"Advanced Evasion Attacks and Mitigations on Practical ML-Based Phishing Website Classifiers. CoRR, abs\/2004.06954","author":"Lei Yusi","year":"2020","unstructured":"Yusi Lei , Sen Chen , Lingling Fan , Fu Song , and Yang Liu . 2020. Advanced Evasion Attacks and Mitigations on Practical ML-Based Phishing Website Classifiers. CoRR, abs\/2004.06954 ( 2020 ). Yusi Lei, Sen Chen, Lingling Fan, Fu Song, and Yang Liu. 2020. Advanced Evasion Attacks and Mitigations on Practical ML-Based Phishing Website Classifiers. CoRR, abs\/2004.06954 (2020)."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-020-0546-7"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.56"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8668044"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00021"},{"key":"e_1_3_2_1_43_1","volume-title":"NIC: Detecting Adversarial Samples with Neural Network Invariant Checking. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019","author":"Ma Shiqing","year":"2019","unstructured":"Shiqing Ma , Yingqi Liu , Guanhong Tao , Wen-Chuan Lee , and Xiangyu Zhang . 2019 . NIC: Detecting Adversarial Samples with Neural Network Invariant Checking. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019 , San Diego, California, USA , February 24-27, 2019. Shiqing Ma, Yingqi Liu, Guanhong Tao, Wen-Chuan Lee, and Xiangyu Zhang. 2019. NIC: Detecting Adversarial Samples with Neural Network Invariant Checking. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February 24-27, 2019."},{"key":"e_1_3_2_1_44_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Ma Xingjun","year":"2018","unstructured":"Xingjun Ma , Bo Li , Yisen Wang , Sarah M. Erfani , Sudanthi N. R. Wijewickrema , Grant Schoenebeck , Dawn Song , Michael E. Houle , and James Bailey . 2018 . Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality . In Proceedings of International Conference on Learning Representations. Xingjun Ma, Bo Li, Yisen Wang, Sarah M. Erfani, Sudanthi N. R. Wijewickrema, Grant Schoenebeck, Dawn Song, Michael E. Houle, and James Bailey. 2018. Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_45_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry , Aleksandar Makelov , Ludwig Schmidt , Dimitris Tsipras , and Adrian Vladu . 2018 . Towards Deep Learning Models Resistant to Adversarial Attacks . In Proceedings of International Conference on Learning Representations. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134057"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.282"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.17"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.172"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"e_1_3_2_1_51_1","volume-title":"Goodfellow","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot , Patrick D. McDaniel , and Ian J . Goodfellow . 2016 . Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. CoRR , abs\/1605.07277 (2016). Nicolas Papernot, Patrick D. McDaniel, and Ian J. Goodfellow. 2016. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. CoRR, abs\/1605.07277 (2016)."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2016.36"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.41"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14295-6_24"},{"key":"e_1_3_2_1_56_1","volume-title":"Foolbox: A Python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131.","author":"Rauber Jonas","year":"2017","unstructured":"Jonas Rauber , Wieland Brendel , and Matthias Bethge . 2017 . Foolbox: A Python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131. Jonas Rauber, Wieland Brendel, and Matthias Bethge. 2017. Foolbox: A Python toolbox to benchmark the robustness of machine learning models. arXiv preprint arXiv:1707.04131."},{"key":"e_1_3_2_1_57_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning. 5498\u20135507","author":"Roth Kevin","year":"2019","unstructured":"Kevin Roth , Yannic Kilcher , and Thomas Hofmann . 2019 . The Odds are Odd: A Statistical Test for Detecting Adversarial Examples . In Proceedings of the 36th International Conference on Machine Learning. 5498\u20135507 . Kevin Roth, Yannic Kilcher, and Thomas Hofmann. 2019. The Odds are Odd: A Statistical Test for Detecting Adversarial Examples. In Proceedings of the 36th International Conference on Machine Learning. 5498\u20135507."},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/368"},{"key":"e_1_3_2_1_59_1","volume-title":"Proceedings of the 32nd Annual Conference on Neural Information Processing Systems. 3353\u20133364","author":"Shafahi Ali","year":"2019","unstructured":"Ali Shafahi , Mahyar Najibi , Amin Ghiasi , Zheng Xu , John P. Dickerson , Christoph Studer , Larry S. Davis , Gavin Taylor , and Tom Goldstein . 2019 . Adversarial training for free! . In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems. 3353\u20133364 . Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John P. Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. 2019. Adversarial training for free!. In Proceedings of the 32nd Annual Conference on Neural Information Processing Systems. 3353\u20133364."},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3417231"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-bioeng-071516-044442"},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C.2018.00032"},{"volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems. 10825\u201310836","author":"Singh Gagandeep","key":"e_1_3_2_1_63_1","unstructured":"Gagandeep Singh , Timon Gehr , Matthew Mirman , Markus P\u00fcschel , and Martin T. Vechev . 2018. Fast and Effective Robustness Certification . In Proceedings of the Annual Conference on Neural Information Processing Systems. 10825\u201310836 . Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus P\u00fcschel, and Martin T. Vechev. 2018. Fast and Effective Robustness Certification. In Proceedings of the Annual Conference on Neural Information Processing Systems. 10825\u201310836."},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290354"},{"key":"e_1_3_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243813"},{"key":"e_1_3_2_1_66_1","volume-title":"Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv:1710.10766.","author":"Song Yang","year":"2017","unstructured":"Yang Song , Taesup Kim , Sebastian Nowozin , Stefano Ermon , and Nate Kushman . 2017 . Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv:1710.10766. Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. 2017. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv:1710.10766."},{"key":"e_1_3_2_1_67_1","volume-title":"Testing Deep Neural Networks. CoRR, abs\/1803.04792","author":"Sun Youcheng","year":"2018","unstructured":"Youcheng Sun , Xiaowei Huang , and Daniel Kroening . 2018. Testing Deep Neural Networks. CoRR, abs\/1803.04792 ( 2018 ). Youcheng Sun, Xiaowei Huang, and Daniel Kroening. 2018. Testing Deep Neural Networks. CoRR, abs\/1803.04792 (2018)."},{"key":"e_1_3_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238172"},{"key":"e_1_3_2_1_69_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Szegedy Christian","year":"2014","unstructured":"Christian Szegedy , Wojciech Zaremba , Ilya Sutskever , Joan Bruna , Dumitru Erhan , Ian Goodfellow , and Rob Fergus . 2014 . Intriguing Properties of Neural Networks . In Proceedings of International Conference on Learning Representations. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing Properties of Neural Networks. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_70_1","unstructured":"Yongqiang Tian Shiqing Ma Ming Wen Yepang Liu Shing-Chi Cheung and Xiangyu Zhang. 2019. Testing Deep Learning Models for Image Analysis Using Object-Relevant Metamorphic Relations. arXiv preprint arXiv:1909.03824.  Yongqiang Tian Shiqing Ma Ming Wen Yepang Liu Shing-Chi Cheung and Xiangyu Zhang. 2019. Testing Deep Learning Models for Image Analysis Using Object-Relevant Metamorphic Relations. arXiv preprint arXiv:1909.03824."},{"key":"e_1_3_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_1_72_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Tjeng Vincent","year":"2019","unstructured":"Vincent Tjeng , Kai Xiao , and Russ Tedrake . 2019 . Evaluating Robustness Of Neural Networks With Mixed Integer Programming . In Proceedings of International Conference on Learning Representations. Vincent Tjeng, Kai Xiao, and Russ Tedrake. 2019. Evaluating Robustness Of Neural Networks With Mixed Integer Programming. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_73_1","volume-title":"On Adaptive Attacks to Adversarial Example Defenses. CoRR, abs\/2002.08347","author":"Tram\u00e8r Florian","year":"2020","unstructured":"Florian Tram\u00e8r , Nicholas Carlini , Wieland Brendel , and Aleksander Madry . 2020. On Adaptive Attacks to Adversarial Example Defenses. CoRR, abs\/2002.08347 ( 2020 ). Florian Tram\u00e8r, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. 2020. On Adaptive Attacks to Adversarial Example Defenses. CoRR, abs\/2002.08347 (2020)."},{"key":"e_1_3_2_1_74_1","volume-title":"Accelerating Robustness Verification of Deep Neural Networks Guided by Target Labels. CoRR, abs\/2007.08520","author":"Wan Wenjie","year":"2020","unstructured":"Wenjie Wan , Zhaodi Zhang , Yiwei Zhu , Min Zhang , and Fu Song . 2020. Accelerating Robustness Verification of Deep Neural Networks Guided by Target Labels. CoRR, abs\/2007.08520 ( 2020 ). Wenjie Wan, Zhaodi Zhang, Yiwei Zhu, Min Zhang, and Fu Song. 2020. Accelerating Robustness Verification of Deep Neural Networks Guided by Target Labels. CoRR, abs\/2007.08520 (2020)."},{"key":"e_1_3_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380379"},{"key":"e_1_3_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00126"},{"key":"e_1_3_2_1_77_1","volume-title":"Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS). 6367\u20136377","author":"Wang Shiqi","year":"2018","unstructured":"Shiqi Wang , Kexin Pei , Justin Whitehouse , Junfeng Yang , and Suman Jana . 2018 . Efficient formal safety analysis of neural networks . In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS). 6367\u20136377 . Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Efficient formal safety analysis of neural networks. In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS). 6367\u20136377."},{"key":"e_1_3_2_1_78_1","volume-title":"Proceedings of the 27th USENIX Security Symposium on Security. 1599\u20131614","author":"Wang Shiqi","year":"2018","unstructured":"Shiqi Wang , Kexin Pei , Justin Whitehouse , Junfeng Yang , and Suman Jana . 2018 . Formal Security Analysis of Neural Networks using Symbolic Intervals . In Proceedings of the 27th USENIX Security Symposium on Security. 1599\u20131614 . Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Formal Security Analysis of Neural Networks using Symbolic Intervals. In Proceedings of the 27th USENIX Security Symposium on Security. 1599\u20131614."},{"key":"e_1_3_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409761"},{"volume-title":"Proceedings of the 35th International Conference on Machine Learning. 5273\u20135282","author":"Weng Tsui-Wei","key":"e_1_3_2_1_80_1","unstructured":"Tsui-Wei Weng , Huan Zhang , Hongge Chen , Zhao Song , Cho-Jui Hsieh , Luca Daniel , Duane S. Boning , and Inderjit S. Dhillon . 2018. Towards Fast Computation of Certified Robustness for ReLU Networks . In Proceedings of the 35th International Conference on Machine Learning. 5273\u20135282 . Tsui-Wei Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsieh, Luca Daniel, Duane S. Boning, and Inderjit S. Dhillon. 2018. Towards Fast Computation of Certified Robustness for ReLU Networks. In Proceedings of the 35th International Conference on Machine Learning. 5273\u20135282."},{"key":"e_1_3_2_1_81_1","volume-title":"Proceedings of International Conference on Learning Representations.","author":"Weng Tsui-Wei","year":"2018","unstructured":"Tsui-Wei Weng , Huan Zhang , Pin-Yu Chen , Jinfeng Yi , Dong Su , Yupeng Gao , Cho-Jui Hsieh , and Luca Daniel . 2018 . Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach . In Proceedings of International Conference on Learning Representations. Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. 2018. Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-89960-2_22"},{"key":"e_1_3_2_1_83_1","unstructured":"Cihang Xie Jianyu Wang Zhishuai Zhang Zhou Ren and Alan Yuille. 2017. Mitigating adversarial effects through randomization. arXiv preprint arXiv:1711.01991.  Cihang Xie Jianyu Wang Zhishuai Zhang Zhou Ren and Alan Yuille. 2017. Mitigating adversarial effects through randomization. arXiv preprint arXiv:1711.01991."},{"key":"e_1_3_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23198"},{"key":"e_1_3_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409671"},{"key":"e_1_3_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409720"},{"key":"e_1_3_2_1_88_1","volume-title":"BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks. CoRR, abs\/2103.07224","author":"Zhang Yedi","year":"2021","unstructured":"Yedi Zhang , Zhe Zhao , Guangke Chen , Fu Song , and Taolue Chen . 2021. BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks. CoRR, abs\/2103.07224 ( 2021 ). Yedi Zhang, Zhe Zhao, Guangke Chen, Fu Song, and Taolue Chen. 2021. BDD4BNN: A BDD-based Quantitative Analysis Framework for Binarized Neural Networks. CoRR, abs\/2103.07224 (2021)."},{"key":"e_1_3_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380422"}],"event":{"name":"ISSTA '21: 30th ACM SIGSOFT International Symposium on Software Testing and Analysis","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"],"location":"Virtual Denmark","acronym":"ISSTA '21"},"container-title":["Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460319.3464822","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460319.3464822","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:31Z","timestamp":1750193311000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460319.3464822"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,11]]},"references-count":89,"alternative-id":["10.1145\/3460319.3464822","10.1145\/3460319"],"URL":"https:\/\/doi.org\/10.1145\/3460319.3464822","relation":{},"subject":[],"published":{"date-parts":[[2021,7,11]]},"assertion":[{"value":"2021-07-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}