{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,26]],"date-time":"2025-10-26T15:00:40Z","timestamp":1761490840369,"version":"3.41.0"},"reference-count":94,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2019,7,25]],"date-time":"2019-07-25T00:00:00Z","timestamp":1564012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGOPS Oper. Syst. Rev."],"published-print":{"date-parts":[[2019,7,25]]},"abstract":"<jats:p>Deep learning (DL) systems are increasingly deployed in safety- and security-critical domains including autonomous driving, robotics, and malware detection, where the correctness and predictability of a system on corner-case inputs are of great importance. Unfortunately, the common practice to validating a deep neural network (DNN) - measuring overall accuracy on a randomly selected test set - is not designed to surface corner-case errors. As recent work shows, even DNNs with state-of-the-art accuracy are easily fooled by human-imperceptible, adversarial perturbations to the inputs. Questions such as how to test corner-case behaviors more thoroughly and whether all adversarial samples have been found remain unanswered.<\/jats:p>\n          <jats:p>In the last few years, we have been working on bringing more engineering rigor into deep learning. Towards this goal, we have built five systems to test DNNs more thoroughly and verify the absence of adversarial samples for given datasets. These systems check a broad spectrum of properties (e.g., rotating an image should never change its classification) and find thousands of error-inducing samples for popular DNNs in critical domains (e.g., ImageNet, autonomous driving, and malware detection). Our DNN verifiers are also orders of magnitude (e.g., 5,000\u00d7) faster than similar tools. This article overviews our systems and discusses three open research challenges to hopefully inspire more future research towards testing and verifying DNNs.<\/jats:p>","DOI":"10.1145\/3352020.3352030","type":"journal-article","created":{"date-parts":[[2019,7,26]],"date-time":"2019-07-26T13:17:18Z","timestamp":1564147038000},"page":"59-67","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Bringing Engineering Rigor to Deep Learning"],"prefix":"10.1145","volume":"53","author":[{"given":"Kexin","family":"Pei","sequence":"first","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shiqi","family":"Wang","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuchi","family":"Tian","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin","family":"Whitehouse","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carl","family":"Vondrick","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yinzhi","family":"Cao","sequence":"additional","affiliation":[{"name":"Johns Hopkins University, Baltimore, MD, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Baishakhi","family":"Ray","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suman","family":"Jana","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Junfeng","family":"Yang","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,7,25]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2010. ImageNet crowdsourcing benchmarking & other cool things. http:\/\/www.image-net.org\/papers\/ImageNet_2010.pdf.  2010. ImageNet crowdsourcing benchmarking & other cool things. http:\/\/www.image-net.org\/papers\/ImageNet_2010.pdf."},{"key":"e_1_2_1_2_1","unstructured":"2015. NASA FAA Industry Conduct Initial Sense-and-Avoid Test. https:\/\/www.nasa.gov\/centers\/armstrong\/Features\/acas_xu_paves_ the_way.html.  2015. NASA FAA Industry Conduct Initial Sense-and-Avoid Test. https:\/\/www.nasa.gov\/centers\/armstrong\/Features\/acas_xu_paves_ the_way.html."},{"key":"e_1_2_1_3_1","unstructured":"2016. Chauffeur model. https:\/\/github.com\/udacity\/self-driving-car\/ tree\/master\/steering-models\/community-models\/chauffeur.  2016. Chauffeur model. https:\/\/github.com\/udacity\/self-driving-car\/ tree\/master\/steering-models\/community-models\/chauffeur."},{"key":"e_1_2_1_4_1","unstructured":"2016. Epoch model. https:\/\/github.com\/udacity\/self-driving-car\/tree\/ master\/steering-models\/community-models\/cg23.  2016. Epoch model. https:\/\/github.com\/udacity\/self-driving-car\/tree\/ master\/steering-models\/community-models\/cg23."},{"key":"e_1_2_1_5_1","unstructured":"2016. Report on autonomous mode disengagements for waymo self-driving vehicles in california. https:\/\/www.dmv.ca.gov\/ portal\/wcm\/connect\/946b3502-c959--4e3b-b119--91319c27788f\/ GoogleAutoWaymo_disengage_report_2016.pdf?MOD=AJPERES.  2016. Report on autonomous mode disengagements for waymo self-driving vehicles in california. https:\/\/www.dmv.ca.gov\/ portal\/wcm\/connect\/946b3502-c959--4e3b-b119--91319c27788f\/ GoogleAutoWaymo_disengage_report_2016.pdf?MOD=AJPERES."},{"key":"e_1_2_1_6_1","unstructured":"2017. Baidu Apollo Autonomous Driving Platform. https:\/\/github.com\/ ApolloAuto\/apollo.  2017. Baidu Apollo Autonomous Driving Platform. https:\/\/github.com\/ ApolloAuto\/apollo."},{"key":"e_1_2_1_7_1","unstructured":"2018. NAVAIR Plans to Install ACAS Xu on MQ- 4C Fleet. https:\/\/www.flightglobal.com\/news\/articles\/ navair-plans-to-install-acas-xu-on-mq-4c-fleet-444989\/.  2018. NAVAIR Plans to Install ACAS Xu on MQ- 4C Fleet. https:\/\/www.flightglobal.com\/news\/articles\/ navair-plans-to-install-acas-xu-on-mq-4c-fleet-444989\/."},{"key":"e_1_2_1_8_1","unstructured":"amazon {n. d.}. Amazon Rekognition deep learning-based image recognition search verify and organize millions of images. https: \/\/aws.amazon.com\/rekognition\/.  amazon {n. d.}. Amazon Rekognition deep learning-based image recognition search verify and organize millions of images. https: \/\/aws.amazon.com\/rekognition\/."},{"key":"e_1_2_1_9_1","volume-title":"DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket.. In Pro- ceedings of the 21st Annual Network and Distributed System Security Symposium.","author":"Arp Daniel","year":"2014","unstructured":"Daniel Arp , Michael Spreitzenbarth , Malte Hubner , Hugo Gascon , Konrad Rieck , and CERT Siemens . 2014 . DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket.. In Pro- ceedings of the 21st Annual Network and Distributed System Security Symposium. Daniel Arp, Michael Spreitzenbarth, Malte Hubner, Hugo Gascon, Konrad Rieck, and CERT Siemens. 2014. DREBIN: Effective and Explainable Detection of Android Malware in Your Pocket.. In Pro- ceedings of the 21st Annual Network and Distributed System Security Symposium."},{"key":"e_1_2_1_10_1","unstructured":"autopilot:dave 2016. Nvidia-Autopilot-Keras. https:\/\/github.com\/ 0bserver07\/Nvidia-Autopilot-Keras.  autopilot:dave 2016. Nvidia-Autopilot-Keras. https:\/\/github.com\/ 0bserver07\/Nvidia-Autopilot-Keras."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/565816.503274"},{"volume-title":"International con- ference on tools and algorithms for the construction and analysis of systems","author":"Biere Armin","key":"e_1_2_1_12_1","unstructured":"Armin Biere , Alessandro Cimatti , Edmund Clarke , and Yunshan Zhu . 1999. Symbolic model checking without BDDs . In International con- ference on tools and algorithms for the construction and analysis of systems . Springer , 193--207. Armin Biere, Alessandro Cimatti, Edmund Clarke, and Yunshan Zhu. 1999. Symbolic model checking without BDDs. In International con- ference on tools and algorithms for the construction and analysis of systems. Springer, 193--207."},{"key":"e_1_2_1_13_1","volume-title":"Symposium on Usable Privacy and Security (SOUPS).","author":"Bloom Cara","year":"2017","unstructured":"Cara Bloom , Joshua Tan , Javed Ramjohn , and Lujo Bauer . 2017 . Selfdriving cars and data collection: Privacy perceptions of networked autonomous vehicles . In Symposium on Usable Privacy and Security (SOUPS). Cara Bloom, Joshua Tan, Javed Ramjohn, and Lujo Bauer. 2017. Selfdriving cars and data collection: Privacy perceptions of networked autonomous vehicles. In Symposium on Usable Privacy and Security (SOUPS)."},{"key":"e_1_2_1_14_1","volume-title":"Coverage-based greybox fuzzing as markov chain","author":"B\u00f6hme Marcel","year":"2017","unstructured":"Marcel B\u00f6hme , Van-Thuan Pham , and Abhik Roychoudhury . 2017. Coverage-based greybox fuzzing as markov chain . IEEE Transactions on Software Engineering ( 2017 ). Marcel B\u00f6hme, Van-Thuan Pham, and Abhik Roychoudhury. 2017. Coverage-based greybox fuzzing as markov chain. IEEE Transactions on Software Engineering (2017)."},{"key":"e_1_2_1_15_1","volume-title":"Davide Del Testa","author":"Bojarski Mariusz","year":"2016","unstructured":"Mariusz Bojarski , Davide Del Testa , Daniel Dworakowski, Bernhard Firner , Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016 . End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016). Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)."},{"key":"e_1_2_1_16_1","first-page":"209","article-title":"KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs","volume":"8","author":"Cadar Cristian","year":"2008","unstructured":"Cristian Cadar , Daniel Dunbar , Dawson R Engler , 2008 . KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs .. In OSDI , Vol. 8. 209 -- 224 . Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. 2008. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs.. In OSDI, Vol. 8. 209--224.","journal-title":"OSDI"},{"key":"e_1_2_1_17_1","volume-title":"RIFD-CNN: Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Cheng Gong","year":"2016","unstructured":"Gong Cheng , Peicheng Zhou , and Junwei Han . 2016 . RIFD-CNN: Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Gong Cheng, Peicheng Zhou, and Junwei Han. 2016. RIFD-CNN: Rotation-Invariant and Fisher Discriminative Convolutional Neural Networks for Object Detection. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_18_1","volume-title":"Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv:1610.02357","author":"Chollet Fran\u00e7ois","year":"2016","unstructured":"Fran\u00e7ois Chollet . 2016 . Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv:1610.02357 (2016). Fran\u00e7ois Chollet. 2016. Xception: Deep Learning with Depthwise Separable Convolutions. arXiv preprint arXiv:1610.02357 (2016)."},{"key":"e_1_2_1_19_1","unstructured":"clarifai 2013. Clarifai API: Large Scale Visual Recognition. https: \/\/developer.clarifai.com\/models\/general-image-recognition-model\/ aaa03c23b3724a16a56b629203edc62c.  clarifai 2013. Clarifai API: Large Scale Visual Recognition. https: \/\/developer.clarifai.com\/models\/general-image-recognition-model\/ aaa03c23b3724a16a56b629203edc62c."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s100090050035"},{"key":"e_1_2_1_21_1","unstructured":"clone:dave 2016. Behavioral cloning: End-to-end learning for selfdriving cars. https:\/\/github.com\/navoshta\/behavioral-cloning.  clone:dave 2016. Behavioral cloning: End-to-end learning for selfdriving cars. https:\/\/github.com\/navoshta\/behavioral-cloning."},{"key":"e_1_2_1_22_1","unstructured":"clone:dave 2017. Rambo model for Udacity self-driving car challenge 2. https:\/\/github.com\/udacity\/self-driving-car\/tree\/master\/ steering-models\/community-models\/rambo.  clone:dave 2017. Rambo model for Udacity self-driving car challenge 2. https:\/\/github.com\/udacity\/self-driving-car\/tree\/master\/ steering-models\/community-models\/rambo."},{"key":"e_1_2_1_23_1","volume-title":"International conference on machine learning. 2990-- 2999","author":"Cohen Taco","year":"2016","unstructured":"Taco Cohen and Max Welling . 2016 . Group equivariant convolutional networks . In International conference on machine learning. 2990-- 2999 . Taco Cohen and Max Welling. 2016. Group equivariant convolutional networks. In International conference on machine learning. 2990-- 2999."},{"key":"e_1_2_1_24_1","unstructured":"contagio 2010. Contagio PDF malware dump. http:\/\/contagiodump. blogspot.de\/2010\/08\/malicious-documents-archive-for.html.  contagio 2010. Contagio PDF malware dump. http:\/\/contagiodump. blogspot.de\/2010\/08\/malicious-documents-archive-for.html."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/512950.512973"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:NUMA.0000049462.70970.b6"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.23919\/MIPRO.2017.7973569"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77935-5_9"},{"key":"e_1_2_1_29_1","volume-title":"Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming. arXiv preprint arXiv:1903.01287","author":"Fazlyab Mahyar","year":"2019","unstructured":"Mahyar Fazlyab , Manfred Morari , and George J Pappas . 2019. Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming. arXiv preprint arXiv:1903.01287 ( 2019 ). Mahyar Fazlyab, Manfred Morari, and George J Pappas. 2019. Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming. arXiv preprint arXiv:1903.01287 (2019)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1047659.1040315"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00058"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations. http: \/\/arxiv.org\/abs\/1412","author":"Goodfellow Ian","year":"2015","unstructured":"Ian Goodfellow , Jonathon Shlens , and Christian Szegedy . 2015 . Explaining and Harnessing Adversarial Examples . In Proceedings of the 3rd International Conference on Learning Representations. http: \/\/arxiv.org\/abs\/1412 .6572 Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In Proceedings of the 3rd International Conference on Learning Representations. http: \/\/arxiv.org\/abs\/1412.6572"},{"key":"e_1_2_1_33_1","unstructured":"google-accident 2016. A Google self-driving car caused a crash for the first time. http:\/\/www.theverge.com\/2016\/2\/29\/11134344\/ google-self-driving-car-crash-report.  google-accident 2016. A Google self-driving car caused a crash for the first time. http:\/\/www.theverge.com\/2016\/2\/29\/11134344\/ google-self-driving-car-crash-report."},{"key":"e_1_2_1_34_1","unstructured":"google-vision-api 2011. Cloud Vision API - Derive insight from images with our powerful Cloud Vision API. https:\/\/cloud.google.com\/vision\/.  google-vision-api 2011. Cloud Vision API - Derive insight from images with our powerful Cloud Vision API. https:\/\/cloud.google.com\/vision\/."},{"key":"e_1_2_1_35_1","volume-title":"Symbolic execution for deep neural networks. arXiv preprint arXiv:1807.10439","author":"Gopinath Divya","year":"2018","unstructured":"Divya Gopinath , Kaiyuan Wang , Mengshi Zhang , Corina S Pasareanu , and Sarfraz Khurshid . 2018. Symbolic execution for deep neural networks. arXiv preprint arXiv:1807.10439 ( 2018 ). Divya Gopinath, Kaiyuan Wang, Mengshi Zhang, Corina S Pasareanu, and Sarfraz Khurshid. 2018. Symbolic execution for deep neural networks. arXiv preprint arXiv:1807.10439 (2018)."},{"key":"e_1_2_1_36_1","volume-title":"Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435","author":"Grosse Kathrin","year":"2016","unstructured":"Kathrin Grosse , Nicolas Papernot , Praveen Manoharan , Michael Backes , and Patrick McDaniel . 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 ( 2016 ). Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 (2016)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/565816.503279"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_2_1_40_1","volume-title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications.","author":"Howard Andrew G","year":"2017","unstructured":"Andrew G Howard , Menglong Zhu , Bo Chen , Dmitry Kalenichenko , Weijun Wang , Tobias Weyand , Marco Andreetto , and Hartwig Adam . 2017 . Mobilenets: Efficient convolutional neural networks for mobile vision applications. (2017). Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. (2017)."},{"volume-title":"Proceed- ings of the IEEE conference on computer vision and pattern recognition","author":"Huang Gao","key":"e_1_2_1_41_1","unstructured":"Gao Huang , Zhuang Liu , Kilian Q Weinberger , and Laurens van der Maaten . 2017. Densely connected convolutional networks . In Proceed- ings of the IEEE conference on computer vision and pattern recognition , Vol. 1 . 3. Gao Huang, Zhuang Liu, Kilian Q Weinberger, and Laurens van der Maaten. 2017. Densely connected convolutional networks. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, Vol. 1. 3."},{"key":"e_1_2_1_42_1","unstructured":"ibm {n. d.}. IBM Watson Visual Recognition Service. https:\/\/www.ibm. com\/watson\/developercloud\/doc\/visual-recognition\/index.html.  ibm {n. d.}. IBM Watson Visual Recognition Service. https:\/\/www.ibm. com\/watson\/developercloud\/doc\/visual-recognition\/index.html."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778091"},{"volume-title":"Proceedings of the 29th International Conference On Computer Aided Verification.","author":"Katz Guy","key":"e_1_2_1_44_1","unstructured":"Guy Katz , Clark Barrett , David L. Dill , Kyle Julian , and Mykel J. Kochenderfer . 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks . In Proceedings of the 29th International Conference On Computer Aided Verification. Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In Proceedings of the 29th International Conference On Computer Aided Verification."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/360248.360252"},{"volume-title":"Proceedings of the 25th International Conference on Neural Informa- tion Processing Systems.","author":"Krizhevsky Alex","key":"e_1_2_1_46_1","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E. Hinton . 2012. ImageNet Classification with Deep Convolutional Neural Networks . In Proceedings of the 25th International Conference on Neural Informa- tion Processing Systems. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Informa- tion Processing Systems."},{"key":"e_1_2_1_47_1","volume-title":"MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2","author":"LeCun Yann","year":"2010","unstructured":"Yann LeCun , Corinna Cortes , and Christopher JC Burges . 2010. MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2 ( 2010 ). Yann LeCun, Corinna Cortes, and Christopher JC Burges. 2010. MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2 (2010)."},{"key":"e_1_2_1_48_1","unstructured":"Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Advances in Neural Informa- tion Processing Systems. 700--708.   Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Advances in Neural Informa- tion Processing Systems. 700--708."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_2_1_50_1","volume-title":"Chao Xie, Li Li, Yang Liu, Jianjun Zhao, et al.","author":"Ma Lei","year":"2018","unstructured":"Lei Ma , Fuyuan Zhang , Jiyuan Sun , Minhui Xue , Bo Li , Felix Juefei- Xu , Chao Xie, Li Li, Yang Liu, Jianjun Zhao, et al. 2018 . Deepmutation : Mutation testing of deep learning systems. In 2018 IEEE 29th Interna- tional Symposium on Software Reliability Engineering (ISSRE). IEEE , 100--111. Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei- Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, et al. 2018. Deepmutation: Mutation testing of deep learning systems. In 2018 IEEE 29th Interna- tional Symposium on Software Reliability Engineering (ISSRE). IEEE, 100--111."},{"key":"e_1_2_1_51_1","volume-title":"Combinatorial testing for deep learning systems. arXiv preprint arXiv:1806.07723","author":"Ma Lei","year":"2018","unstructured":"Lei Ma , Fuyuan Zhang , Minhui Xue , Bo Li , Yang Liu , Jianjun Zhao , and Yadong Wang . 2018. Combinatorial testing for deep learning systems. arXiv preprint arXiv:1806.07723 ( 2018 ). Lei Ma, Fuyuan Zhang, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, and Yadong Wang. 2018. Combinatorial testing for deep learning systems. arXiv preprint arXiv:1806.07723 (2018)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236082"},{"key":"e_1_2_1_54_1","unstructured":"microsoft {n. d.}. Microsoft Computer Vision API. https:\/\/azure. microsoft.com\/en-us\/services\/cognitive-services\/computer-vision\/.  microsoft {n. d.}. Microsoft Computer Vision API. https:\/\/azure. microsoft.com\/en-us\/services\/cognitive-services\/computer-vision\/."},{"key":"e_1_2_1_55_1","volume-title":"International Conference on Machine Learning. 3575--3583","author":"Mirman Matthew","year":"2018","unstructured":"Matthew Mirman , Timon Gehr , and Martin Vechev . 2018 . Differentiable abstract interpretation for provably robust neural networks . In International Conference on Machine Learning. 3575--3583 . Matthew Mirman, Timon Gehr, and Martin Vechev. 2018. Differentiable abstract interpretation for provably robust neural networks. In International Conference on Machine Learning. 3575--3583."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"e_1_2_1_57_1","volume-title":"Airborne Collision Avoidance System X. MIT Lincoln Laboratory","author":"Tech Notes MIT","year":"2015","unstructured":"MIT Tech Notes . 2015. Airborne Collision Avoidance System X. MIT Lincoln Laboratory ( 2015 ). MIT Tech Notes. 2015. Airborne Collision Avoidance System X. MIT Lincoln Laboratory (2015)."},{"key":"e_1_2_1_58_1","volume-title":"Tensorfuzz: Debugging neural networks with coverage-guided fuzzing. arXiv preprint arXiv:1807.10875","author":"Odena Augustus","year":"2018","unstructured":"Augustus Odena and Ian Goodfellow . 2018 . Tensorfuzz: Debugging neural networks with coverage-guided fuzzing. arXiv preprint arXiv:1807.10875 (2018). Augustus Odena and Ian Goodfellow. 2018. Tensorfuzz: Debugging neural networks with coverage-guided fuzzing. arXiv preprint arXiv:1807.10875 (2018)."},{"key":"e_1_2_1_59_1","unstructured":"OpenAI. 2018. OpenAI Five. https:\/\/blog.openai.com\/openai-five\/.  OpenAI. 2018. OpenAI Five. https:\/\/blog.openai.com\/openai-five\/."},{"key":"e_1_2_1_60_1","unstructured":"pdfrate 2012. PDFRate A machine learning based classifier operating on document metadata and structure. http:\/\/pdfrate.com\/.  pdfrate 2012. PDFRate A machine learning based classifier operating on document metadata and structure. http:\/\/pdfrate.com\/."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_2_1_62_1","volume-title":"Towards practical verification of machine learning: The case of computer vision systems. arXiv preprint arXiv:1712.01785","author":"Pei Kexin","year":"2017","unstructured":"Kexin Pei , Yinzhi Cao , Junfeng Yang , and Suman Jana . 2017. Towards practical verification of machine learning: The case of computer vision systems. arXiv preprint arXiv:1712.01785 ( 2017 ). Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017. Towards practical verification of machine learning: The case of computer vision systems. arXiv preprint arXiv:1712.01785 (2017)."},{"key":"e_1_2_1_63_1","unstructured":"Michael J. Cloud Ramon E. Moore R. Baker Kearfott. 2009. Introduc- tion to Interval Analysis. SIAM.   Michael J. Cloud Ramon E. Moore R. Baker Kearfott. 2009. Introduc- tion to Interval Analysis. SIAM."},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems -","volume":"1","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren , Kaiming He , Ross Girshick , and Jian Sun . 2015 . Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks . In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'15). MIT Press, Cambridge, MA, USA, 91--99. http:\/\/dl.acm.org\/citation.cfm? id=2969239.2969250 Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'15). MIT Press, Cambridge, MA, USA, 91--99. http:\/\/dl.acm.org\/citation.cfm? id=2969239.2969250"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/MALWARE.2015.7413680"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2010.26"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1095430.1081750"},{"key":"e_1_2_1_68_1","doi-asserted-by":"crossref","unstructured":"David Silver Julian Schrittwieser Karen Simonyan Ioannis Antonoglou Aja Huang Arthur Guez Thomas Hubert Lucas Baker Matthew Lai Adrian Bolton etal 2017. Mastering the game of go without human knowledge. Nature 550 7676 (2017) 354.  David Silver Julian Schrittwieser Karen Simonyan Ioannis Antonoglou Aja Huang Arthur Guez Thomas Hubert Lucas Baker Matthew Lai Adrian Bolton et al. 2017. Mastering the game of go without human knowledge. Nature 550 7676 (2017) 354.","DOI":"10.1038\/nature24270"},{"key":"e_1_2_1_69_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_2_1_70_1","unstructured":"Gagandeep Singh Timon Gehr Matthew Mirman Markus P\u00fcschel and Martin Vechev. 2018. Fast and effective robustness certification. In Advances in Neural Information Processing Systems. 10802--10813.   Gagandeep Singh Timon Gehr Matthew Mirman Markus P\u00fcschel and Martin Vechev. 2018. Fast and effective robustness certification. In Advances in Neural Information Processing Systems. 10802--10813."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290354"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2420950.2420987"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/2480362.2480701"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238172"},{"key":"e_1_2_1_75_1","first-page":"12","article-title":"Inception-v4, inception-resnet and the impact of residual connections on learning","volume":"4","author":"Szegedy Christian","year":"2017","unstructured":"Christian Szegedy , Sergey Ioffe , Vincent Vanhoucke , and Alexander A Alemi . 2017 . Inception-v4, inception-resnet and the impact of residual connections on learning .. In AAAI , Vol. 4. 12 . Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning.. In AAAI, Vol. 4. 12.","journal-title":"AAAI"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308"},{"key":"e_1_2_1_77_1","volume-title":"Proceedings of the 2nd International Conference on Learning Representations.","author":"Szegedy Christian","year":"2014","unstructured":"Christian Szegedy , Wojciech Zaremba , Ilya Sutskever , Joan Bruna , Dumitru Erhan , Ian Goodfellow , and Rob Fergus . 2014 . Intriguing properties of neural networks . In Proceedings of the 2nd International Conference on Learning Representations. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Representations."},{"key":"e_1_2_1_78_1","unstructured":"tesla-accident 2016. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/electrek.co\/2016\/07\/01\/ understanding-fatal-tesla-accident-autopilot-nhtsa-probe\/.  tesla-accident 2016. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/electrek.co\/2016\/07\/01\/ understanding-fatal-tesla-accident-autopilot-nhtsa-probe\/."},{"key":"e_1_2_1_79_1","unstructured":"tesla-accident-2019 2019. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/abcnews.go.com\/Politics\/ teslas-autopilot-engaged-fatal-florida-crash-ntsb\/story?id=63107290.  tesla-accident-2019 2019. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/abcnews.go.com\/Politics\/ teslas-autopilot-engaged-fatal-florida-crash-ntsb\/story?id=63107290."},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_2_1_81_1","volume-title":"Evaluating Robustness of Neural Networks with Mixed Integer Programming. In International Conference on Learning Representations. https:\/\/ openreview.net\/forum?id=HyGIdiRqtm","author":"Tjeng Vincent","year":"2019","unstructured":"Vincent Tjeng , Kai Y. Xiao , and Russ Tedrake . 2019 . Evaluating Robustness of Neural Networks with Mixed Integer Programming. In International Conference on Learning Representations. https:\/\/ openreview.net\/forum?id=HyGIdiRqtm Vincent Tjeng, Kai Y. Xiao, and Russ Tedrake. 2019. Evaluating Robustness of Neural Networks with Mixed Integer Programming. In International Conference on Learning Representations. https:\/\/ openreview.net\/forum?id=HyGIdiRqtm"},{"key":"e_1_2_1_82_1","unstructured":"virustotal 2004. VirusTotal a free service that analyzes suspicious files and URLs and facilitates the quick detection of viruses worms trojans and all kinds of malware. https:\/\/www.virustotal.com\/.  virustotal 2004. VirusTotal a free service that analyzes suspicious files and URLs and facilitates the quick detection of viruses worms trojans and all kinds of malware. https:\/\/www.virustotal.com\/."},{"key":"e_1_2_1_83_1","unstructured":"visualize:dave 2016. Visualizations for understanding the regressed wheel steering angle for self driving cars. https:\/\/github.com\/jacobgil\/ keras-steering-angle-visualizations.  visualize:dave 2016. Visualizations for understanding the regressed wheel steering angle for self driving cars. https:\/\/github.com\/jacobgil\/ keras-steering-angle-visualizations."},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.20"},{"key":"e_1_2_1_85_1","unstructured":"Shiqi Wang Kexin Pei Justin Whitehouse Junfeng Yang and Suman Jana. 2018. Efficient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems. 6367--6377.   Shiqi Wang Kexin Pei Justin Whitehouse Junfeng Yang and Suman Jana. 2018. Efficient formal safety analysis of neural networks. In Advances in Neural Information Processing Systems. 6367--6377."},{"volume-title":"27th {USENIX} Security Symposium ({USENIX} Security 18). 1599--1614.","author":"Wang Shiqi","key":"e_1_2_1_86_1","unstructured":"Shiqi Wang , Kexin Pei , Justin Whitehouse , Junfeng Yang , and Suman Jana . 2018. Formal security analysis of neural networks using symbolic intervals . In 27th {USENIX} Security Symposium ({USENIX} Security 18). 1599--1614. Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Formal security analysis of neural networks using symbolic intervals. In 27th {USENIX} Security Symposium ({USENIX} Security 18). 1599--1614."},{"key":"e_1_2_1_87_1","volume-title":"Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wang Xiaolong","year":"2017","unstructured":"Xiaolong Wang , Abhinav Shrivastava , and Abhinav Gupta . 2017 . AFast- RCNN: Hard Positive Generation via Adversary for Object Detection . In Conference on Computer Vision and Pattern Recognition (CVPR). Xiaolong Wang, Abhinav Shrivastava, and Abhinav Gupta. 2017. AFast- RCNN: Hard Positive Generation via Adversary for Object Detection. In Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_88_1","volume-title":"Jan Hendrik Metzen, and J Zico Kolter","author":"Wong Eric","year":"2018","unstructured":"Eric Wong , Frank Schmidt , Jan Hendrik Metzen, and J Zico Kolter . 2018 . Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems . 8400--8409. Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. 2018. Scaling provable adversarial defenses. In Advances in Neural Information Processing Systems. 8400--8409."},{"key":"e_1_2_1_89_1","volume-title":"Coverage-Guided Fuzzing for Deep Neural Networks. arXiv preprint arXiv:1809.01266","author":"Xie Xiaofei","year":"2018","unstructured":"Xiaofei Xie , Lei Ma , Felix Juefei-Xu , Hongxu Chen , Minhui Xue , Bo Li , Yang Liu , Jianjun Zhao , Jianxiong Yin , and Simon See . 2018. Coverage-Guided Fuzzing for Deep Neural Networks. arXiv preprint arXiv:1809.01266 ( 2018 ). Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Hongxu Chen, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, and Simon See. 2018. Coverage-Guided Fuzzing for Deep Neural Networks. arXiv preprint arXiv:1809.01266 (2018)."},{"key":"e_1_2_1_90_1","volume-title":"Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256","author":"Xiong Wayne","year":"2016","unstructured":"Wayne Xiong , Jasha Droppo , Xuedong Huang , Frank Seide , Mike Seltzer , Andreas Stolcke , Dong Yu , and Geoffrey Zweig . 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 ( 2016 ). Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig. 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 (2016)."},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2016.23115"},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2631434"},{"key":"e_1_2_1_93_1","volume-title":"Deeproad: Gan-based metamorphic autonomous driving system testing. arXiv preprint arXiv:1802.02295","author":"Zhang Mengshi","year":"2018","unstructured":"Mengshi Zhang , Yuqun Zhang , Lingming Zhang , Cong Liu , and Sarfraz Khurshid . 2018 . Deeproad: Gan-based metamorphic autonomous driving system testing. arXiv preprint arXiv:1802.02295 (2018). Mengshi Zhang, Yuqun Zhang, Lingming Zhang, Cong Liu, and Sarfraz Khurshid. 2018. Deeproad: Gan-based metamorphic autonomous driving system testing. arXiv preprint arXiv:1802.02295 (2018)."},{"key":"e_1_2_1_94_1","volume-title":"Generating Natural Adversarial Examples. In International Conference on Learning Representations (ICLR).","author":"Zhao Zhengli","year":"2018","unstructured":"Zhengli Zhao , Dheeru Dua , and Sameer Singh . 2018 . Generating Natural Adversarial Examples. In International Conference on Learning Representations (ICLR). Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating Natural Adversarial Examples. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_1_95_1","volume-title":"Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012","author":"Zoph Barret","year":"2017","unstructured":"Barret Zoph , Vijay Vasudevan , Jonathon Shlens , and Quoc V Le. 2017. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012 ( 2017 ). 67 Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. 2017. Learning transferable architectures for scalable image recognition. arXiv preprint arXiv:1707.07012 (2017). 67"}],"container-title":["ACM SIGOPS Operating Systems Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3352020.3352030","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3352020.3352030","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:26:15Z","timestamp":1750206375000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3352020.3352030"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,25]]},"references-count":94,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,7,25]]}},"alternative-id":["10.1145\/3352020.3352030"],"URL":"https:\/\/doi.org\/10.1145\/3352020.3352030","relation":{},"ISSN":["0163-5980"],"issn-type":[{"type":"print","value":"0163-5980"}],"subject":[],"published":{"date-parts":[[2019,7,25]]},"assertion":[{"value":"2019-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}