{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T14:47:18Z","timestamp":1786114038046,"version":"3.56.0"},"publisher-location":"New York, NY, USA","reference-count":89,"publisher":"ACM","license":[{"start":{"date-parts":[[2017,10,14]],"date-time":"2017-10-14T00:00:00Z","timestamp":1507939200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2017,10,14]]},"DOI":"10.1145\/3132747.3132785","type":"proceedings-article","created":{"date-parts":[[2017,10,12]],"date-time":"2017-10-12T12:51:09Z","timestamp":1507812669000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":993,"title":["DeepXplore"],"prefix":"10.1145","author":[{"given":"Kexin","family":"Pei","sequence":"first","affiliation":[{"name":"Columbia University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yinzhi","family":"Cao","sequence":"additional","affiliation":[{"name":"Lehigh University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junfeng","family":"Yang","sequence":"additional","affiliation":[{"name":"Columbia University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suman","family":"Jana","sequence":"additional","affiliation":[{"name":"Columbia University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,10,14]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"2010. ImageNet crowdsourcing benchmarking & other cool things. http\/\/:www.image-net.org\/papers\/ImageNet2010.pdf. (2010).  2010. ImageNet crowdsourcing benchmarking & other cool things. http\/\/:www.image-net.org\/papers\/ImageNet2010.pdf. (2010)."},{"key":"e_1_3_2_2_2_1","unstructured":"2016. Google auto Waymo disengagement report for autonomous driving. https:\/\/www.dmv.ca.gov\/portal\/wcm\/connect\/946b3502-c959-4e3b-b119-91319c27788f\/GoogleAutoWaymodisengagereport2016.pdf?MOD=AJPERES. (2016).  2016. Google auto Waymo disengagement report for autonomous driving. https:\/\/www.dmv.ca.gov\/portal\/wcm\/connect\/946b3502-c959-4e3b-b119-91319c27788f\/GoogleAutoWaymodisengagereport2016.pdf?MOD=AJPERES. (2016)."},{"key":"e_1_3_2_2_3_1","unstructured":"2016. Report on autonomous mode disengagements for waymo self-driving vehicles in california. https:\/\/www.dmv.ca.gov\/portal\/wcm\/connect\/946b3502-c959-4e3b-b119-91319c27788f\/GoogleAutoWaymodisengagereport2016.pdf?MOD=AJPERES. (2016).  2016. Report on autonomous mode disengagements for waymo self-driving vehicles in california. https:\/\/www.dmv.ca.gov\/portal\/wcm\/connect\/946b3502-c959-4e3b-b119-91319c27788f\/GoogleAutoWaymodisengagereport2016.pdf?MOD=AJPERES. (2016)."},{"key":"e_1_3_2_2_4_1","unstructured":"2017. Inside Waymo's secret world for training self-driving cars. https:\/\/www.theatlantic.com\/technology\/archive\/2017\/08\/inside-waymos-secret-testing-and-simulation-facilities\/537648\/. (2017).  2017. Inside Waymo's secret world for training self-driving cars. https:\/\/www.theatlantic.com\/technology\/archive\/2017\/08\/inside-waymos-secret-testing-and-simulation-facilities\/537648\/. (2017)."},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation.","author":"Abadi Mart\u00edn","year":"2016","unstructured":"Mart\u00edn Abadi , Paul Barham , Jianmin Chen , Zhifeng Chen , Andy Davis , Jeffrey Dean , Matthieu Devin , Sanjay Ghemawat , Geoffrey Irving , Michael Isard , 2016 . TensorFlow: A system for large-scale machine learning . In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation. Mart\u00edn Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation."},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978383"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2014.23247"},{"key":"e_1_3_2_2_8_1","unstructured":"autopilot:dave 2016. Nvidia-Autopilot-Keras. https:\/\/github.com\/0bserver07\/Nvidia-Autopilot-Keras. (2016).  autopilot:dave 2016. Nvidia-Autopilot-Keras. https:\/\/github.com\/0bserver07\/Nvidia-Autopilot-Keras. (2016)."},{"key":"e_1_3_2_2_9_1","volume-title":"Proceedings of the 29th Advances in Neural Information Processing Systems.","author":"Bastani Osbert","year":"2016","unstructured":"Osbert Bastani , Yani Ioannou , Leonidas Lampropoulos , Dimitrios Vytiniotis , Aditya Nori , and Antonio Criminisi . 2016 . Measuring neural net robustness with constraints . In Proceedings of the 29th Advances in Neural Information Processing Systems. Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. 2016. Measuring neural net robustness with constraints. In Proceedings of the 29th Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_2_10_1","volume-title":"Davide Del Testa","author":"Bojarski Mariusz","year":"2016","unstructured":"Mariusz Bojarski , Davide Del Testa , Daniel Dworakowski, Bernhard Firner , Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016 . End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016). Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, et al. 2016. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016)."},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.15"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2015.35"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908095"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786835"},{"key":"e_1_3_2_2_16_1","unstructured":"Fran\u00e7ois Chollet. 2015. Keras. https:\/\/github.com\/fchollet\/keras. (2015).  Fran\u00e7ois Chollet. 2015. Keras. https:\/\/github.com\/fchollet\/keras. (2015)."},{"key":"e_1_3_2_2_17_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning.","author":"Cisse Moustapha","year":"2017","unstructured":"Moustapha Cisse , Piotr Bojanowski , Edouard Grave , Yann Dauphin , and Nicolas Usunier . 2017 . Parseval networks: Improving robustness to adversarial examples . In Proceedings of the 34th International Conference on Machine Learning. Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier. 2017. Parseval networks: Improving robustness to adversarial examples. In Proceedings of the 34th International Conference on Machine Learning."},{"key":"e_1_3_2_2_18_1","unstructured":"clone:dave 2016. Behavioral cloning: end-to-end learning for self-driving cars. https:\/\/github.com\/navoshta\/behavioral-cloning. (2016).  clone:dave 2016. Behavioral cloning: end-to-end learning for self-driving cars. https:\/\/github.com\/navoshta\/behavioral-cloning. (2016)."},{"key":"e_1_3_2_2_19_1","unstructured":"contagio 2010. Contagio PDF malware dump. http:\/\/contagiodump.blogspot.de\/2010\/08\/malicious-documents-archive-for.html. (2010).  contagio 2010. Contagio PDF malware dump. http:\/\/contagiodump.blogspot.de\/2010\/08\/malicious-documents-archive-for.html. (2010)."},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2810103.2813677"},{"key":"e_1_3_2_2_22_1","volume-title":"Proceedings of the 23rd USENIX Security Symposium (USENIX Security 14)","author":"Fredrikson Matthew","year":"2014","unstructured":"Matthew Fredrikson , Eric Lantz , Somesh Jha , Simon Lin , David Page , and Thomas Ristenpart . 2014 . Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing . In Proceedings of the 23rd USENIX Security Symposium (USENIX Security 14) . Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. 2014. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In Proceedings of the 23rd USENIX Security Symposium (USENIX Security 14)."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/646943.712093"},{"key":"e_1_3_2_2_24_1","volume-title":"A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576","author":"Gatys Leon A","year":"2015","unstructured":"Leon A Gatys , Alexander S Ecker , and Matthias Bethge . 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 ( 2015 ). Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)."},{"key":"e_1_3_2_2_25_1","unstructured":"Ian Goodfellow and Nicolas Papernot. 2017. The challenge of verification and testing of machine learning. http\/\/:www.cleverhans.io\/security\/privacy\/ml\/2017\/06\/14\/verification.html. (2017).  Ian Goodfellow and Nicolas Papernot. 2017. The challenge of verification and testing of machine learning. http\/\/:www.cleverhans.io\/security\/privacy\/ml\/2017\/06\/14\/verification.html. (2017)."},{"key":"e_1_3_2_2_26_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations. http:\/\/arxiv.org\/abs\/1412","author":"Goodfellow Ian","year":"2015","unstructured":"Ian Goodfellow , Jonathon Shlens , and Christian Szegedy . 2015 . Explaining and Harnessing Adversarial Examples . In Proceedings of the 3rd International Conference on Learning Representations. http:\/\/arxiv.org\/abs\/1412 .6572 Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In Proceedings of the 3rd International Conference on Learning Representations. http:\/\/arxiv.org\/abs\/1412.6572"},{"key":"e_1_3_2_2_27_1","unstructured":"google-accident 2016. A Google self-driving car caused a crash for the first time. http\/\/:www.theverge.com\/2016\/2\/29\/11134344\/google-self-driving-car-crash-report. (2016).  google-accident 2016. A Google self-driving car caused a crash for the first time. http\/\/:www.theverge.com\/2016\/2\/29\/11134344\/google-self-driving-car-crash-report. (2016)."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.68"},{"key":"e_1_3_2_2_29_1","volume-title":"Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435","author":"Grosse Kathrin","year":"2016","unstructured":"Kathrin Grosse , Nicolas Papernot , Praveen Manoharan , Michael Backes , and Patrick McDaniel . 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 ( 2016 ). Kathrin Grosse, Nicolas Papernot, Praveen Manoharan, Michael Backes, and Patrick McDaniel. 2016. Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 (2016)."},{"key":"e_1_3_2_2_30_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations.","author":"Gu Shixiang","year":"2015","unstructured":"Shixiang Gu and Luca Rigazio . 2015 . Towards deep neural network architectures robust to adversarial examples . In Proceedings of the 3rd International Conference on Learning Representations. Shixiang Gu and Luca Rigazio. 2015. Towards deep neural network architectures robust to adversarial examples. In Proceedings of the 3rd International Conference on Learning Representations."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63387-9_1"},{"key":"e_1_3_2_2_33_1","volume-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 ( 2015 ). Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)."},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3079856.3080246"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778091"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1455770.1455806"},{"key":"e_1_3_2_2_37_1","volume":"201","author":"Katz Guy","unstructured":"Guy Katz , Clark Barrett , David L. Dill , Kyle Julian , and Mykel J. Kochenderfer. 201 7. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In Proceedings of the 29th International Conference On Computer Aided Verification. Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel J. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In Proceedings of the 29th International Conference On Computer Aided Verification.","journal-title":"Mykel J. Kochenderfer."},{"key":"e_1_3_2_2_39_1","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems.","author":"Krizhevsky Alex","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E. Hinton . 2012. ImageNet Classification with Deep Convolutional Neural Networks . In Proceedings of the 25th International Conference on Neural Information Processing Systems. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems."},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_2_2_41_1","unstructured":"Yann LeCun Corinna Cortes and Christopher JC Burges. 1998. The MNIST database of handwritten digits. (1998).  Yann LeCun Corinna Cortes and Christopher JC Burges. 1998. The MNIST database of handwritten digits. (1998)."},{"key":"e_1_3_2_2_42_1","volume-title":"MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2","author":"LeCun Yann","year":"2010","unstructured":"Yann LeCun , Corinna Cortes , and Christopher JC Burges . 2010. MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2 ( 2010 ). Yann LeCun, Corinna Cortes, and Christopher JC Burges. 2010. MNIST handwritten digit database. AT&T Labs {Online}. Available: http:\/\/yann.lecun.com\/exdb\/mnist 2 (2010)."},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.272"},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299155"},{"key":"e_1_3_2_2_45_1","volume-title":"Differential testing for software. Digital Technical Journal","author":"McKeeman William M","year":"1998","unstructured":"William M McKeeman . 1998. Differential testing for software. Digital Technical Journal ( 1998 ). William M McKeeman. 1998. Differential testing for software. Digital Technical Journal (1998)."},{"key":"e_1_3_2_2_46_1","volume-title":"Proceedings of the 6th International Conference on Learning Representations.","author":"Metzen Jan Hendrik","year":"2017","unstructured":"Jan Hendrik Metzen , Tim Genewein , Volker Fischer , and Bastian Bischoff . 2017 . On detecting adversarial perturbations . In Proceedings of the 6th International Conference on Learning Representations. Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff. 2017. On detecting adversarial perturbations. In Proceedings of the 6th International Conference on Learning Representations."},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104425"},{"key":"e_1_3_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"e_1_3_2_2_50_1","unstructured":"Nvidia. 2008. CUDA Programming guide. (2008).  Nvidia. 2008. CUDA Programming guide. (2008)."},{"key":"e_1_3_2_2_51_1","volume-title":"Extending defensive distillation. arXiv preprint arXiv:1705.05264","author":"Papernot Nicolas","year":"2017","unstructured":"Nicolas Papernot and Patrick McDaniel . 2017. Extending defensive distillation. arXiv preprint arXiv:1705.05264 ( 2017 ). Nicolas Papernot and Patrick McDaniel. 2017. Extending defensive distillation. arXiv preprint arXiv:1705.05264 (2017)."},{"key":"e_1_3_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2016.36"},{"key":"e_1_3_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.41"},{"key":"e_1_3_2_2_54_1","unstructured":"pdfrate 2012. PDFRate A machine learning based classifier operating on document metadata and structure. http:\/\/pdfrate.com\/. (2012).  pdfrate 2012. PDFRate A machine learning based classifier operating on document metadata and structure. http:\/\/pdfrate.com\/. (2012)."},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2006.26"},{"key":"e_1_3_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.27"},{"key":"e_1_3_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14295-6_24"},{"key":"e_1_3_2_2_58_1","volume-title":"Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444","author":"Radford Alec","year":"2017","unstructured":"Alec Radford , Rafal Jozefowicz , and Ilya Sutskever . 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 ( 2017 ). Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. 2017. Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 (2017)."},{"key":"e_1_3_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-45886-1_3"},{"key":"e_1_3_2_2_60_1","volume":"198","author":"Rumelhart David E","unstructured":"David E Rumelhart , Geoffrey E Hinton , and Ronald J Williams. 198 8. Learning representations by back-propagating errors. Cognitive modeling (1988). David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1988. Learning representations by back-propagating errors. Cognitive modeling (1988).","journal-title":"Ronald J Williams."},{"key":"e_1_3_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_2_62_1","volume-title":"Understanding adversarial training: Increasing local stability of neural nets through robust optimization. arXiv preprint arXiv:1511.05432","author":"Shaham Uri","year":"2015","unstructured":"Uri Shaham , Yutaro Yamada , and Sahand Negahban . 2015. Understanding adversarial training: Increasing local stability of neural nets through robust optimization. arXiv preprint arXiv:1511.05432 ( 2015 ). Uri Shaham, Yutaro Yamada, and Sahand Negahban. 2015. Understanding adversarial training: Increasing local stability of neural nets through robust optimization. arXiv preprint arXiv:1511.05432 (2015)."},{"key":"e_1_3_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978392"},{"key":"e_1_3_2_2_64_1","volume-title":"Mastering the game of Go with deep neural networks and tree search. Nature","author":"Silver David","year":"2016","unstructured":"David Silver , Aja Huang , Christopher J. Maddison , Arthur Guez , Laurent Sifre , George van den Driessche , Julian Schrittwieser , Ioannis Antonoglou , Veda Panneershelvam , Marc Lanctot , Sander Dieleman , Dominik Grewe , John Nham , Nal Kalchbrenner , Ilya Sutskever , Timothy Lillicrap , Madeleine Leach , Koray Kavukcuoglu , Thore Graepel , and Demis Hassabis . 2016. Mastering the game of Go with deep neural networks and tree search. Nature ( 2016 ). David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature (2016)."},{"key":"e_1_3_2_2_65_1","volume-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034","author":"Simonyan Karen","year":"2013","unstructured":"Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 ( 2013 ). Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013)."},{"key":"e_1_3_2_2_66_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_3_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.46"},{"key":"e_1_3_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2420950.2420987"},{"key":"e_1_3_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/2480362.2480701"},{"key":"e_1_3_2_2_70_1","volume-title":"Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research","author":"Srivastava Nitish","year":"2014","unstructured":"Nitish Srivastava , Geoffrey E Hinton , Alex Krizhevsky , Ilya Sutskever , and Ruslan Salakhutdinov . 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research ( 2014 ). Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research (2014)."},{"key":"e_1_3_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_2_72_1","volume-title":"Proceedings of the 2nd International Conference on Learning Representations.","author":"Szegedy Christian","year":"2014","unstructured":"Christian Szegedy , Wojciech Zaremba , Ilya Sutskever , Joan Bruna , Dumitru Erhan , Ian Goodfellow , and Rob Fergus . 2014 . Intriguing properties of neural networks . In Proceedings of the 2nd International Conference on Learning Representations. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Representations."},{"key":"e_1_3_2_2_73_1","unstructured":"tesla-accident 2016. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/electrek.co\/2016\/07\/01\/understanding-fatal-tesla-accident-autopilot-nhtsa-probe\/. (2016).  tesla-accident 2016. Understanding the fatal Tesla accident on Autopilot and the NHTSA probe. https:\/\/electrek.co\/2016\/07\/01\/understanding-fatal-tesla-accident-autopilot-nhtsa-probe\/. (2016)."},{"key":"e_1_3_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.5555\/3241094.3241142"},{"key":"e_1_3_2_2_75_1","unstructured":"udacity-challenge 2016. Using Deep Learning to Predict Steering Angles. https:\/\/github.com\/udacity\/self-driving-car. (2016).  udacity-challenge 2016. Using Deep Learning to Predict Steering Angles. https:\/\/github.com\/udacity\/self-driving-car. (2016)."},{"key":"e_1_3_2_2_76_1","unstructured":"Vladimir Naumovich Vapnik. 1998. Statistical learning theory.  Vladimir Naumovich Vapnik. 1998. Statistical learning theory."},{"key":"e_1_3_2_2_77_1","unstructured":"virustotal 2004. VirusTotal a free service that analyzes suspicious files and URLs and facilitates the quick detection of viruses worms trojans and all kinds of malware. https:\/\/www.virustotal.com\/. (2004).  virustotal 2004. VirusTotal a free service that analyzes suspicious files and URLs and facilitates the quick detection of viruses worms trojans and all kinds of malware. https:\/\/www.virustotal.com\/. (2004)."},{"key":"e_1_3_2_2_78_1","unstructured":"visualize:dave 2016. Visualizations for understanding the regressed wheel steering angle for self driving cars. https:\/\/github.com\/jacobgil\/keras-steering-angle-visualizations. (2016).  visualize:dave 2016. Visualizations for understanding the regressed wheel steering angle for self driving cars. https:\/\/github.com\/jacobgil\/keras-steering-angle-visualizations. (2016)."},{"key":"e_1_3_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2014.20"},{"key":"e_1_3_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_3_2_2_81_1","volume":"201","author":"Witten Ian H","unstructured":"Ian H Witten , Eibe Frank , Mark A Hall , and Christopher J Pal. 201 6. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.","journal-title":"Christopher J Pal."},{"key":"e_1_3_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1109\/CSF.2016.32"},{"key":"e_1_3_2_2_83_1","volume-title":"Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256","author":"Xiong Wayne","year":"2016","unstructured":"Wayne Xiong , Jasha Droppo , Xuedong Huang , Frank Seide , Mike Seltzer , Andreas Stolcke , Dong Yu , and Geoffrey Zweig . 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 ( 2016 ). Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig. 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 (2016)."},{"key":"e_1_3_2_2_84_1","volume-title":"Feature squeezing: detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155","author":"Xu Weilin","year":"2017","unstructured":"Weilin Xu , David Evans , and Yanjun Qi. 2017. Feature squeezing: detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155 ( 2017 ). Weilin Xu, David Evans, and Yanjun Qi. 2017. Feature squeezing: detecting adversarial examples in deep neural networks. arXiv preprint arXiv:1704.01155 (2017)."},{"key":"e_1_3_2_2_85_1","volume-title":"Proceedings of the 23rd Network and Distributed Systems Symposium.","author":"Xu Weilin","year":"2016","unstructured":"Weilin Xu , Yanjun Qi , and David Evans . 2016 . Automatically evading classifiers . In Proceedings of the 23rd Network and Distributed Systems Symposium. Weilin Xu, Yanjun Qi, and David Evans. 2016. Automatically evading classifiers. In Proceedings of the 23rd Network and Distributed Systems Symposium."},{"key":"e_1_3_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993316.1993532"},{"key":"e_1_3_2_2_87_1","volume-title":"2015 ICML Workshop on Deep Learning.","author":"Yosinski Jason","year":"2015","unstructured":"Jason Yosinski , Jeff Clune , Thomas Fuchs , and Hod Lipson . 2015 . Understanding neural networks through deep visualization . In 2015 ICML Workshop on Deep Learning. Jason Yosinski, Jeff Clune, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. In 2015 ICML Workshop on Deep Learning."},{"key":"e_1_3_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2631434"},{"key":"e_1_3_2_2_89_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.120"},{"key":"e_1_3_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.485"}],"event":{"name":"SOSP '17: ACM SIGOPS 26th Symposium on Operating Systems Principles","location":"Shanghai China","acronym":"SOSP '17","sponsor":["SIGOPS ACM Special Interest Group on Operating Systems","USENIX Assoc USENIX Assoc"]},"container-title":["Proceedings of the 26th Symposium on Operating Systems Principles"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132747.3132785","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3132747.3132785","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:10:57Z","timestamp":1750212657000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3132747.3132785"}},"subtitle":["Automated Whitebox Testing of Deep Learning Systems"],"short-title":[],"issued":{"date-parts":[[2017,10,14]]},"references-count":89,"alternative-id":["10.1145\/3132747.3132785","10.1145\/3132747"],"URL":"https:\/\/doi.org\/10.1145\/3132747.3132785","relation":{},"subject":[],"published":{"date-parts":[[2017,10,14]]},"assertion":[{"value":"2017-10-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}