{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T13:56:21Z","timestamp":1777470981383,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":49,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,18]],"date-time":"2020-07-18T00:00:00Z","timestamp":1595030400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,18]]},"DOI":"10.1145\/3395363.3397357","type":"proceedings-article","created":{"date-parts":[[2020,7,13]],"date-time":"2020-07-13T21:44:18Z","timestamp":1594676658000},"page":"177-188","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":192,"title":["DeepGini: prioritizing massive tests to enhance the robustness of deep neural networks"],"prefix":"10.1145","author":[{"given":"Yang","family":"Feng","sequence":"first","affiliation":[{"name":"Nanjing University, China"}]},{"given":"Qingkai","family":"Shi","sequence":"additional","affiliation":[{"name":"Hong Kong University of Science and Technology, China"}]},{"given":"Xinyu","family":"Gao","sequence":"additional","affiliation":[{"name":"Nanjing University, China"}]},{"given":"Jun","family":"Wan","sequence":"additional","affiliation":[{"name":"Ant Financial Services, China"}]},{"given":"Chunrong","family":"Fang","sequence":"additional","affiliation":[{"name":"Nanjing University, China"}]},{"given":"Zhenyu","family":"Chen","sequence":"additional","affiliation":[{"name":"Nanjing University, China"}]}],"member":"320","published-online":{"date-parts":[[2020,7,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","volume-title":"Calculus: concepts and methods","author":"Binmore Ken","DOI":"10.1017\/CBO9780511802997"},{"key":"e_1_3_2_1_2_1","volume-title":"Davide Del Testa","author":"Bojarski Mariusz","year":"2016"},{"key":"e_1_3_2_1_3_1","unstructured":"Timothy Alan Budd. 1981. Mutation Analysis of Program Test Data. ( 1981 ).  Timothy Alan Budd. 1981. Mutation Analysis of Program Test Data. ( 1981 )."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/AITest.2019.000-6"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_1_6_1","volume-title":"Introduction to algorithms","author":"Cormen Thomas H"},{"key":"e_1_3_2_1_7_1","unstructured":"Alex Davies. [n. d.]. Tesla's Latest Autopilot Death Looks Just Like a Prior Crash. Available at https:\/\/www.wired.com\/story\/teslas-latest-autopilot-death-lookslike-prior-crash\/ ( 2020 \/01\/27). ([n. d.]).  Alex Davies. [n. d.]. Tesla's Latest Autopilot Death Looks Just Like a Prior Crash. Available at https:\/\/www.wired.com\/story\/teslas-latest-autopilot-death-lookslike-prior-crash\/ ( 2020 \/01\/27). ([n. d.])."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_1_9_1","unstructured":"John S Denker and Yann Lecun. 1991. Transforming neural-net output levels to probability distributions. In Advances in neural information processing systems. 853-859.  John S Denker and Yann Lecun. 1991. Transforming neural-net output levels to probability distributions. In Advances in neural information processing systems. 853-859."},{"key":"e_1_3_2_1_10_1","volume-title":"Verification and Validation (ICST), 2013 IEEE Sixth International Conference on. IEEE, 302-311","author":"Nardo Daniel Di","year":"2013"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of 2015 3rd International Conference on Learning Representations (ICLR).","author":"Goodfellow Ian J","year":"2015"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0164-1212(99)00037-0"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2003.1183927"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1291535.1291539"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2008.4658073"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2005.87"},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of 2017 5th International Conference on Learning Representations (ICLR).","author":"Kurakin Alexey","year":"2017"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2003.1251065"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER.2019.00031"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_3_2_1_23_1","volume-title":"DeepMutation: Mutation Testing of Deep Learning Systems. In 2018 IEEE 29th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 100-111","author":"Ma L."},{"key":"e_1_3_2_1_24_1","volume-title":"2016 IEEE European Symposium on. IEEE, 372-387","author":"Papernot Nicolas","year":"2016"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","unstructured":"J. Ross Quinlan. 1986. Induction of decision trees. Machine learning 1 1 ( 1986 ) 81-106.  J. Ross Quinlan. 1986. Induction of decision trees. Machine learning 1 1 ( 1986 ) 81-106.","DOI":"10.1007\/BF00116251"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:AMAI.0000018580.96245.c6"},{"key":"e_1_3_2_1_28_1","series-title":"SIAM review 35, 2 ( 1993 ), 183-238","volume-title":"Lagrange multipliers and optimality","author":"Rockafellar R Tyrrell"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.536955"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1999.792604"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.962562"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell system technical journal 27 3 ( 1948 ) 379-423.  Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell system technical journal 27 3 ( 1948 ) 379-423.","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2007.25"},{"key":"e_1_3_2_1_35_1","volume-title":"Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.","author":"Silver David","year":"2016"},{"key":"e_1_3_2_1_36_1","unstructured":"Jack Stewart. [n. d.]. Tesla's Autopilot Was Involved in Another Deadly Car Crash. Available at https:\/\/www.wired.com\/story\/tesla-autopilot-self-drivingcrash-california\/ ( 2020 \/01\/27). ([n. d.]).  Jack Stewart. [n. d.]. Tesla's Autopilot Was Involved in Another Deadly Car Crash. Available at https:\/\/www.wired.com\/story\/tesla-autopilot-self-drivingcrash-california\/ ( 2020 \/01\/27). ([n. d.])."},{"key":"e_1_3_2_1_37_1","volume-title":"Testing Deep Neural Networks. arXiv preprint arXiv","author":"Sun Youcheng","year":"1803"},{"key":"e_1_3_2_1_38_1","volume-title":"Concolic Testing for Deep Neural Networks. arXiv preprint arXiv","author":"Sun Youcheng","year":"1805"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2006.74"},{"key":"e_1_3_2_1_41_1","volume-title":"d.]. Kernel Smoothing","author":"Wand Matt P"},{"key":"e_1_3_2_1_42_1","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems","author":"Wicker Matthew"},{"key":"e_1_3_2_1_43_1","volume-title":"Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.","author":"Witten Ian H","year":"2016"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"W Eric Wong Joseph R Horgan Saul London and Aditya P Mathur. 1998. Efect of test set minimization on fault detection efectiveness. Software: Practice and Experience 28 4 ( 1998 ) 347-369.  W Eric Wong Joseph R Horgan Saul London and Aditya P Mathur. 1998. Efect of test set minimization on fault detection efectiveness. Software: Practice and Experience 28 4 ( 1998 ) 347-369.","DOI":"10.1002\/(SICI)1097-024X(19980410)28:4<347::AID-SPE145>3.0.CO;2-L"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Wayne Xiong Jasha Droppo Xuedong Huang Frank Seide Mike Seltzer Andreas Stolcke Dong Yu and Geofrey Zweig. 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 ( 2016 ).  Wayne Xiong Jasha Droppo Xuedong Huang Frank Seide Mike Seltzer Andreas Stolcke Dong Yu and Geofrey Zweig. 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 ( 2016 ).","DOI":"10.1109\/ICASSP.2017.7953159"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Shin Yoo and Mark Harman. 2012. Regression testing minimization selection and prioritization: a survey. Software Testing Verification and Reliability 22 2 ( 2012 ) 67-120.  Shin Yoo and Mark Harman. 2012. Regression testing minimization selection and prioritization: a survey. Software Testing Verification and Reliability 22 2 ( 2012 ) 67-120.","DOI":"10.1002\/stv.430"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572272.1572296"},{"key":"e_1_3_2_1_48_1","unstructured":"Long Zhang Xuechao Sun Yong Li and Zhenyu Zhang. 2019. A noise-sensitivityanalysis-based test prioritization technique for deep neural networks. arXiv preprint arXiv:1901. 00054 ( 2019 ).  Long Zhang Xuechao Sun Yong Li and Zhenyu Zhang. 2019. A noise-sensitivityanalysis-based test prioritization technique for deep neural networks. arXiv preprint arXiv:1901. 00054 ( 2019 )."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_1_50_1","unstructured":"Chris Ziegler. [n. d.]. A Google self-driving car caused a crash for the first time. Available at https:\/\/www.theverge.com\/ 2016 \/2\/29\/11134344\/google-self-drivingcar-crash-report ( 2020 \/01\/27). ([n. d.]).  Chris Ziegler. [n. d.]. A Google self-driving car caused a crash for the first time. Available at https:\/\/www.theverge.com\/ 2016 \/2\/29\/11134344\/google-self-drivingcar-crash-report ( 2020 \/01\/27). ([n. d.])."}],"event":{"name":"ISSTA '20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis","location":"Virtual Event USA","acronym":"ISSTA '20","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3395363.3397357","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3395363.3397357","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:38:35Z","timestamp":1750199915000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3395363.3397357"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,18]]},"references-count":49,"alternative-id":["10.1145\/3395363.3397357","10.1145\/3395363"],"URL":"https:\/\/doi.org\/10.1145\/3395363.3397357","relation":{},"subject":[],"published":{"date-parts":[[2020,7,18]]},"assertion":[{"value":"2020-07-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}