{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T08:15:18Z","timestamp":1773389718101,"version":"3.50.1"},"reference-count":84,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"DOI":"10.13039\/501100001809","name":"Joint Funds of the National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U2241216"],"award-info":[{"award-number":["U2241216"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202223"],"award-info":[{"award-number":["62202223"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"crossref","award":["BK20220881"],"award-info":[{"award-number":["BK20220881"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Open Fund of the State Key Laboratory for Novel Software Technology","award":["KFKT2024B27"],"award-info":[{"award-number":["KFKT2024B27"]}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["NT2024020"],"award-info":[{"award-number":["NT2024020"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>Deep Neural Network (DNN) testing has emerged as an effective way of uncovering erroneous behaviors in DNN models and further enhancing their performance. Research on test input generation has gained much attention from both researchers and practitioners, aiming to expose faults in models. The newly generated inputs subsequently serve as additional training instances for model refinement through retraining. Existing approaches generate test inputs by optimizing an objective function based on testing metrics such as neuron coverage and property-related metrics, and the gradient of the objective is used to perturb seed inputs. However, these approaches pay limited attention to the model\u2019s decision logic, particularly the erroneous decision patterns learned during training. Furthermore, they primarily focus on detecting faults without considering the diversity of detected misbehaviors, which limits the models\u2019 ability to learn diverse features through retraining. To address these limitations, this article introduces SUNTest, a novel test input generation approach designed to detect diverse faults and enhance the robustness of DNN models. SUNTest focuses on erroneous decision-making by localizing suspicious neurons responsible for misbehaviors through the execution spectrum analysis of neurons. To guide input mutations toward inducing diverse faults, SUNTest designs a hybrid fitness function that incorporates two types of feedback derived from neuron behaviors, including the fault-revealing capability of test inputs guided by suspicious neurons and the diversity of test inputs. Additionally, SUNTest adopts an adaptive selection strategy for mutation operators to prioritize operators likely to induce new fault types and improve the fitness value in each iteration. Experiments conducted on eight DNN models demonstrate the effectiveness of SUNTest in fault localization and test input generation. It outperforms existing test input generators in the number of detected faults, uncovering up to 80.9 more distinct fault types. In terms of model enhancement, SUNTest increases the average accuracy improvement by up to 8.04% compared to baseline approaches.<\/jats:p>","DOI":"10.1145\/3736305","type":"journal-article","created":{"date-parts":[[2025,5,19]],"date-time":"2025-05-19T09:11:41Z","timestamp":1747645901000},"page":"1-45","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["White-Box Test Input Generation for Enhancing Deep Neural Network Models through Suspicious Neuron Awareness"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5286-874X","authenticated-orcid":false,"given":"Hongjing","family":"Guo","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0698-7307","authenticated-orcid":false,"given":"Chuanqi","family":"Tao","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6843-1892","authenticated-orcid":false,"given":"Zhiqiu","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0913-1539","authenticated-orcid":false,"given":"Weiqin","family":"Zou","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,3,12]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"GitHub. n.\u2009d. The Online Artifact of this Paper. Retrieved from https:\/\/github.com\/TestingAIGroup\/SUNTest"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.5555\/1308173.1308264"},{"key":"e_1_3_2_4_2","unstructured":"Uber Accident. 2018. After Fatal Uber Crash a Self-Driving Start-Up Moves Forward. Retrieved August 1 2021 from https:\/\/www.nytimes.com\/2018\/05\/07\/technology\/uber-crash-autonomous-driveai.html"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3644388"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1316"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510099"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csda.2005.12.017"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01258-8_15"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3582573"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394112"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/IVS.2017.7995975"},{"key":"e_1_3_2_13_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT \u201919)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT \u201919). Association for Computational Linguistics, 4171\u20134186."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338954"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE52982.2021.00037"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-16722-6_10"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature21056"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510137"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397357"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510232"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510087"},{"key":"e_1_3_2_22_2","first-page":"702","volume-title":"Proceedings of the 42nd International Conference on Software Engineering (ICSE \u201920)","author":"Gerasimou Simos","year":"2020","unstructured":"Simos Gerasimou, Hasan Ferit Eniser, Alper Sen, and Alper Cakan. 2020. Importance-driven deep learning system testing. In Proceedings of the 42nd International Conference on Software Engineering (ICSE \u201920). ACM, 702\u2013713."},{"key":"e_1_3_2_23_2","unstructured":"Ian J. Goodfellow Nicolas Papernot and Patrick D. McDaniel. 2016. cleverhans v0.1: An adversarial machine learning library. arXiv:1610.00768v2. Retrieved from https:\/\/arxiv.org\/abs\/1610.00768v2"},{"key":"e_1_3_2_24_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915)","author":"Goodfellow Ian J.","year":"2015","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915). Retrieved from https:\/\/arxiv.org\/abs\/1412.6572"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE59848.2023.00051"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3264835"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409754"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_29_2","first-page":"6626","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS \u201917)","author":"Heusel Martin","year":"2017","unstructured":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS \u201917), 6626\u20136637."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511598"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2021.3080664"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST60714.2024.00030"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00054"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/1101908.1101949"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.2514\/1.G003724"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1080\/00031305.1998.10480547"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3546947"},{"key":"e_1_3_2_39_2","unstructured":"Nair Krizhevsky Hinton Vinod Christopher Geoffrey Mike Papadakis and Anthony Ventresque. 2014. The Cifar-10 Dataset. Retrieved from https:\/\/www.cs.toronto.edu\/~{}kriz\/cifar.html"},{"key":"e_1_3_2_40_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917)","author":"Kurakin Alexey","year":"2017","unstructured":"Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=HJGU3Rodl"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_2_42_2","unstructured":"Yann LeCun and Corinna Cortes. 1998. The MNIST Database of Handwritten Digits. Retrieved from http:\/\/yann.lecun.com\/exdb\/mnist\/"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397346"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER.2019.00031"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338930"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00106"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8668044"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236082"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3417330"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3047766"},{"key":"e_1_3_2_53_2","unstructured":"Yuval Netzer Tao Wang Adam Coates A. Bissacco Bo Wu and A. Ng. 2011. Reading Digits in Natural Images with Unsupervised Feature Learning. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:16852518"},{"key":"e_1_3_2_54_2","first-page":"4901","volume-title":"Proceedings of the 36th International Conference on Machine Learning (ICML \u201919)","author":"Odena Augustus","year":"2019","unstructured":"Augustus Odena, Catherine Olsson, David G. Andersen, and Ian J. Goodfellow. 2019. TensorFuzz: Debugging neural networks with coverage-guided fuzzing. In Proceedings of the 36th International Conference on Machine Learning (ICML \u201919). PMLR, 4901\u20134911."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2663435"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2016.36"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.62"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_59_2","first-page":"1","volume-title":"Annual Meeting of the Southern Association for Institutional Research","author":"Romano Jeanine","year":"2006","unstructured":"Jeanine Romano, Jeffrey D. Kromrey, Jesse Coraggio, Jeff Skowronek, and Linda Devine. 2006. Exploring methods for evaluating group differences on the NSSE and other surveys: Are the t-test and cohen\u2019sd indices the most appropriate choices. In Annual Meeting of the Southern Association for Institutional Research. Citeseer, 1\u201351."},{"key":"e_1_3_2_60_2","first-page":"2226","volume-title":"Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS \u201916)","author":"Salimans Tim","year":"2016","unstructured":"Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training GANs. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS \u201916), 2226\u20132234."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416621"},{"key":"e_1_3_2_62_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_3_2_63_2","unstructured":"Department of Motor Vehicles State of California. 2020. Autonomous Vehicle Collision Reports. Retrieved from https:\/\/www.dmv.ca.gov\/portal\/vehicle-industry-services\/autonomous-vehicles\/autonomous-vehicle-collision-reports\/"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380423"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583564"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238165"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00038"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409761"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00034"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510041"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2013.2285319"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/2522920.2522924"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3490489"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330579"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616345"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00019"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00107"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1559"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380331"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409676"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460120.3484818"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510123"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598109"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2892102"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3736305","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T05:28:29Z","timestamp":1773379709000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3736305"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,12]]},"references-count":84,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3736305"],"URL":"https:\/\/doi.org\/10.1145\/3736305","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,12]]},"assertion":[{"value":"2024-07-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}