{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T06:44:24Z","timestamp":1779345864443,"version":"3.51.4"},"reference-count":144,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T00:00:00Z","timestamp":1710460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62202324 and 62322208"],"award-info":[{"award-number":["62202324 and 62322208"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>Deep learning (DL) techniques have attracted much attention in recent years and have been applied to many application scenarios. To improve the performance of DL models regarding different properties, many approaches have been proposed in the past decades, such as improving the robustness and fairness of DL models to meet the requirements for practical use. Among existing approaches, post-training is an effective method that has been widely adopted in practice due to its high efficiency and good performance. Nevertheless, its performance is still limited due to the incompleteness of training data. Additionally, existing approaches are always specifically designed for certain tasks, such as improving model robustness, which cannot be used for other purposes.<\/jats:p>\n          <jats:p>\n            In this article, we aim to fill this gap and propose an effective and general post-training framework, which can be adapted to improve the model performance from different aspects. Specifically, it incorporates a novel model transformation technique that transforms a classification model into an isomorphic regression model for fine-tuning, which can effectively overcome the problem of incomplete training data by forcing the model to strengthen the memory of crucial input features and thus improve the model performance eventually. To evaluate the performance of our framework, we have adapted it to two emerging tasks for improving DL models, i.e., robustness and fairness improvement, and conducted extensive studies by comparing it with state-of-the-art approaches. The experimental results demonstrate that our framework is indeed general, as it is effective in both tasks. Specifically, in the task of robustness improvement, our approach\n            <jats:sc>Dare<\/jats:sc>\n            has achieved the best results on 61.1% cases (vs. 11.1% cases achieved by baselines). In the task of fairness improvement, our approach\n            <jats:sc>FMT<\/jats:sc>\n            can effectively improve the fairness without sacrificing the accuracy of the models.\n          <\/jats:p>","DOI":"10.1145\/3630011","type":"journal-article","created":{"date-parts":[[2023,10,23]],"date-time":"2023-10-23T21:21:10Z","timestamp":1698096070000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["A Post-training Framework for Improving the Performance of Deep Learning Models via Model Transformation"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1983-6572","authenticated-orcid":false,"given":"Jiajun","family":"Jiang","sequence":"first","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1066-1190","authenticated-orcid":false,"given":"Junjie","family":"Yang","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1398-8903","authenticated-orcid":false,"given":"Yingyi","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6173-8170","authenticated-orcid":false,"given":"Zan","family":"Wang","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2870-7801","authenticated-orcid":false,"given":"Hanmo","family":"You","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3056-9962","authenticated-orcid":false,"given":"Junjie","family":"Chen","sequence":"additional","affiliation":[{"name":"Tianjin University, Tianjin, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,3,15]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Evan Ackerman. News IEEE Spectrum. Retrieved from: https:\/\/spectrum.ieee.org\/three-small-stickers-on-road-can-steer-tesla-autopilot-into-oncoming-lane"},{"key":"e_1_3_2_3_2","first-page":"29590","volume-title":"NeurIPS Conference","author":"Bai Yang","year":"2021","unstructured":"Yang Bai, Xin Yan, Yong Jiang, Shu-Tao Xia, and Yisen Wang. 2021. Clustering effect of adversarial robust models. In NeurIPS Conference. 29590\u201329601."},{"key":"e_1_3_2_4_2","first-page":"65","volume-title":"29th International Conference on Artificial Neural Networks","author":"Beckov\u00e1 Iveta","year":"2020","unstructured":"Iveta Beckov\u00e1, Stefan P\u00f3cos, and Igor Farkas. 2020. Computational analysis of robustness in neural network classifiers. In 29th International Conference on Artificial Neural Networks. Springer, 65\u201376."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2019.2942287"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409704"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468536"},{"key":"e_1_3_2_8_2","article-title":"Help Wanted: An Examination of Hiring Algorithms, Equity, and Bias","author":"Bogen Miranda","year":"2018","unstructured":"Miranda Bogen and Aaron Rieke. 2018. Help Wanted: An Examination of Hiring Algorithms, Equity, and Bias. Technical Report, Technical report, Upturn.","journal-title":"Technical Report"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Evgeny Burnaev Pavel Erofeev and Artem Papanov. 2015. Influence of resampling on accuracy of imbalanced classification. In 8th International Conference on Machine Vision(SPIE Proceedings Vol. 9875) Antanas Verikas Petia Radeva and Dmitry P. Nikolaev (Eds.). SPIE 987521.","DOI":"10.1117\/12.2228523"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583131.3590477"},{"key":"e_1_3_2_11_2","article-title":"On evaluating adversarial robustness","volume":"1902","author":"Carlini Nicholas","year":"2019","unstructured":"Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian J. Goodfellow, Aleksander Madry, and Alexey Kurakin. 2019. On evaluating adversarial robustness. CoRR abs\/1902.06705 (2019).","journal-title":"CoRR"},{"key":"e_1_3_2_12_2","first-page":"39","article-title":"Towards evaluating the robustness of neural networks","author":"Carlini Nicholas","year":"2017","unstructured":"Nicholas Carlini and David A. Wagner. 2017. Towards evaluating the robustness of neural networks. In IEEE Symposium on Security and Privacy. 39\u201357.","journal-title":"IEEE Symposium on Security and Privacy"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468537"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409697"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.312"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00042"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394112"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549093"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062522"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00949"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3129391"},{"key":"e_1_3_2_23_2","first-page":"13042","volume-title":"NeurIPS Conference","author":"Dong Li","year":"2019","unstructured":"Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019. Unified language model pre-training for natural language understanding and generation. In NeurIPS Conference. 13042\u201313054."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2019.00025"},{"key":"e_1_3_2_25_2","first-page":"1802","volume-title":"International Conference on Machine Learning","author":"Engstrom Logan","year":"2019","unstructured":"Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. 2019. Exploring the landscape of spatial robustness. In International Conference on Machine Learning. PMLR, 1802\u20131811."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00175"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2021.3074750"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380415"},{"key":"e_1_3_2_29_2","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow Ian J.","year":"2015","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. 3rd International Conference on Learning Representations.","journal-title":"3rd International Conference on Learning Representations"},{"key":"e_1_3_2_30_2","first-page":"313","volume-title":"30th IEEE International Symposium on Software Reliability Engineering","author":"Gopinath Divya","year":"2019","unstructured":"Divya Gopinath, Mengshi Zhang, Kaiyuan Wang, Ismet Burak Kadron, Corina S. Pasareanu, and Sarfraz Khurshid. 2019. Symbolic execution for importance analysis and adversarial generation in neural networks. In 30th IEEE International Symposium on Software Reliability Engineering. IEEE, 313\u2013322."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1925844.1926423"},{"key":"e_1_3_2_33_2","volume-title":"Int. J. Curr. Advan. Res","author":"Gulwani Sumit","year":"2016","unstructured":"Sumit Gulwani. 2016. Programming by examples: Applications, algorithms, and ambiguity resolution. Int. J. Curr. Advan. Res. 9706 (2016), 9\u201314."},{"key":"e_1_3_2_34_2","volume-title":"Asian Symposium on Programming Languages and Systems","author":"Gulwani Sumit","year":"2017","unstructured":"Sumit Gulwani and Prateek Jain. 2017. Programming by examples: PL meets ML. In Asian Symposium on Programming Languages and Systems."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1561\/2500000010"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/911909"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_38_2","first-page":"1031","volume-title":"Symposium on Applied Computing","author":"Henriksen Patrick","year":"2022","unstructured":"Patrick Henriksen, Francesco Leofante, and Alessio Lomuscio. 2022. Repairing misclassifications in neural networks using limited data. In Symposium on Applied Computing. ACM, 1031\u20131038."},{"key":"e_1_3_2_39_2","article-title":"Distilling the knowledge in a neural network","volume":"1503","author":"Hinton Geoffrey E.","year":"2015","unstructured":"Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. CoRR abs\/1503.02531 (2015).","journal-title":"CoRR"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468565"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2019.00017"},{"key":"e_1_3_2_42_2","article-title":"Learning with a strong adversary","author":"Huang Ruitong","year":"2015","unstructured":"Ruitong Huang, Bing Xu, Dale Schuurmans, and Csaba Szepesv\u00e1ri. 2015. Learning with a strong adversary. arXiv preprint arXiv:1511.03034 (2015).","journal-title":"arXiv preprint arXiv:1511.03034"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380378"},{"key":"e_1_3_2_45_2","article-title":"FAR: A general framework for attributional robustness","volume":"2010","author":"Ivankay Adam","year":"2020","unstructured":"Adam Ivankay, Ivan Girardi, Chiara Marchiori, and Pascal Frossard. 2020. FAR: A general framework for attributional robustness. CoRR abs\/2010.07393 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428292"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59410-7_40"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.197"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00033"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-018-1465-6"},{"key":"e_1_3_2_51_2","volume-title":"International Symposium on Software Testing and Analysis","author":"Jiang Jiajun","year":"2018","unstructured":"Jiajun Jiang, Yingfei Xiong, Hongyu Zhang, Qing Gao, and Xiangqun Chen. 2018. Shaping program repair space with existing patches and similar code. In International Symposium on Software Testing and Analysis."},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2020.3033746"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/DASC.2016.7778091"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01194"},{"key":"e_1_3_2_55_2","first-page":"1700","volume-title":"Conference on Empirical Methods in Natural Language Processing","author":"Kalchbrenner Nal","year":"2013","unstructured":"Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Conference on Empirical Methods in Natural Language Processing. ACL, 1700\u20131709."},{"key":"e_1_3_2_56_2","volume-title":"International Conference on Learning Representations (ICLR\u201918)","author":"Kalyan Ashwin","year":"2018","unstructured":"Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-guided deductive search for real-time program synthesis from examples. In International Conference on Learning Representations (ICLR\u201918)."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-011-0463-8"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2012.45"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387940.3391456"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00650"},{"key":"e_1_3_2_61_2","first-page":"50","volume-title":"International Workshop on Approaches and Applications of Inductive Programming","author":"Kitzelmann Emanuel","year":"2009","unstructured":"Emanuel Kitzelmann. 2009. Inductive programming: A survey of program synthesis techniques. In International Workshop on Approaches and Applications of Inductive Programming. Springer, 50\u201373."},{"key":"e_1_3_2_62_2","unstructured":"Alex Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images. Technical report University of Toronto University of Toronto Toronto ON."},{"key":"e_1_3_2_63_2","first-page":"1106","volume-title":"26th Annual Conference on Neural Information Processing Systems","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In 26th Annual Conference on Neural Information Processing Systems. 1106\u20131114."},{"key":"e_1_3_2_64_2","volume-title":"International Conference on Learning Representations","author":"Kurakin Alexey","year":"2017","unstructured":"Alexey Kurakin, Ian Goodfellow, and Samy Bengio. 2017. Adversarial machine learning at scale. In International Conference on Learning Representations."},{"key":"e_1_3_2_65_2","article-title":"Adversarial examples in the physical world","author":"Kurakin Alexey","year":"2017","unstructured":"Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial examples in the physical world. In 5th International Conference on Learning Representations.","journal-title":"5th International Conference on Learning Representations"},{"key":"e_1_3_2_66_2","first-page":"165","volume-title":"29th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Lee Seokhyun","year":"2020","unstructured":"Seokhyun Lee, Sooyoung Cha, Dain Lee, and Hakjoo Oh. 2020. Effective white-box testing of deep neural networks with adaptive neuron-selection strategy. In 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, Sarfraz Khurshid and Corina S. Pasareanu (Eds.). ACM, 165\u2013176."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST49551.2021.00034"},{"key":"e_1_3_2_68_2","first-page":"169","volume-title":"28th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Li Xia","year":"2019","unstructured":"Xia Li, Wei Li, Yuqun Zhang, and Lingming Zhang. 2019. DeepFL: Integrating multiple fault diagnosis dimensions for deep fault localization. In 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 169\u2013180."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510091"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678883"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236082"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473925"},{"key":"e_1_3_2_73_2","volume-title":"6th International Conference on Learning Representations","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_3_2_74_2","volume-title":"6th International Conference on Learning Representations","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_3_2_75_2","article-title":"Towards deep learning models resistant to adversarial attacks","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations.","journal-title":"6th International Conference on Learning Representations"},{"key":"e_1_3_2_76_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1007\/978-3-030-65474-0_13","volume-title":"27th International Symposium on Static Analysis","author":"Mangal Ravi","year":"2020","unstructured":"Ravi Mangal, Kartik Sarangmath, Aditya V. Nori, and Alessandro Orso. 2020. Probabilistic Lipschitz analysis of neural networks. In 27th International Symposium on Static Analysis(Lecture Notes in Computer Science, Vol. 12389). Springer, 274\u2013309."},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1249"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1111\/1475-3995.00375"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00078"},{"key":"e_1_3_2_80_2","volume-title":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning","author":"Netzer Yuval","year":"2011","unstructured":"Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning."},{"key":"e_1_3_2_81_2","volume-title":"An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence","author":"Osoba Osonde A.","year":"2017","unstructured":"Osonde A. Osoba and William Welser IV. 2017. An Intelligence in Our Image: The Risks of Bias and Errors in Artificial Intelligence. Rand Corporation."},{"key":"e_1_3_2_82_2","first-page":"372","volume-title":"IEEE European Symposium on Security and Privacy (EuroS&P\u201916)","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z. Berkay Celik, and Ananthram Swami. 2016. The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy (EuroS&P\u201916). IEEE, 372\u2013387."},{"key":"e_1_3_2_83_2","first-page":"582","volume-title":"IEEE Symposium on Security and Privacy (SP\u201916)","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy (SP\u201916). IEEE, 582\u2013597."},{"key":"e_1_3_2_84_2","first-page":"582","volume-title":"IEEE Symposium on Security and Privacy","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick D. McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In IEEE Symposium on Security and Privacy. IEEE Computer Society, 582\u2013597."},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416545"},{"key":"e_1_3_2_86_2","article-title":"ArchRepair: Block-level architecture-oriented repairing for deep neural networks","author":"Qi Hua","year":"2021","unstructured":"Hua Qi, Zhijie Wang, Qing Guo, Jianlang Chen, Felix Juefei-Xu, Lei Ma, and Jianjun Zhao. 2021. ArchRepair: Block-level architecture-oriented repairing for deep neural networks. arXiv preprint arXiv:2111.13330 (2021).","journal-title":"arXiv preprint arXiv:2111.13330"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-68238-5_32"},{"key":"e_1_3_2_88_2","volume-title":"International Conference on Learning Representations","author":"Romero Adriana","year":"2015","unstructured":"Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for thin deep nets. In International Conference on Learning Representations."},{"key":"e_1_3_2_89_2","first-page":"618","article-title":"Grad-CAM: Visual explanations from deep networks via gradient-based localization","author":"Selvaraju Ramprasaath R.","year":"2017","unstructured":"Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision. 618\u2013626.","journal-title":"IEEE International Conference on Computer Vision"},{"key":"e_1_3_2_90_2","volume-title":"Conference on Advances in Neural Information Processing Systems","author":"Shafahi Ali","year":"2019","unstructured":"Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. 2019. Adversarial training for free! In Conference on Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556953"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.5555\/2946645.3007060"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00036"},{"key":"e_1_3_2_94_2","volume-title":"3rd International Conference on Learning Representations","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In 3rd International Conference on Learning Representations."},{"key":"e_1_3_2_95_2","article-title":"Arachne: Search based repair of deep neural networks","author":"Sohn Jeongju","year":"2022","unstructured":"Jeongju Sohn, Sungmin Kang, and Shin Yoo. 2022. Arachne: Search based repair of deep neural networks. ACM Trans. Softw. Eng. Methodol. 32, 4 (2022), 85:1\u201385:26.","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454064"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.53637\/ZGUX2213"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510080"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2017.41"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_101_2","article-title":"Intriguing properties of neural networks","author":"Szegedy Christian","year":"2014","unstructured":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations.","journal-title":"2nd International Conference on Learning Representations"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00715"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556949"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00128"},{"key":"e_1_3_2_105_2","volume-title":"International Conference on Learning Representations","author":"Tram\u00e8r Florian","year":"2018","unstructured":"Florian Tram\u00e8r, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2018. Ensemble adversarial training: Attacks and defenses. In International Conference on Learning Representations."},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1515\/ajle-2020-0008"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-81685-8_1"},{"key":"e_1_3_2_108_2","first-page":"300","article-title":"RobOT: Robustness-oriented testing for deep learning systems","author":"Wang Jingyi","year":"2021","unstructured":"Jingyi Wang, Jialuo Chen, Youcheng Sun, Xingjun Ma, Dongxia Wang, Jun Sun, and Peng Cheng. 2021. RobOT: Robustness-oriented testing for deep learning systems. In 43rd IEEE\/ACM International Conference on Software Engineering. 300\u2013311.","journal-title":"43rd IEEE\/ACM International Conference on Software Engineering"},{"key":"e_1_3_2_109_2","series-title":"36th International Conference on Machine Learning","first-page":"6586","volume":"97","author":"Wang Yisen","year":"2019","unstructured":"Yisen Wang, Xingjun Ma, James Bailey, Jinfeng Yi, Bowen Zhou, and Quanquan Gu. 2019. On the convergence and robustness of adversarial training. In 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 6586\u20136595."},{"key":"e_1_3_2_110_2","volume-title":"8th International Conference on Learning Representations","author":"Wang Yisen","year":"2020","unstructured":"Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. 2020. Improving adversarial robustness requires revisiting misclassified examples. In 8th International Conference on Learning Representations. OpenReview.net."},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00046"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070536"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180233"},{"key":"e_1_3_2_114_2","article-title":"Evaluating the robustness of neural networks: An extreme value theory approach","author":"Weng Tsui-Wei","year":"2018","unstructured":"Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel. 2018. Evaluating the robustness of neural networks: An extreme value theory approach. The 6th International Conference on Learning Representations, ICLR 2018 (2018).","journal-title":"The 6th International Conference on Learning Representations, ICLR 2018"},{"key":"e_1_3_2_115_2","first-page":"1","article-title":"Wilcoxon signed-rank test","author":"Woolson Robert F.","year":"2007","unstructured":"Robert F. Woolson. 2007. Wilcoxon signed-rank test. Wiley Encyclopedia of Clinical Trials (2007), 1\u20133.","journal-title":"Wiley Encyclopedia of Clinical Trials"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1142\/S1793962323410088"},{"key":"e_1_3_2_117_2","article-title":"Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms","author":"Xiao Han","year":"2017","unstructured":"Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. CoRRabs\/1708.07747 (2017).","journal-title":"CoRR"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.45"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23198"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00130"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3421841"},{"key":"e_1_3_2_122_2","volume-title":"International Conference on Software Engineering","author":"You Hanmo","year":"2023","unstructured":"Hanmo You, Zan Wang, Junjie Chen, Shuang Liu, and Shuochuan Li. 2023. Regression fuzzing for deep learning systems. In International Conference on Software Engineering."},{"key":"e_1_3_2_123_2","article-title":"DeepRepair: Style-guided repairing for deep neural networks in the real-world operational environment","author":"Yu Bing","year":"2021","unstructured":"Bing Yu, Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, and Jianjun Zhao. 2021. DeepRepair: Style-guided repairing for deep neural networks in the real-world operational environment. IEEE Trans. Reliab. 71, 4 (2021), 1401\u20131416.","journal-title":"IEEE Trans. Reliab."},{"key":"e_1_3_2_124_2","first-page":"39","volume-title":"10th ACM Workshop on Artificial Intelligence and Security","author":"Zantedeschi Valentina","year":"2017","unstructured":"Valentina Zantedeschi, Maria-Irina Nicolae, and Ambrish Rawat. 2017. Efficient defenses against adversarial attacks. In 10th ACM Workshop on Artificial Intelligence and Security. 39\u201349."},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1145\/605466.605468"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.988498"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278779"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00043"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00129"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2662206"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549103"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238187"},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380331"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-70096-0_38"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380362"},{"key":"e_1_3_2_137_2","first-page":"104","volume-title":"IEEE 30th International Symposium on Software Reliability Engineering (ISSRE\u201919)","author":"Zhang Tianyi","year":"2019","unstructured":"Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael Lyu, and Miryung Kim. 2019. An empirical study of common challenges in developing deep learning applications. In IEEE 30th International Symposium on Software Reliability Engineering (ISSRE\u201919). IEEE, 104\u2013115."},{"key":"e_1_3_2_138_2","first-page":"129","volume-title":"27th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Zhang Yuhao","year":"2018","unstructured":"Yuhao Zhang, Yifan Chen, Shing-Chi Cheung, Yingfei Xiong, and Lu Zhang. 2018. An empirical study on TensorFlow program bugs. In 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 129\u2013140."},{"key":"e_1_3_2_139_2","doi-asserted-by":"crossref","unstructured":"Yingyi Zhang Zan Wang Jiajun Jiang Hanmo You and Junjie Chen. 2022. Toward improving the robustness of deep learning models via model transformation. 37th IEEE\/ACM International Conference on Automated Software Engineering (ASE\u201922) ACM Rochester MI 104:1\u2013104:13.","DOI":"10.1145\/3551349.3556920"},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409676"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464822"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510123"},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-71500-7_16"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380422"},{"key":"e_1_3_2_145_2","unstructured":"Donglin Zhuang Xingyao Zhang Shuaiwen Song and Sara Hooker. 2022. Randomness in neural network training: Characterizing the impact of tooling. In Conference on Machine Learning and Systems. mlsys.org."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630011","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3630011","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:01Z","timestamp":1750291021000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3630011"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,15]]},"references-count":144,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3630011"],"URL":"https:\/\/doi.org\/10.1145\/3630011","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,15]]},"assertion":[{"value":"2023-01-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}