{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T02:38:30Z","timestamp":1774579110272,"version":"3.50.1"},"reference-count":155,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T00:00:00Z","timestamp":1741132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2023YFB3107400"],"award-info":[{"award-number":["2023YFB3107400"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62006181, 62132011, 62161160337, 62206217, U20A20177, and U21B2018"],"award-info":[{"award-number":["62006181, 62132011, 62161160337, 62206217, U20A20177, and U21B2018"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Shaanxi Province Key Industry Innovation Program","award":["2021ZDLGY01-02 and 2023-ZDLGY-38"],"award-info":[{"award-number":["2021ZDLGY01-02 and 2023-ZDLGY-38"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,7,31]]},"abstract":"<jats:p>Recently, software systems powered by deep learning (DL) techniques have significantly facilitated people\u2019s lives in many aspects. As the backbone of these DL systems, various DL libraries undertake the underlying optimization and computation. However, like traditional software, DL libraries are not immune to bugs. These bugs may be propagated to programs and software developed based on DL libraries, thereby posing serious threats to users\u2019 personal property and safety. Studying the characteristics of DL libraries, their associated bugs, and the corresponding testing methods is crucial for enhancing the security of DL systems and advancing the widespread application of DL technology. This paper provides an overview of the testing research on various DL libraries, discusses the strengths and weaknesses of existing methods, and provides guidance and reference for the application of DL library testing methods. This paper first introduces the workflow of DL underlying libraries and the characteristics of three kinds of DL libraries involved, namely DL framework, DL compiler, and DL hardware library. Subsequently, this paper constructs a literature collection pipeline and comprehensively summarizes existing testing methods on these DL libraries to analyze their effectiveness and limitations. It also reports findings and the challenges of existing DL library testing in real-world applications for future research.<\/jats:p>","DOI":"10.1145\/3716497","type":"journal-article","created":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T07:15:56Z","timestamp":1738826156000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Deep Learning Library Testing: Definition, Methods and Challenges"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7010-6749","authenticated-orcid":false,"given":"Xiaoyu","family":"Zhang","sequence":"first","affiliation":[{"name":"Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0382-6401","authenticated-orcid":false,"given":"Weipeng","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6959-0569","authenticated-orcid":false,"given":"Chao","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8776-8730","authenticated-orcid":false,"given":"Qi","family":"Li","sequence":"additional","affiliation":[{"name":"Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8967-8525","authenticated-orcid":false,"given":"Qian","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Wuhan University, Wuhan, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6265-7345","authenticated-orcid":false,"given":"Chenhao","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Xi'an Jiaotong University, Xi'an, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8826-0362","authenticated-orcid":false,"given":"Xiaohong","family":"Guan","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Engineering, Xi'an Jiaotong University, Xi'an, China"}]}],"member":"320","published-online":{"date-parts":[[2025,3,5]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","unstructured":"2010. IEEE standard classification for software anomalies. IEEE Std 1044-2009 (Revision of IEEE Std 1044-1993) (2010) 1\u201323. DOI:10.1109\/IEEESTD.2010.5399061","DOI":"10.1109\/IEEESTD.2010.5399061"},{"key":"e_1_3_3_3_2","unstructured":"2024. Our repository with more experiment details. (2024). https:\/\/github.com\/shiningrain\/CSUR_DL_library_survey."},{"issue":"1","key":"e_1_3_3_4_2","first-page":"1","article-title":"Large scale performance analysis of distributed deep learning frameworks for convolutional neural networks","volume":"10","author":"Aach Marcel","year":"2023","unstructured":"Marcel Aach, Eray Inanc, Rakesh Sarma, Morris Riedel, and Andreas Lintermann. 2023. Large scale performance analysis of distributed deep learning frameworks for convolutional neural networks. Journal of Big Data 10, 1 (2023), 1\u201323.","journal-title":"Journal of Big Data"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/2951913.2976746"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3512290.3528697"},{"key":"e_1_3_3_7_2","first-page":"66","volume-title":"2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)","author":"Ben-Nun Tal","year":"2019","unstructured":"Tal Ben-Nun, Maciej Besta, Simon Huber, Alexandros Nikolaos Ziogas, Daniel Peter, and Torsten Hoefler. 2019. A modular benchmarking infrastructure for high-performance and reproducible deep learning. In 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 66\u201377."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110542"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549123"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660796"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587155"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3508362"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00172"},{"key":"e_1_3_3_14_2","first-page":"578","volume-title":"13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)","author":"Chen Tianqi","year":"2018","unstructured":"Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et\u00a0al. 2018. TVM: An automated End-to-End optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 578\u2013594."},{"key":"e_1_3_3_15_2","article-title":"Metamorphic testing: A new approach for generating next test cases","author":"Chen Tsong Y.","year":"1998","unstructured":"Tsong Y. Chen, Shing C. Cheung, and Shiu Ming Yiu. 1998. Metamorphic testing: A new approach for generating next test cases. Technical Report Hkust-cs98-01 (1998).","journal-title":"Technical Report Hkust-cs98-01"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409759"},{"key":"e_1_3_3_17_2","article-title":"cuDNN: Efficient primitives for deep learning","author":"Chetlur Sharan","year":"2014","unstructured":"Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. 2014. cuDNN: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759 (2014).","journal-title":"arXiv preprint arXiv:1410.0759"},{"key":"e_1_3_3_18_2","first-page":"2383","volume-title":"32nd USENIX Security Symposium (USENIX Security 23)","author":"Christou Neophytos","year":"2023","unstructured":"Neophytos Christou, Di Jin, Vaggelis Atlidakis, Baishakhi Ray, and Vasileios P. Kemerlis. 2023. IvySyn: Automated vulnerability discovery in deep learning frameworks. In 32nd USENIX Security Symposium (USENIX Security 23). 2383\u20132400."},{"key":"e_1_3_3_19_2","first-page":"249","volume-title":"Proceedings of the 16th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement","author":"Cui Di","year":"2022","unstructured":"Di Cui, Xingyu Li, Feiyang Liu, Siqi Wang, Jie Dai, Lu Wang, and Qingshan Li. 2022. Towards demystifying the impact of dependency structures on bug locations in deep learning libraries. In Proceedings of the 16th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement. 249\u2013260."},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598067"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3623343"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549085"},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MET.2017.2","volume-title":"2017 IEEE\/ACM 2nd International Workshop on Metamorphic Testing (MET)","author":"Ding Junhua","year":"2017","unstructured":"Junhua Ding, Xiaojun Kang, and Xin-Hua Hu. 2017. Validating a deep learning framework by metamorphic testing. In 2017 IEEE\/ACM 2nd International Workshop on Metamorphic Testing (MET). IEEE, 28\u201334."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2017.05.138"},{"key":"e_1_3_3_25_2","article-title":"An empirical study of fault triggers in deep learning frameworks","author":"Du Xiaoting","year":"2022","unstructured":"Xiaoting Du, Yulei Sui, Zhihao Liu, and Jun Ai. 2022. An empirical study of fault triggers in deep learning frameworks. IEEE Transactions on Dependable and Secure Computing (2022).","journal-title":"IEEE Transactions on Dependable and Secure Computing"},{"key":"e_1_3_3_26_2","first-page":"184","volume-title":"2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE)","author":"Du Xiaoting","year":"2021","unstructured":"Xiaoting Du, Zheng Zheng, Lei Ma, and Jianjun Zhao. 2021. An empirical study on common bugs in deep learning compilers. In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 184\u2013195."},{"key":"e_1_3_3_27_2","article-title":"Mokav: Execution-driven differential testing with LLMs","author":"Etemadi Khashayar","year":"2024","unstructured":"Khashayar Etemadi, Bardia Mohammadi, Zhendong Su, and Martin Monperrus. 2024. Mokav: Execution-driven differential testing with LLMs. arXiv preprint arXiv:2406.10375 (2024).","journal-title":"arXiv preprint arXiv:2406.10375"},{"issue":"9","key":"e_1_3_3_28_2","first-page":"0","article-title":"Just-in-time defect prediction for intelligent computing frameworks (in Chinese)","volume":"34","author":"Ge Jian","year":"2023","unstructured":"Jian Ge, Huiqun Yu, Guisheng Fan, Jianhao Tang, and Zijie Huang. 2023. Just-in-time defect prediction for intelligent computing frameworks (in Chinese). Journal of Software 34, 9 (2023), 0\u20130.","journal-title":"Journal of Software"},{"issue":"3","key":"e_1_3_3_29_2","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1007\/s10664-021-10105-2","article-title":"Systematic literature review on software quality for AI-based software","volume":"27","author":"Gezici Bahar","year":"2022","unstructured":"Bahar Gezici and Ay\u00e7a Koluk\u0131sa Tarhan. 2022. Systematic literature review on software quality for AI-based software. Empirical Software Engineering 27, 3 (2022), 66.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21918"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.68"},{"issue":"2","key":"e_1_3_3_32_2","first-page":"240","article-title":"Defect detection for deep learning frameworks based on meta operators (in Chinese)","volume":"45","author":"Gu Diandian","year":"2022","unstructured":"Diandian Gu, Yining Shi, Haozhe Liu, Ge Wu, Haiou Jiang, Yaoshuai Zhao, and Yun Ma. 2022. Defect detection for deep learning frameworks based on meta operators (in Chinese). Chinese Journal of Computers 45, 2 (2022), 240\u2013255.","journal-title":"Chinese Journal of Computers"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510092"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00080"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416571"},{"key":"e_1_3_3_36_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MSR59073.2023.00018","volume-title":"2023 IEEE\/ACM 20th International Conference on Mining Software Repositories (MSR)","author":"Harzevili Nima Shiri","year":"2023","unstructured":"Nima Shiri Harzevili, Jiho Shin, Junjie Wang, Song Wang, and Nachiappan Nagappan. 2023. Characterizing and understanding software security vulnerabilities in machine learning libraries. In 2023 IEEE\/ACM 20th International Conference on Mining Software Repositories (MSR). IEEE, 27\u201338."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_3_38_2","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1109\/ITC50571.2021.00017","volume-title":"2021 IEEE International Test Conference (ITC)","author":"He Yi","year":"2021","unstructured":"Yi He, Takumi Uezono, and Yanjing Li. 2021. Efficient functional in-field self-test for deep learning accelerators. In 2021 IEEE International Test Conference (ITC). IEEE, 93\u2013102."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10073-7"},{"key":"e_1_3_3_40_2","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1109\/SANER60148.2024.00035","volume-title":"2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Hong Shuo","year":"2024","unstructured":"Shuo Hong, Hailong Sun, Xiang Gao, and Shin Hwei Tan. 2024. Investigating and detecting silent bugs in PyTorch programs. In 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 272\u2013283."},{"key":"e_1_3_3_41_2","article-title":"TOGLL: Correct and strong test oracle generation with LLMs","author":"Hossain Soneya Binta","year":"2024","unstructured":"Soneya Binta Hossain and Matthew Dwyer. 2024. TOGLL: Correct and strong test oracle generation with LLMs. arXiv preprint arXiv:2405.03786 (2024).","journal-title":"arXiv preprint arXiv:2405.03786"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650215.3650237"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616325"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_3_45_2","first-page":"461","volume-title":"2023 8th International Conference on Data Science in Cyberspace (DSC)","author":"Ji Jiahe","year":"2023","unstructured":"Jiahe Ji, Wei Kong, Jianwen Tian, Taotao Gu, Yuanping Nie, and Xiaohui Kuang. 2023. Survey on fuzzing techniques in deep learning libraries. In 2023 8th International Conference on Data Science in Cyberspace (DSC). IEEE, 461\u2013467."},{"key":"e_1_3_3_46_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1109\/ICSME52107.2021.00011","volume-title":"2021 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Jia Li","year":"2021","unstructured":"Li Jia, Hao Zhong, and Linpeng Huang. 2021. The unit test quality of deep learning libraries: A mutation analysis. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 47\u201357."},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1109\/SANER53432.2022.00097","volume-title":"2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Jia Li","year":"2022","unstructured":"Li Jia, Hao Zhong, Xiaoyin Wang, Linpeng Huang, and Zexuan Li. 2022. How do injected bugs affect deep learning?. In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 793\u2013804."},{"key":"e_1_3_3_48_2","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1007\/978-3-030-59410-7_40","volume-title":"Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24\u201327, 2020, Proceedings, Part I 25","author":"Jia Li","year":"2020","unstructured":"Li Jia, Hao Zhong, Xiaoyin Wang, Linpeng Huang, and Xuansheng Lu. 2020. An empirical study on bugs inside TensorFlow. In Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24\u201327, 2020, Proceedings, Part I 25. Springer, 604\u2013620."},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.110935"},{"key":"e_1_3_3_50_2","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1109\/COMPSAC48688.2020.0-156","volume-title":"2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC)","author":"Jiang Bo","year":"2020","unstructured":"Bo Jiang, Xiaoyan Wang, Wing Kwong Chan, T. H. Tse, Na Li, Yongfeng Yin, and Zhenyu Zhang. 2020. CUDAsmith: A fuzzer for CUDA compilers. In 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 861\u2013871."},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2345156.2254075"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330648"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2918851"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628055"},{"key":"e_1_3_3_55_2","article-title":"SkipFuzz: Active learning-based input selection for fuzzing deep learning libraries","author":"Kang Hong Jin","year":"2022","unstructured":"Hong Jin Kang, Pattarakrit Rattanukul, Stefanus Agus Haryono, Truong Giang Nguyen, Chaiyong Ragkhitwetsagul, Corina Pasareanu, and David Lo. 2022. SkipFuzz: Active learning-based input selection for fuzzing deep learning libraries. arXiv preprint arXiv:2212.04038 (2022).","journal-title":"arXiv preprint arXiv:2212.04038"},{"key":"e_1_3_3_56_2","unstructured":"K. Ken. 2023. Exclusive: Surveillance footage of Tesla crash on SF\u2019s Bay Bridge hours after Elon Musk announces \u201cself-driving\u201d feature. (2023). https:\/\/theintercept.com\/2023\/01\/10\/tesla-crash-footage-autopilot\/"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4842-2766-4"},{"key":"e_1_3_3_58_2","first-page":"540","volume-title":"2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR)","author":"Kim Misoo","year":"2021","unstructured":"Misoo Kim, Youngkyoung Kim, and Eunseok Lee. 2021. Denchmark: A bug benchmark of deep learning-related software. In 2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR). IEEE, 540\u2013544."},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.09.009"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3174092"},{"key":"e_1_3_3_61_2","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1145\/3510003.3510095","volume-title":"Proceedings of the 44th International Conference on Software Engineering","author":"Kloberdanz Eliska","year":"2022","unstructured":"Eliska Kloberdanz, Kyle G. Kloberdanz, and Wei Le. 2022. DeepStability: A study of unstable numerical methods and their solutions in deep learning. In Proceedings of the 44th International Conference on Software Engineering. 586\u2013597."},{"key":"e_1_3_3_62_2","article-title":"Quantifying the carbon emissions of machine learning","author":"Lacoste Alexandre","year":"2019","unstructured":"Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019).","journal-title":"arXiv preprint arXiv:1910.09700"},{"key":"e_1_3_3_63_2","article-title":"On security weaknesses and vulnerabilities in deep learning systems","author":"Lai Zhongzheng","year":"2024","unstructured":"Zhongzheng Lai, Huaming Chen, Ruoxi Sun, Yu Zhang, Minhui Xue, and Dong Yuan. 2024. On security weaknesses and vulnerabilities in deep learning systems. arXiv preprint arXiv:2406.08688 (2024).","journal-title":"arXiv preprint arXiv:2406.08688"},{"key":"e_1_3_3_64_2","article-title":"Comparing the costs of abstraction for DL frameworks","author":"Levental Maksim","year":"2020","unstructured":"Maksim Levental and Elena Orlova. 2020. Comparing the costs of abstraction for DL frameworks. arXiv preprint arXiv:2012.07163 (2020).","journal-title":"arXiv preprint arXiv:2012.07163"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2019.2928289"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/IISWC.2018.8573483"},{"key":"e_1_3_3_67_2","first-page":"440","volume-title":"2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)","author":"Li Cheng","year":"2020","unstructured":"Cheng Li, Abdul Dakkak, Jinjun Xiong, and Wen-mei Hwu. 2020. Benanza: Automatic \\(\\mu\\) Benchmark generation to compute\u201d lower-bound\u201d latency and inform optimizations of deep learning models on GPUs. In 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 440\u2013450."},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwx110"},{"key":"e_1_3_3_69_2","first-page":"6103","volume-title":"GLOBECOM 2022-2022 IEEE Global Communications Conference","author":"Li Junqiang","year":"2022","unstructured":"Junqiang Li, Senyi Li, Jiawei Wu, Long Luo, Yang Bai, and Hongfang Yu. 2022. MMOS: Multi-staged mutation operator scheduling for deep learning library testing. In GLOBECOM 2022-2022 IEEE Global Communications Conference. IEEE, 6103\u20136108."},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583566"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2020.3030548"},{"key":"e_1_3_3_72_2","first-page":"12559","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"36","author":"Li Xiaoting","year":"2022","unstructured":"Xiaoting Li, Xiao Liu, Lingwei Chen, Rupesh Prajapati, and Dinghao Wu. 2022. ALPHAPROG: Reinforcement generation of valid programs for compiler fuzzing. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 12559\u201312565."},{"key":"e_1_3_3_73_2","first-page":"328","volume-title":"2023 IEEE\/ACM 31st International Conference on Program Comprehension (ICPC)","author":"Li Zengyang","year":"2023","unstructured":"Zengyang Li, Sicheng Wang, Wenshuo Wang, Peng Liang, Ran Mo, and Bing Li. 2023. Understanding bugs in multi-language deep learning frameworks. In 2023 IEEE\/ACM 31st International Conference on Program Comprehension (ICPC). IEEE, 328\u2013338."},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330260"},{"key":"e_1_3_3_75_2","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1145\/3510454.3516826","volume-title":"Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings","author":"Liang Yunkai","year":"2022","unstructured":"Yunkai Liang, Yun Lin, Xuezhi Song, Jun Sun, Zhiyong Feng, and Jin Song Dong. 2022. gDefects4DL: A dataset of general real-world deep learning program defects. In Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings, 90\u201394."},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3672758.3672777"},{"key":"e_1_3_3_77_2","first-page":"616","volume-title":"2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS)","author":"Lin Kuiliang","year":"2023","unstructured":"Kuiliang Lin, Xiangpu Song, Yingpei Zeng, and Shanqing Guo. 2023. DeepDiffer: Find deep learning compiler bugs via priority-guided differential fuzzing. In 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS). IEEE, 616\u2013627."},{"key":"e_1_3_3_78_2","first-page":"152","volume-title":"2012 Fourth International Conference on Multimedia Information Networking and Security","author":"Liu Bingchang","year":"2012","unstructured":"Bingchang Liu, Liang Shi, Zhuhua Cai, and Min Li. 2012. Software vulnerability discovery techniques: A survey. In 2012 Fourth International Conference on Multimedia Information Networking and Security. IEEE, 152\u2013156."},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377815.3381377"},{"issue":"2","key":"e_1_3_3_80_2","first-page":"1","article-title":"Generation-based differential fuzzing for deep learning libraries","volume":"33","author":"Liu Jiawei","year":"2023","unstructured":"Jiawei Liu, Yuheng Huang, Zhijie Wang, Lei Ma, Chunrong Fang, Mingzheng Gu, Xufan Zhang, and Zhenyu Chen. 2023. Generation-based differential fuzzing for deep learning libraries. ACM Transactions on Software Engineering and Methodology 33, 2 (2023), 1\u201328.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3575693.3575707"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616337"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527317"},{"key":"e_1_3_3_84_2","first-page":"1258","volume-title":"2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)","author":"Liu Ling","year":"2018","unstructured":"Ling Liu, Yanzhao Wu, Wenqi Wei, Wenqi Cao, Semih Sahin, and Qi Zhang. 2018. Benchmarking deep learning frameworks: Design considerations, metrics and beyond. In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS). IEEE, 1258\u20131269."},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597204"},{"key":"e_1_3_3_86_2","volume-title":"Forty-First International Conference on Machine Learning","author":"Liu Zechun","unstructured":"Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, et\u00a0al. MobileLLM: Optimizing sub-billion parameter language models for on-device use cases. In Forty-First International Conference on Machine Learning."},{"key":"e_1_3_3_87_2","first-page":"423","volume-title":"2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE)","author":"Liu Zhihao","year":"2022","unstructured":"Zhihao Liu, Yang Zheng, Xiaoting Du, Zheng Hu, Wenjie Ding, Yanming Miao, and Zheng Zheng. 2022. Taxonomy of aging-related bugs in deep learning libraries. In 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE). IEEE, 423\u2013434."},{"key":"e_1_3_3_88_2","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/ICSME58846.2023.00054","volume-title":"2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Louloudakis Nikolaos","year":"2023","unstructured":"Nikolaos Louloudakis, Perry Gibson, Jos\u00e9 Cano, and Ajitha Rajan. 2023. DeltaNN: Assessing the impact of computational environment parameters on the performance of image recognition models. In 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 414\u2013424."},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598053"},{"key":"e_1_3_3_90_2","article-title":"A survey on testing of deep learning frameworks (in Chinese)","author":"Ma Xiangyue","year":"2023","unstructured":"Xiangyue Ma, Xiaoting Du, Qing Cai, Yang Zheng, Jing Hu, and Zheng Zheng. 2023. A survey on testing of deep learning frameworks (in Chinese). Journal of Software (2023).","journal-title":"Journal of Software"},{"key":"e_1_3_3_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2946563"},{"key":"e_1_3_3_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"issue":"1","key":"e_1_3_3_93_2","first-page":"100","article-title":"Differential testing for software","volume":"10","author":"McKeeman William M.","year":"1998","unstructured":"William M. McKeeman. 1998. Differential testing for software. Digital Technical Journal 10, 1 (1998), 100\u2013107.","journal-title":"Digital Technical Journal"},{"key":"e_1_3_3_94_2","unstructured":"Microsoft. 2023. ONNX Github repository. (2023). https:\/\/github.com\/onnx\/onnx"},{"key":"e_1_3_3_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695523"},{"key":"e_1_3_3_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555776.3578729"},{"key":"e_1_3_3_97_2","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1109\/ASE.2019.00078","volume-title":"2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Nejadgholi Mahdi","year":"2019","unstructured":"Mahdi Nejadgholi and Jinqiu Yang. 2019. A study of oracle approximations in testing deep learning libraries. In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 785\u2013796."},{"key":"e_1_3_3_98_2","first-page":"1","volume-title":"2024 International Conference on Electronic Engineering and Information Systems (EEISS)","author":"Nie Yuanping","year":"2024","unstructured":"Yuanping Nie, Xiong Xiao, Bing Yang, Hanqing Li, Long Luo, Hongfang Yu, and Gang Sun. 2024. Python coverage guided fuzzing for deep learning framework. In 2024 International Conference on Electronic Engineering and Information Systems (EEISS). IEEE, 1\u20136."},{"key":"e_1_3_3_99_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.100"},{"key":"e_1_3_3_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2005.55"},{"issue":"76","key":"e_1_3_3_101_2","doi-asserted-by":"crossref","first-page":"40","DOI":"10.21205\/deufmd.2024267606","article-title":"Quantitative performance analysis of BLAS libraries on GPU architectures","volume":"26","author":"\u00d6z I\u015f\u0131l","year":"2024","unstructured":"I\u015f\u0131l \u00d6z. 2024. Quantitative performance analysis of BLAS libraries on GPU architectures. Dokuz Eyl\u00fcl \u00dcniversitesi M\u00fchendislik Fak\u00fcltesi Fen ve M\u00fchendislik Dergisi 26, 76 (2024), 40\u201348.","journal-title":"Dokuz Eyl\u00fcl \u00dcniversitesi M\u00fchendislik Fak\u00fcltesi Fen ve M\u00fchendislik Dergisi"},{"key":"e_1_3_3_102_2","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et\u00a0al. 2019. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems 32 (2019).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_103_2","article-title":"Carbon emissions and large neural network training","author":"Patterson David","year":"2021","unstructured":"David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. 2021. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 (2021).","journal-title":"arXiv preprint arXiv:2104.10350"},{"key":"e_1_3_3_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00107"},{"key":"e_1_3_3_105_2","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1145\/3510454.3516835","volume-title":"Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings","author":"Prochnow Alexander","year":"2022","unstructured":"Alexander Prochnow and Jinqiu Yang. 2022. DiffWatch: Watch out for the evolving differential testing in deep learning libraries. In Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings. 46\u201350."},{"key":"e_1_3_3_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3560427"},{"key":"e_1_3_3_107_2","article-title":"Effective random test generation for deep learning compilers","author":"Ren Luyao","year":"2023","unstructured":"Luyao Ren, ZiHeng Wang, Yingfei Xiong, Li Zhang, Guoyue Jiang, and Tao Xie. 2023. Effective random test generation for deep learning compilers. arXiv preprint arXiv:2302.00842 (2023).","journal-title":"arXiv preprint arXiv:2302.00842"},{"key":"e_1_3_3_108_2","article-title":"Glow: Graph lowering compiler techniques for neural networks","author":"Rotem Nadav","year":"2018","unstructured":"Nadav Rotem, Jordan Fix, Saleem Abdulrasool, Garret Catron, Summer Deng, Roman Dzhabarov, Nick Gibson, James Hegeman, Meghan Lele, Roman Levenstein, et\u00a0al. 2018. Glow: Graph lowering compiler techniques for neural networks. arXiv preprint arXiv:1805.00907 (2018).","journal-title":"arXiv preprint arXiv:1805.00907"},{"key":"e_1_3_3_109_2","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1145\/3510003.3510112","volume-title":"Proceedings of the 44th International Conference on Software Engineering","author":"Schumi Richard","year":"2022","unstructured":"Richard Schumi and Jun Sun. 2022. ExAIS: Executable AI semantics. In Proceedings of the 44th International Conference on Software Engineering. 859\u2013870."},{"key":"e_1_3_3_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_3_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468591"},{"key":"e_1_3_3_112_2","first-page":"305","volume-title":"2025 IEEE\/ACM 47th International Conference on Software Engineering (ICSE)","author":"Shen Qingchao","year":"2024","unstructured":"Qingchao Shen, Yongqiang Tian, Haoyang Ma, Junjie Chen, Lili Huang, Ruifeng Fu, Shing-Chi Cheung, and Zan Wang. 2024. A tale of two DL cities: When library tests meet compiler. In 2025 IEEE\/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 305\u2013316."},{"key":"e_1_3_3_113_2","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1109\/DSC53577.2021.00059","volume-title":"2021 IEEE Sixth International Conference on Data Science in Cyberspace (DSC)","author":"Shen Xiangzhong","year":"2021","unstructured":"Xiangzhong Shen, Jieyi Zhang, Xiaonan Wang, Hongfang Yu, and Gang Sun. 2021. Deep learning framework fuzzing based on model mutation. In 2021 IEEE Sixth International Conference on Data Science in Cyberspace (DSC). IEEE, 375\u2013380."},{"key":"e_1_3_3_114_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598088"},{"key":"e_1_3_3_115_2","unstructured":"ABC7.com staff. 2018. Uber gives up testing of self-driving cars in California in wake of fatal Arizona crash. (2018). https:\/\/abc7.com\/self-driving-uber-crash-video-pedestrian-hit-by-car-autonomous-vehicles\/3269690\/"},{"key":"e_1_3_3_116_2","unstructured":"Y. Sun. 2020. Tesla and PyTorch: PyTorch Developer Conference Highlights. (2020). https:\/\/medium.com\/data-science-bootcamp\/tesla-and-pytorch-pytorch-developer-conference-highlights-part-3ed36f2c9d5e"},{"key":"e_1_3_3_117_2","first-page":"209","volume-title":"2018 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)","author":"Sun Yifan","year":"2018","unstructured":"Yifan Sun, Saoni Mukherjee, Trinayan Baruah, Shi Dong, Julian Gutierrez, Prannoy Mohan, and David Kaeli. 2018. Evaluating performance tradeoffs on the radeon open compute platform. In 2018 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). IEEE, 209\u2013218."},{"key":"e_1_3_3_118_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10389-6"},{"key":"e_1_3_3_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2010.39"},{"key":"e_1_3_3_120_2","unstructured":"TensorFlow. 2020. Learn how TensorFlow solves real everyday machine learning problems. (2020). https:\/\/www.tensorflow.org\/about\/case-studies"},{"key":"e_1_3_3_121_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITC50671.2022.00054"},{"key":"e_1_3_3_122_2","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1145\/3524842.3528455","volume-title":"Proceedings of the 19th International Conference on Mining Software Repositories","author":"V\u00e9lez Tatiana Castro","year":"2022","unstructured":"Tatiana Castro V\u00e9lez, Raffi Khatchadourian, Mehdi Bagherzadeh, and Anita Raja. 2022. Challenges in migrating imperative deep learning programs to graph execution: An empirical study. In Proceedings of the 19th International Conference on Mining Software Repositories. 469\u2013481."},{"key":"e_1_3_3_123_2","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1145\/3528416.3530251","volume-title":"Proceedings of the 19th ACM International Conference on Computing Frontiers","author":"Verma Gaurav","year":"2022","unstructured":"Gaurav Verma, Swetang Finviya, Abid M. Malik, Murali Emani, and Barbara Chapman. 2022. Towards neural architecture-aware exploration of compiler optimizations in a deep learning \\(\\lbrace\\) graph \\(\\rbrace\\) compiler. In Proceedings of the 19th ACM International Conference on Computing Frontiers. 244\u2013250."},{"key":"e_1_3_3_124_2","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1109\/IPDPSW52791.2021.00128","volume-title":"2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)","author":"Verma Gaurav","year":"2021","unstructured":"Gaurav Verma, Yashi Gupta, Abid M. Malik, and Barbara Chapman. 2021. Performance evaluation of deep learning compilers for edge inference. In 2021 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 858\u2013865."},{"key":"e_1_3_3_125_2","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/AITEST49225.2020.00015","volume-title":"2020 IEEE International Conference On Artificial Intelligence Testing (AITest)","author":"Wang Chaojin","year":"2020","unstructured":"Chaojin Wang, Jian Shen, Chunrong Fang, Xiangsheng Guan, Kaitao Wu, and Jiang Wang. 2020. Accuracy measurement of deep neural network accelerator via metamorphic testing. In 2020 IEEE International Conference On Artificial Intelligence Testing (AITest). IEEE, 55\u201361."},{"key":"e_1_3_3_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510165"},{"key":"e_1_3_3_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616321"},{"key":"e_1_3_3_128_2","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1109\/ASE51524.2021.9678792","volume-title":"2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Wang Jiyuan","year":"2021","unstructured":"Jiyuan Wang, Qian Zhang, Guoqing Harry Xu, and Miryung Kim. 2021. QDiff: Differential testing of quantum software stacks. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 692\u2013704."},{"key":"e_1_3_3_129_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598105"},{"key":"e_1_3_3_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409761"},{"key":"e_1_3_3_131_2","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1109\/ICSE43902.2021.00034","volume-title":"2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE)","author":"Wardat Mohammad","year":"2021","unstructured":"Mohammad Wardat, Wei Le, and Hridesh Rajan. 2021. DeepLocalize: Fault localization for deep neural networks. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 251\u2013262."},{"key":"e_1_3_3_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510041"},{"key":"e_1_3_3_133_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639177"},{"key":"e_1_3_3_134_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_3_135_2","first-page":"399","volume-title":"2022 7th IEEE International Conference on Data Science in Cyberspace (DSC)","author":"Wu Jiawei","year":"2022","unstructured":"Jiawei Wu, Senyi Li, Junqiang Li, Long Luo, Hongfang Yu, and Gang Sun. 2022. DeepCov: Coverage guided deep learning framework fuzzing. In 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC). IEEE, 399\u2013404."},{"key":"e_1_3_3_136_2","first-page":"56","volume-title":"2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Wu Mingyuan","year":"2023","unstructured":"Mingyuan Wu, Minghai Lu, Heming Cui, Junjie Chen, Yuqun Zhang, and Lingming Zhang. 2023. JITfuzz: Coverage-guided fuzzing for JVM just-in-time compilers. In 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 56\u201368."},{"key":"e_1_3_3_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/3549497"},{"key":"e_1_3_3_138_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534220"},{"key":"e_1_3_3_139_2","volume-title":"International Conference on Software Analysis, Evolution, and Reengineering,","author":"Xie Danning","year":"2024","unstructured":"Danning Xie, Jiannan Wang, Hung Viet Pham, Lin Tan, Yu Guo, Adnan Aziz, and Erik Meijer. 2024. CEDAR: Continuous testing of deep learning libraries. In International Conference on Software Analysis, Evolution, and Reengineering,. IEEE."},{"key":"e_1_3_3_140_2","article-title":"Fuzzing automatic differentiation in deep-learning libraries","author":"Yang Chenyuan","year":"2023","unstructured":"Chenyuan Yang, Yinlin Deng, Jiayi Yao, Yuxing Tu, Hanchi Li, and Lingming Zhang. 2023. Fuzzing automatic differentiation in deep-learning libraries. arXiv preprint arXiv:2302.04351 (2023).","journal-title":"arXiv preprint arXiv:2302.04351"},{"key":"e_1_3_3_141_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.107004"},{"key":"e_1_3_3_142_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_3_143_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380362"},{"key":"e_1_3_3_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00020"},{"key":"e_1_3_3_145_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2021.3107165"},{"issue":"12","key":"e_1_3_3_146_2","first-page":"2884","article-title":"The testing and repairing methods for machine learning model security","volume":"50","author":"Zhang Xiaoyu","year":"2022","unstructured":"Xiaoyu Zhang, Chao Shen, Chenhao Lin, Qian Li, Qian Wang, Qi Li, and Xiaohong Guan. 2022. The testing and repairing methods for machine learning model security. Acta Electonica Sinica 50, 12 (2022), 2884.","journal-title":"Acta Electonica Sinica"},{"key":"e_1_3_3_147_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464843"},{"key":"e_1_3_3_148_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00043"},{"key":"e_1_3_3_149_2","article-title":"CITADEL: Context similarity based deep learning framework bug finding","author":"Zhang Xiaoyu","year":"2024","unstructured":"Xiaoyu Zhang, Juan Zhai, Shiqing Ma, Shiwei Wang, and Chao Shen. 2024. CITADEL: Context similarity based deep learning framework bug finding. arXiv preprint arXiv:2406.12196 (2024).","journal-title":"arXiv preprint arXiv:2406.12196"},{"key":"e_1_3_3_150_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"},{"key":"e_1_3_3_151_2","volume-title":"Forty-First International Conference on Machine Learning","author":"Zhang Yihua","unstructured":"Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenqing Zheng, Pin-Yu Chen, Jason D. Lee, Wotao Yin, Mingyi Hong, et\u00a0al. Revisiting zeroth-order optimization for memory-efficient LLM fine-tuning: A benchmark. In Forty-First International Conference on Machine Learning."},{"key":"e_1_3_3_152_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106701"},{"key":"e_1_3_3_153_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556894"},{"key":"e_1_3_3_154_2","doi-asserted-by":"publisher","DOI":"10.1145\/3689757"},{"key":"e_1_3_3_155_2","volume-title":"Proceedings of the 2025 IEEE Symposium on Security and Privacy (S&P)","author":"Zhu Ruofan","year":"2025","unstructured":"Ruofan Zhu, Ganhao Chen, Wenbo Shen, Xiaofei Xie, and Rui Chang. 2025. My model is malware to you: Transforming AI models into malware by abusing TensorFlow APIs. In Proceedings of the 2025 IEEE Symposium on Security and Privacy (S&P). IEEE, IEEE."},{"key":"e_1_3_3_156_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111681"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3716497","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3716497","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:19:10Z","timestamp":1750295950000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3716497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,5]]},"references-count":155,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2025,7,31]]}},"alternative-id":["10.1145\/3716497"],"URL":"https:\/\/doi.org\/10.1145\/3716497","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,5]]},"assertion":[{"value":"2024-02-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-05","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}