{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T16:16:14Z","timestamp":1783613774614,"version":"3.55.0"},"reference-count":223,"publisher":"Association for Computing Machinery (ACM)","issue":"1","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,1,31]]},"abstract":"<jats:p>The rapid advancement of AI has led to its integration into various areas, especially with Large Language Models (LLMs) significantly enhancing capabilities in Artificial Intelligence Generated Content (AIGC). However, the complexity of AI systems has also exposed their vulnerabilities, necessitating robust methods for Failure Analysis (FA) and Fault Injection (FI) to ensure resilience and reliability. Despite the importance of these techniques, there lacks a comprehensive review of FA and FI methodologies in AI systems. This study fills this gap by presenting a detailed survey of existing FA and FI approaches across six layers of AI systems. We systematically analyze 142 studies to answer three research questions including (1) what are the prevalent failures in AI systems, (2) what types of faults can current FI tools simulate, (3) what gaps exist between the simulated faults and real-world failures. Our findings reveal a taxonomy of AI system failures, assess the capabilities of existing FI tools, and highlight discrepancies between real-world and simulated failures. Moreover, this survey contributes to the field by providing a framework for fault diagnosis, evaluating the state-of-the-art in FI, and identifying areas for improvement in FI techniques to enhance the resilience of AI systems.<\/jats:p>","DOI":"10.1145\/3732777","type":"journal-article","created":{"date-parts":[[2025,5,16]],"date-time":"2025-05-16T12:49:28Z","timestamp":1747399768000},"page":"1-42","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["A Survey on Failure Analysis and Fault Injection in AI Systems"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6195-9088","authenticated-orcid":false,"given":"Guangba","family":"Yu","sequence":"first","affiliation":[{"name":"Sun Yat-Sen University, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-6580-1470","authenticated-orcid":false,"given":"Gou","family":"Tan","sequence":"additional","affiliation":[{"name":"Sun Yat-Sen University, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5592-7782","authenticated-orcid":false,"given":"Haojia","family":"Huang","sequence":"additional","affiliation":[{"name":"Sun Yat-Sen University, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-5130-0567","authenticated-orcid":false,"given":"Zhenyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Sun Yat-Sen University, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0972-6900","authenticated-orcid":false,"given":"Pengfei","family":"Chen","sequence":"additional","affiliation":[{"name":"Sun Yat-Sen University, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1084-4824","authenticated-orcid":false,"given":"Roberto","family":"Natella","sequence":"additional","affiliation":[{"name":"University of Naples Federico II, Napoli, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7878-4330","authenticated-orcid":false,"given":"Zibin","family":"Zheng","sequence":"additional","affiliation":[{"name":"Sun Yat-Sen University, Zhuhai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3666-5798","authenticated-orcid":false,"given":"Michael R.","family":"Lyu","sequence":"additional","affiliation":[{"name":"Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,12,12]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Mart\u00edn Abadi Paul Barham Jianmin Chen Zhifeng Chen Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Geoffrey Irving Michael Isard et al. 2016. TensorFlow: A system for large-scale machine learning. In OSDI 2016. USENIX Association 265\u2013283. Retrieved from https:\/\/www.usenix.org\/conference\/osdi16\/technical-sessions\/presentation\/abadi"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3605764.3623985"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-S58398.2023.00063"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3127960"},{"key":"e_1_3_3_6_2","unstructured":"Alexei95. 2024. Enpheeph Github. Retrieved from https:\/\/github.com\/Alexei95\/enpheeph"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/DFTVS.2002.1173521"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2004.2"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.nlposs-1.24"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.NEUCOM.2019.10.008"},{"key":"e_1_3_3_11_2","doi-asserted-by":"crossref","unstructured":"Michael Beyer Andrey Morozov Emil Valiev Christoph Schorn Lydia Gauerhof Kai Ding and Klaus Janschek. 2020. Fault injectors for TensorFlow: Evaluation of the impact of random hardware faults on deep CNNs. arXiv:2012.07037. Retrieved fromhttps:\/\/arxiv.org\/abs\/2012.07037","DOI":"10.3850\/978-981-14-8593-0_5754-cd"},{"key":"e_1_3_3_12_2","unstructured":"Bfgoldstein. 2024. Torchfi GitHub. Retrieved fromhttps:\/\/github.com\/bfgoldstein\/torchfi"},{"issue":"2010","key":"e_1_3_3_13_2","first-page":"1601","article-title":"MOA: Massive online analysis","volume":"11","author":"Bifet Albert","year":"2010","unstructured":"Albert Bifet, Geoff Holmes, Richard Kirkby, and Bernhard Pfahringer. 2010. MOA: Massive online analysis. Journal of Machine Learning Research 11 (2010), 1601\u20131604. Retrieved fromhttps:\/\/api.semanticscholar.org\/CorpusID:12397401","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNS.2024.3359524"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2021.3105697"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549123"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN53405.2022.00027"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS54544.2021.00118"},{"key":"e_1_3_3_19_2","unstructured":"Chaosblade. 2024. Chaosblade Github. Retrieved from https:\/\/github.com\/chaosblade-io\/chaosblade"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2024.3363902"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416548"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP40000.2020.00045"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3587155"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE5003.2020.00047"},{"key":"e_1_3_3_25_2","unstructured":"ChrisLidbury. 2024. CLSmith Github. Retrieved from https:\/\/github.com\/ChrisLidbury\/CLSmith\/"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ATS.2001.990301"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37036-6_16"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/IROS47612.2022.9982181"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/VTS50974.2021.9441044"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TII.2009.2039938"},{"key":"e_1_3_3_32_2","first-page":"56","volume-title":"CIbSE24","author":"Costal Dolors","year":"2021","unstructured":"Dolors Costal, Carles Farr\u00e9, Xavier Franch, and Carme Quer. 2021. How tertiary studies perform quality assessment of secondary studies in software engineering. In CIbSE24. Curran Associates, 56\u201369."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-S60304.2024.00016"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.JNCA.2022.103334"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00022"},{"key":"e_1_3_3_36_2","unstructured":"Saikat Das. 2022. 10 Frequently Encountered Issues in Data Preprocessing. Retrieved from https:\/\/www.analyticsvidhya.com\/blog\/2022\/08\/10-frequently-encountered-issues-in-data-preprocessing\/"},{"key":"e_1_3_3_37_2","doi-asserted-by":"crossref","unstructured":"Jiachao Deng Yuntan Fang Zidong Du Ying Wang Huawei Li Olivier Temam Paolo Ienne David Novo Xiaowei Li Yunji Chen et al. 2015. Retraining-based timing error mitigation for hardware neural networks. In DATE 2015. ACM 593\u2013596. Retrieved from http:\/\/dl.acm.org\/citation.cfm?id=2755887","DOI":"10.7873\/DATE.2015.0849"},{"key":"e_1_3_3_38_2","unstructured":"DependableSystemsLab. 2024. TensorFI Github. Retrieved from https:\/\/github.com\/DependableSystemsLab\/TensorFI"},{"key":"e_1_3_3_39_2","unstructured":"DependableSystemsLab. 2024. TensorFI2 Github. Retrieved from https:\/\/github.com\/DependableSystemsLab\/TensorFI2"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2014.62"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2406.04594"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE5003.2020.00010"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.NEUCOM.2022.06.111"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.MICROREL.2014.01.002"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2014.6844486"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2015.31"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00186"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP58684.2023.00052"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-021-10105-2"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3542929.3563482"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/LASCAS45839.2020.9069026"},{"key":"e_1_3_3_52_2","unstructured":"Gongbell. 2024. CUDAsmith Github. Retrieved from https:\/\/github.com\/gongbell\/CUDAsmith"},{"key":"e_1_3_3_53_2","unstructured":"Ian J. Goodfellow Jonathon Shlens and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In ICLR 2015. Retrieved from http:\/\/arxiv.org\/abs\/1412.6572"},{"key":"e_1_3_3_54_2","unstructured":"Google. 2024. Google Gemini. Retrieved from https:\/\/ai.google.dev\/"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/2987550.2987583"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126937"},{"key":"e_1_3_3_57_2","unstructured":"Wes Gurnee Neel Nanda Matthew Pauly Katherine Harvey Dmitrii Troitskii and Dimitris Bertsimas. 2023. Finding neurons in a haystack: Case studies with sparse probing. Transactions on Machine Learning Research 2023 (2023) 1\u201347. Retrieved from https:\/\/openreview.net\/forum?id=JYs1R9IMJr"},{"key":"e_1_3_3_58_2","first-page":"2715","volume-title":"Journal of Archaeology of Egypt\/Egyptology","author":"Haji Saad Hikmat","year":"2021","unstructured":"Saad Hikmat Haji and Adnan Mohsin Abdulazeez. 2021. Comparison of optimization techniques based on gradient descent algorithm: A review. Journal of Archaeology of Egypt\/Egyptology 18, 4 (2021), 2715\u20132743."},{"key":"e_1_3_3_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPASS.2017.7975296"},{"key":"e_1_3_3_60_2","unstructured":"Fengxiang He Tongliang Liu and Dacheng Tao .2019. Control batch size and learning rate to generalize well: Theoretical and empirical evidence. In NeurIPS 2019 1141\u20131150. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/dc6a70712a252123c40d2adba6a11d84-Abstract.html"},{"key":"e_1_3_3_61_2","unstructured":"Dan Hendrycks and Thomas G. Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. In ICLR 2019. OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=HJz6tiCqYm"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2302.09210"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/979-8-8688-0376-5"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ATS52891.2021.00034"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00126"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458817.3476223"},{"key":"e_1_3_3_67_2","unstructured":"Qinghao Hu Zhisheng Ye Zerui Wang Guoteng Wang Meng Zhang Qiaoling Chen Peng Sun Dahua Lin Xiaolin Wang Yingwei Luo et al. 2024. Characterization of large language model development in the datacenter. In NSDI 2024. USENIX Association 709\u2013729. Retrieved from https:\/\/www.usenix.org\/conference\/nsdi24\/presentation\/hu"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616325"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616298"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464825"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_3_3_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00028"},{"key":"e_1_3_3_74_2","unstructured":"Istio. 2024. Istio HTTPDirectResponse Fault Injection. Retrieved from https:\/\/istio.io\/latest\/docs\/reference\/config\/networking\/virtual-service\/#HTTPDirectResponse"},{"key":"e_1_3_3_75_2","unstructured":"Istio. 2024. Istio Network Fault Injection. Retrieved from https:\/\/istio.io\/latest\/docs\/tasks\/traffic-management\/fault-injection\/"},{"key":"e_1_3_3_76_2","unstructured":"Myeongjae Jeon Shivaram Venkataraman Amar Phanishayee Unjie Qian Wencong Xiao and Fan Yang. 2019. Analysis of large-scale multi-tenant GPU clusters for DNN training workloads. In USENIX ATC 2019. USENIX 947\u2013960. Retrieved from https:\/\/www.usenix.org\/conference\/atc19\/presentation\/jeon"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00097"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59410-7_40"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.62"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC48688.2020.0-156"},{"key":"e_1_3_3_81_2","first-page":"745","volume-title":"NSDI 2024","author":"Jiang Ziheng","year":"2024","unstructured":"Ziheng Jiang, Haibin Lin, Yinmin Zhong, Qi Huang, Yangrui Chen, Zhi Zhang, Yanghua Peng, Xiang Li, Cong Xie, Shibiao Nong, et al. 2024. MegaScale: Scaling large language model training to more than 10, 000 GPUs. In NSDI 2024. USENIX Association, 745\u2013760. Retrieved from https:\/\/www.usenix.org\/conference\/nsdi24\/presentation\/jiang-ziheng"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1002\/SAM.11583"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2889253"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/RADECS55911.2022.10412572"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.23919\/DATE54114.2022.9774600"},{"key":"e_1_3_3_86_2","unstructured":"Keras. 2024. Keras. Retrieved from https:\/\/keras.io\/"},{"key":"e_1_3_3_87_2","doi-asserted-by":"publisher","DOI":"10.5220\/0011851400003488"},{"key":"e_1_3_3_88_2","unstructured":"Kmaork. 2024. Inject Python Code into a Running Python Process. Retrieved from https:\/\/github.com\/kmaork\/hypno"},{"key":"e_1_3_3_89_2","unstructured":"Tom Kocmi and Christian Federmann .2023. Large language models are state-of-the-art evaluators of translation quality. In EAMT 2023. European Association for Machine Translation 193\u2013203. Retrieved from https:\/\/aclanthology.org\/2023.eamt-1.19"},{"key":"e_1_3_3_90_2","doi-asserted-by":"publisher","DOI":"10.1007\/S11280-020-00778-Y"},{"key":"e_1_3_3_91_2","unstructured":"Alyssa Lamberti. 2023. 16 Most Common Network Problems: How to Find & Fix Them. Retrieved from https:\/\/obkio.com\/blog\/common-network-problems\/"},{"key":"e_1_3_3_92_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-74643-7_21"},{"key":"e_1_3_3_93_2","unstructured":"Lebronmydx. 2024. Simulee Github. Retrieved from https:\/\/github.com\/Lebronmydx\/Simulee"},{"key":"e_1_3_3_94_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-65854-0_4"},{"key":"e_1_3_3_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126964"},{"key":"e_1_3_3_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2016.20"},{"key":"e_1_3_3_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE55969.2022.00022"},{"key":"e_1_3_3_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC58990.2023.00047"},{"key":"e_1_3_3_99_2","doi-asserted-by":"publisher","DOI":"10.1038\/S42256-022-00548-7"},{"key":"e_1_3_3_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2737986"},{"key":"e_1_3_3_101_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579370.3594777"},{"key":"e_1_3_3_102_2","doi-asserted-by":"publisher","DOI":"10.1145\/3317550.3321438"},{"key":"e_1_3_3_103_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173162.3177161"},{"key":"e_1_3_3_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597204"},{"key":"e_1_3_3_105_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2306.05499"},{"key":"e_1_3_3_106_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE55969.2022.00048"},{"key":"e_1_3_3_107_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICWS.2017.135"},{"key":"e_1_3_3_108_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-21213-0_2"},{"key":"e_1_3_3_109_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.SYSARC.2022.102701"},{"key":"e_1_3_3_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISOCC.2017.8368905"},{"key":"e_1_3_3_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00021"},{"key":"e_1_3_3_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3558946"},{"key":"e_1_3_3_113_2","unstructured":"Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In ICLR 2018. OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=rJzIBfZAb"},{"key":"e_1_3_3_114_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.gltp.2022.04.020"},{"key":"e_1_3_3_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-W50199.2020.00014"},{"key":"e_1_3_3_116_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME55016.2022.00012"},{"key":"e_1_3_3_117_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCC.2022.3194128"},{"key":"e_1_3_3_118_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"key":"e_1_3_3_119_2","unstructured":"Molin Matti and B\u00f6hme Fredrik. 2023. AI for Cybersecurity: A Study on Machine Learning and DoS Attacks AI Robustness and Bypassing Detection Methods. Retrieved from https:\/\/www.diva-portal.org\/smash\/get\/diva2:1777110\/FULLTEXT02"},{"key":"e_1_3_3_120_2","unstructured":"Mbsa-tud. 2024. InjectTF Github. Retrieved from https:\/\/github.com\/mbsa-tud\/InjectTF"},{"key":"e_1_3_3_121_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.net.2016.03.004"},{"key":"e_1_3_3_122_2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.03853"},{"key":"e_1_3_3_123_2","unstructured":"Microsoft. 2024. Microsoft Bing Chat. Retrieved from https:\/\/www.bing.com\/chat"},{"key":"e_1_3_3_124_2","unstructured":"Mindspore-ai. 2024. MindSpore Github. Retrieved from https:\/\/github.com\/mindspore-ai\/mindspore"},{"key":"e_1_3_3_125_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2304.01964"},{"key":"e_1_3_3_126_2","first-page":"1","article-title":"Scikit-Multiflow: A multi-output streaming framework","volume":"19","author":"Montiel Jacob","year":"2018","unstructured":"Jacob Montiel, Jesse Read, Albert Bifet, and Talel Abdessalem. 2018. Scikit-Multiflow: A multi-output streaming framework. Journal of Machine Learning Research 19, 72 (2018), 1\u20135. Retrieved from http:\/\/jmlr.org\/papers\/v19\/18-251.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_3_127_2","unstructured":"Philipp Moritz Robert Nishihara Stephanie Wang Alexey Tumanov Richard Liaw Eric Liang Melih Elibol Zongheng Yang William Paul Michael I. Jordan et al. 2018. Ray: A distributed framework for emerging AI applications. In OSDI 2018. USENIX Association 561\u2013577. Retrieved from https:\/\/www.usenix.org\/conference\/osdi18\/presentation\/nishihara"},{"key":"e_1_3_3_128_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.INFSOF.2020.106294"},{"key":"e_1_3_3_129_2","doi-asserted-by":"publisher","unstructured":"Seyed Mahed Mousavi Simone Alghisi and Giuseppe Riccardi. 2024. Is your LLM outdated? Benchmarking LLMs & alignment algorithms for time-sensitive knowledge. arXiv:2404.08700. DOI: 10.48550\/ARXIV.2404.08700","DOI":"10.48550\/ARXIV.2404.08700"},{"key":"e_1_3_3_130_2","unstructured":"mpich. 2024. MPICH. Retrieved from https:\/\/www.mpich.org\/"},{"key":"e_1_3_3_131_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2022.3175930"},{"key":"e_1_3_3_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/2841425"},{"key":"e_1_3_3_133_2","doi-asserted-by":"publisher","DOI":"10.1109\/DFT.2012.6378216"},{"key":"e_1_3_3_134_2","doi-asserted-by":"publisher","DOI":"10.1002\/STVR.1809"},{"key":"e_1_3_3_135_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2016.7446091"},{"key":"e_1_3_3_136_2","doi-asserted-by":"publisher","DOI":"10.1109\/MASCOTS.2017.12"},{"key":"e_1_3_3_137_2","doi-asserted-by":"crossref","unstructured":"Bin Nie Ji Xue Saurabh Gupta Tirthak Patel Christian Engelmann Evgenia Smirni and Devesh Tiwari. 2018. Machine learning models for GPU error prediction in a large scale HPC system. In DSN 2018. IEEE\/IFIP 95\u2013106.","DOI":"10.1109\/DSN.2018.00022"},{"key":"e_1_3_3_138_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10515-021-00313-X"},{"key":"e_1_3_3_139_2","unstructured":"NumPy. 2024. NumPy Github. Retrieved from https:\/\/github.com\/numpy\/numpy"},{"key":"e_1_3_3_140_2","unstructured":"NVIDIA. 2024. CUDA Toolkit. Retrieved from https:\/\/developer.nvidia.com\/cuda-toolkit"},{"key":"e_1_3_3_141_2","unstructured":"NVIDIA. 2024. NVIDIA DCGM Error Injection. Retrieved from https:\/\/docs.nvidia.com\/datacenter\/dcgm\/latest\/user-guide\/dcgm-error-injection.html"},{"key":"e_1_3_3_142_2","unstructured":"NVIDIA. 2024. NVIDIA NCCL. Retrieved from https:\/\/developer.nvidia.com\/nccl"},{"key":"e_1_3_3_143_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2017.41"},{"key":"e_1_3_3_144_2","unstructured":"OpenAI. 2024. OpenAI Chatgpt. Retrieved from https:\/\/openai.com\/chatgpt"},{"key":"e_1_3_3_145_2","unstructured":"OpenAI. 2024. OpenAI Incidents. Retrieved from https:\/\/status.openai.com\/history"},{"key":"e_1_3_3_146_2","doi-asserted-by":"publisher","DOI":"10.1109\/SC41405.2020.00045"},{"key":"e_1_3_3_147_2","first-page":"8024","volume-title":"NeurIPS 2019. Curran Associates Inc","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. In NeurIPS 2019. Curran Associates Inc., 8024\u20138035. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/bdbca288fee7f92f2bfa9f7012727740-Abstract.html"},{"key":"e_1_3_3_148_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179324"},{"key":"e_1_3_3_149_2","unstructured":"PyTorch. 2024. PyTorch Issue #4546. Retrieved from https:\/\/github.com\/pytorch\/pytorch\/issues\/4546"},{"key":"e_1_3_3_150_2","unstructured":"PyTorch. 2024. PyTorch Issue#42265. Retrieved from https:\/\/github.com\/pytorch\/pytorch\/issues\/42265\/"},{"key":"e_1_3_3_151_2","unstructured":"Pytorchfi. 2024. Torchfi Github. Retrieved from https:\/\/github.com\/pytorchfi\/pytorchfi"},{"key":"e_1_3_3_152_2","doi-asserted-by":"publisher","DOI":"10.1609\/AAAI.V38I19.30150"},{"key":"e_1_3_3_153_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3560427"},{"key":"e_1_3_3_154_2","doi-asserted-by":"publisher","DOI":"10.1109\/LISAT.2014.6845211"},{"key":"e_1_3_3_155_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00847"},{"key":"e_1_3_3_156_2","doi-asserted-by":"publisher","unstructured":"Abhinav Rao Sachin Vashistha Atharva Naik Somak Aditya and Monojit Choudhury. 2023. Tricking LLMs into disobedience: Understanding analyzing and preventing jailbreaks. arXiv:2305.14965. DOI: 10.48550\/ARXIV.2305.14965","DOI":"10.48550\/ARXIV.2305.14965"},{"key":"e_1_3_3_157_2","unstructured":"Ray. 2020. Fix Bug That test_multi_node.py::test_multi_driver_logging Hangs When GCS Actor Management Is Turned On. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/9539"},{"key":"e_1_3_3_158_2","unstructured":"Ray. 2020. Fix Issue #10319 Dashboard Autoscaler Crash. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/10323"},{"key":"e_1_3_3_159_2","unstructured":"Ray. 2021. Fix Memory Leak in JNI. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/17177"},{"key":"e_1_3_3_160_2","unstructured":"Ray. 2021. Fix Unexpected Error When Handling the Process That Has Exited in Memory Monitor. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/14932"},{"key":"e_1_3_3_161_2","unstructured":"Ray. 2021. [Windows] Correct Symlinks for Files (node.py). Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/16817"},{"key":"e_1_3_3_162_2","unstructured":"Ray. 2022. Fix SciPy Pinning. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/25148"},{"key":"e_1_3_3_163_2","unstructured":"Ray. 2022. Fixed MRO for DerivedActorClass. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/22113"},{"key":"e_1_3_3_164_2","unstructured":"Ray. 2022. Fixing Flaky TestRemovingLeasingPlacementGroup Caused by Race Condition. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/29694"},{"key":"e_1_3_3_165_2","unstructured":"Ray. 2022. [Tune] Catch SyncerCallback Failure with Dead Node. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/29438"},{"key":"e_1_3_3_166_2","unstructured":"Ray. 2022. [Tune] [PBT] Maintain Consistent TrialTrialRunner State When Pausing and Resuming Trial. Retrieved from https:\/\/github.com\/ray-project\/ray\/pull\/28511"},{"key":"e_1_3_3_167_2","unstructured":"Ray-Project. 2024. Ray Github. Retrieved from https:\/\/github.com\/ray-project\/ray"},{"key":"e_1_3_3_168_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN-W58399.2023.00057"},{"key":"e_1_3_3_169_2","doi-asserted-by":"publisher","DOI":"10.1109\/CAHPC.2018.8645906"},{"key":"e_1_3_3_170_2","doi-asserted-by":"publisher","DOI":"10.1145\/1670679.1670680"},{"key":"e_1_3_3_171_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445518"},{"key":"e_1_3_3_172_2","doi-asserted-by":"publisher","DOI":"10.5441\/002\/EDBT.2021.63"},{"key":"e_1_3_3_173_2","unstructured":"Scikit-Learn. 2024. Scikit-Learn Github. Retrieved from https:\/\/github.com\/scikit-learn\/scikit-learn"},{"key":"e_1_3_3_174_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-19-6525-8_16"},{"key":"e_1_3_3_175_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3098426"},{"key":"e_1_3_3_176_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468591"},{"key":"e_1_3_3_177_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C.2018.00032"},{"key":"e_1_3_3_178_2","doi-asserted-by":"publisher","unstructured":"Farhad MortezaPour Shiri Thinagaran Perumal Norwati Mustapha and Raihani Mohamed. 2023. A comprehensive overview and comparative analysis on deep learning models: CNN RNN LSTM GRU. arXiv:2305.17473. DOI: 10.48550\/ARXIV.2305.17473","DOI":"10.48550\/ARXIV.2305.17473"},{"key":"e_1_3_3_179_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME46990.2020.00027"},{"key":"e_1_3_3_180_2","first-page":"9712","article-title":"PopSkipJump: Decision-based attack for probabilistic classifiers","volume":"139","author":"Simon-Gabriel Carl-Johann","year":"2021","unstructured":"Carl-Johann Simon-Gabriel, Noman Ahmed Sheikh, and Andreas Krause. 2021. PopSkipJump: Decision-based attack for probabilistic classifiers. In ICML 2021, Vol. 139. PMLR, 9712\u20139721. Retrieved from http:\/\/proceedings.mlr.press\/v139\/simon-gabriel21a.html","journal-title":"In ICML 2021"},{"key":"e_1_3_3_181_2","doi-asserted-by":"publisher","DOI":"10.1007\/S44163-024-00151-2"},{"key":"e_1_3_3_182_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2019.00010"},{"key":"e_1_3_3_183_2","unstructured":"Spark. 2021. A Memory Leak Occurs When We Clone the Spark Session. Retrieved from https:\/\/issues.apache.org\/jira\/browse\/SPARK-34087"},{"key":"e_1_3_3_184_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN48987.2021.00043"},{"key":"e_1_3_3_185_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST57152.2023.00026"},{"key":"e_1_3_3_186_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10389-6"},{"key":"e_1_3_3_187_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS60937.2023.00011"},{"key":"e_1_3_3_188_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2012.6237031"},{"key":"e_1_3_3_189_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.INFFUS.2021.11.005"},{"key":"e_1_3_3_190_2","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807666"},{"key":"e_1_3_3_191_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2015.7056044"},{"key":"e_1_3_3_192_2","unstructured":"Toxiproxy. 2024. Toxiproxy Github. Retrieved from https:\/\/github.com\/Shopify\/toxiproxy"},{"key":"e_1_3_3_193_2","unstructured":"Trailofbits. 2024. A BPF-Based Syscall Fault Injector. Retrieved from https:\/\/github.com\/trailofbits\/ebpfault"},{"key":"e_1_3_3_194_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN48987.2021.00041"},{"key":"e_1_3_3_195_2","doi-asserted-by":"publisher","DOI":"10.1145\/3568026"},{"key":"e_1_3_3_196_2","doi-asserted-by":"publisher","DOI":"10.1109\/INMIC50486.2020.9318195"},{"key":"e_1_3_3_197_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00024"},{"key":"e_1_3_3_198_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3559561"},{"key":"e_1_3_3_199_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40745-020-00253-5"},{"key":"e_1_3_3_200_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.INS.2022.07.157"},{"key":"e_1_3_3_201_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616271"},{"key":"e_1_3_3_202_2","doi-asserted-by":"publisher","DOI":"10.1007\/S00778-022-00775-9"},{"key":"e_1_3_3_203_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_3_204_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380358"},{"key":"e_1_3_3_205_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00075"},{"key":"e_1_3_3_206_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00129"},{"key":"e_1_3_3_207_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2020.3046075"},{"key":"e_1_3_3_208_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.107004"},{"key":"e_1_3_3_209_2","doi-asserted-by":"publisher","DOI":"10.1109\/4235.771163"},{"key":"e_1_3_3_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616345"},{"key":"e_1_3_3_211_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449905"},{"key":"e_1_3_3_212_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00151"},{"key":"e_1_3_3_213_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616249"},{"key":"e_1_3_3_214_2","unstructured":"Matei Zaharia Mosharaf Chowdhury Michael J. Franklin Scott Shenker and Ion Stoica. 2010. Spark: Cluster computing with working sets. In HotCloud 2010. USENIX Association. Retrieved from https:\/\/www.usenix.org\/conference\/hotcloud-10\/spark-cluster-computing-working-sets"},{"key":"e_1_3_3_215_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM59182.2024.00119"},{"key":"e_1_3_3_216_2","doi-asserted-by":"publisher","DOI":"10.1109\/MDAT.2019.2947271"},{"key":"e_1_3_3_217_2","doi-asserted-by":"publisher","DOI":"10.1109\/MDAT.2019.2915656"},{"key":"e_1_3_3_218_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380362"},{"key":"e_1_3_3_219_2","doi-asserted-by":"publisher","unstructured":"Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen Shuohui Chen Christopher Dewan Mona T. Diab Xian Li Xi Victoria Lin et al. 2022. OPT: Open pre-trained transformer language models. arXiv:2205.01068. DOI: 10.48550\/ARXIV.2205.01068","DOI":"10.48550\/ARXIV.2205.01068"},{"key":"e_1_3_3_220_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"},{"key":"e_1_3_3_221_2","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2019.8890989"},{"key":"e_1_3_3_222_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.FINDINGS-EMNLP.298"},{"key":"e_1_3_3_223_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSREW53611.2021.00068"},{"key":"e_1_3_3_224_2","doi-asserted-by":"publisher","DOI":"10.1145\/3591225"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3732777","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,17]],"date-time":"2026-01-17T07:54:05Z","timestamp":1768636445000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3732777"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,12]]},"references-count":223,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,31]]}},"alternative-id":["10.1145\/3732777"],"URL":"https:\/\/doi.org\/10.1145\/3732777","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,12]]},"assertion":[{"value":"2024-10-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}