{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T20:28:45Z","timestamp":1768076925773,"version":"3.49.0"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"abstract":"<jats:p>Deep learning (DL) models have proven to be highly successful and are now essential to our everyday routines. However, DL models, like traditional software, inevitably contain bugs that affect their performance in real-world scenarios. Effective software engineering techniques are necessary to ensure their dependability. In recent years, fault localization methods for DL models have gained significant attention as a valuable tool for improving the reliability of DL models. Owing to the data-driven programming paradigm, traditional fault localization techniques are challenging to apply directly to DL programs. Previous studies have shown that neuron errors within models can lead to abnormal behavior, and they fix the DL model errors from the perspective of neurons. Nonetheless, there remains a significant gap between the DL program statement and model errors.<\/jats:p>\n          <jats:p>\n            To tackle this problem, this paper proposes a novel fault localization method for DL models, named wei\n            <jats:bold>G<\/jats:bold>\n            hted s\n            <jats:bold>U<\/jats:bold>\n            sp\n            <jats:bold>I<\/jats:bold>\n            ciousness an\n            <jats:bold>D<\/jats:bold>\n            balanc\n            <jats:bold>E<\/jats:bold>\n            d agg\n            <jats:bold>R<\/jats:bold>\n            egation (\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\mathsf{GUIDER}\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            ) that revisits the idea and challenge of spectrum-based fault localization in the context of DL models. For pre-trained DL models,\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\mathsf{GUIDER}\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            utilizes neuron coverage information and test case confidence to compute weighted neuron suspiciousness values and employs balanced aggregation methods to elevate these values from the neuron level to the layer level, which establishes a bridge between the DL model and the DL program, facilitating the developers\u2019 debugging process. We evaluate\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\mathsf{GUIDER}\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            using 161 real model bugs collected from StackOverflow and five state-of-the-art fault localization methods for DL models as baselines. The results indicate that (a) our method successfully localizes 67% of the model bugs by ranking the buggy layer to the first place (i.e., top-\n            <jats:inline-formula content-type=\"math\/tex\">\n              <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(1\\)<\/jats:tex-math>\n            <\/jats:inline-formula>\n            ), significantly outperforming all five baselines, and (b) our method maintains an acceptable time overhead compared with all baseline methods.\n          <\/jats:p>","DOI":"10.1145\/3716849","type":"journal-article","created":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T16:03:42Z","timestamp":1739376222000},"update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Weighted Suspiciousness and Balanced Aggregation to Boost Spectrum-based Fault Localization of Deep Learning Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-1884-7700","authenticated-orcid":false,"given":"Wenjie","family":"Xu","sequence":"first","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2282-7175","authenticated-orcid":false,"given":"Yanhui","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4800-6255","authenticated-orcid":false,"given":"Mingliang","family":"Ma","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2352-2226","authenticated-orcid":false,"given":"Lin","family":"Chen","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4645-2526","authenticated-orcid":false,"given":"Yuming","family":"Zhou","sequence":"additional","affiliation":[{"name":"State Key Laboratory for Novel Software Technology, Nanjing University, China"}]}],"member":"320","published-online":{"date-parts":[[2025,2,12]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2007. GitHub - The world\u2019s leading software development platform. https:\/\/github.com"},{"key":"e_1_2_1_2_1","unstructured":"2008. Stack Overflow - Where Developers Learn Share Build Careers. https:\/\/stackoverflow.com"},{"key":"e_1_2_1_3_1","unstructured":"2015. Keras: Deep Learning for humans. https:\/\/keras.io\/"},{"key":"e_1_2_1_4_1","unstructured":"2017. Testng documentation. https:\/\/testng.org\/"},{"key":"e_1_2_1_5_1","unstructured":"2019. Junit. http:\/\/junit.org\/"},{"key":"e_1_2_1_6_1","unstructured":"2020. Why am I getting a constant loss and accuracy? https:\/\/stackoverflow.com\/questions\/63532837\/"},{"key":"e_1_2_1_7_1","unstructured":"2024. Stack Exchange Data Explorer. https:\/\/data.stackexchange.com\/"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.06.035"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2009.25"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351752"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510099"},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3582573","article-title":"QuoTe: Quality-oriented testing for deep learning systems","volume":"32","author":"Chen Jialuo","year":"2023","unstructured":"Jialuo Chen, Jingyi Wang, Xingjun Ma, Youcheng Sun, Jun Sun, Peixin Zhang, and Peng Cheng. 2023. QuoTe: Quality-oriented testing for deep learning systems. ACM Transactions on Software Engineering and Methodology 32, 5 (2023), 1\u201333.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/53990.54003"},{"key":"e_1_2_1_14_1","volume-title":"Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347","author":"de Souza Higor A","year":"2016","unstructured":"Higor A de Souza, Marcos L Chaim, and Fabio Kon. 2016. Spectrum-based software fault localization: A survey of techniques, advances, and challenges. arXiv preprint arXiv:1607.04347 (2016)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-16722-6_10"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2012.6248074"},{"key":"e_1_2_1_17_1","volume-title":"Mutation-based Fault Localization of Deep Neural Networks. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1301\u20131313","author":"Ghanbari Ali","year":"2023","unstructured":"Ali Ghanbari, Deepak-George Thomas, Muhammad Arbab Arshad, and Hridesh Rajan. 2023. Mutation-based Fault Localization of Deep Neural Networks. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1301\u20131313."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.1994.8753425"},{"key":"e_1_2_1_19_1","volume-title":"Measurement of inequality of incomes. The economic journal 31, 121","author":"Gini Corrado","year":"1921","unstructured":"Corrado Gini. 1921. Measurement of inequality of incomes. The economic journal 31, 121 (1921), 124\u2013125."},{"key":"e_1_2_1_20_1","volume-title":"Generative adversarial nets. Advances in neural information processing systems 27","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems 27 (2014)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24797-2"},{"key":"e_1_2_1_22_1","volume-title":"Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. nature 405, 6789","author":"Hahnloser Richard HR","year":"2000","unstructured":"Richard HR Hahnloser, Rahul Sarpeshkar, Misha A Mahowald, Rodney J Douglas, and H Sebastian Seung. 2000. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. nature 405, 6789 (2000), 947\u2013951."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/357172.357173"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2021.3080664"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338955"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380378"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101908.1101949"},{"key":"e_1_2_1_29_1","volume-title":"Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)."},{"key":"e_1_2_1_30_1","volume-title":"Deep learning. nature 521, 7553","author":"LeCun Yann","year":"2015","unstructured":"Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510091"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2022.106954"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236082"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491411.2501854"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3578938"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-023-10291-1"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104322.3104425"},{"key":"e_1_2_1_38_1","volume-title":"Hua Jie Lee, and Kotagiri Ramamohanarao","author":"Naish Lee","year":"2011","unstructured":"Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectra-based software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3 (2011), 1\u201332."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3470006"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.62"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/143062.143098"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.341844"},{"key":"e_1_2_1_44_1","volume-title":"An analysis of ISO 26262: Using machine learning safely in automotive software. arXiv preprint arXiv:1709.02435","author":"Salay Rick","year":"2017","unstructured":"Rick Salay, Rodrigo Queiroz, and Krzysztof Czarnecki. 2017. An analysis of ISO 26262: Using machine learning safely in automotive software. arXiv preprint arXiv:1709.02435 (2017)."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445538"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598045"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556953"},{"key":"e_1_2_1_48_1","volume-title":"Search based repair of deep neural networks. arXiv preprint arXiv:1912.12463","author":"Sohn Jeongju","year":"2019","unstructured":"Jeongju Sohn, Sungmin Kang, and Shin Yoo. 2019. Search based repair of deep neural networks. arXiv preprint arXiv:1912.12463 (2019)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510080"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00128"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-81685-8_1"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00126"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510071"},{"key":"e_1_2_1_54_1","volume-title":"Wei Le, and Hridesh Rajan.","author":"Wardat Mohammad","year":"2023","unstructured":"Mohammad Wardat, Breno Dantas Cruz, Wei Le, and Hridesh Rajan. 2023. An Effective Data-Driven Approach for Localizing Deep Learning Faults. arXiv preprint arXiv:2307.08947 (2023)."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00034"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2948158"},{"key":"e_1_2_1_57_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Euclidean_distance, note = \u201d[Online","year":"2024","unstructured":"Wikipedia. 2024. Euclidean distance \u2014 Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Euclidean_distance, note = \u201d[Online; accessed 09-March-2024]\u201d."},{"key":"e_1_2_1_58_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Feature_scaling, note = \u201d[Online","year":"2024","unstructured":"Wikipedia. 2024. Feature scaling \u2014 Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Feature_scaling, note = \u201d[Online; accessed 09-March-2024]\u201d."},{"key":"e_1_2_1_59_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Gini_coefficient, note = \u201d[Online","year":"2024","unstructured":"Wikipedia. 2024. Gini coefficient \u2014 Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/wiki\/Gini_coefficient, note = \u201d[Online; accessed 09-March-2024]\u201d."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2013.2285319"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2521368"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409671"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126912"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2599161"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00043"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092731"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134324"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3716849","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T16:04:09Z","timestamp":1739376249000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3716849"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,12]]},"references-count":69,"alternative-id":["10.1145\/3716849"],"URL":"https:\/\/doi.org\/10.1145\/3716849","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,12]]},"assertion":[{"value":"2024-07-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"3716849"}}