{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T15:26:56Z","timestamp":1784302016868,"version":"3.55.0"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T00:00:00Z","timestamp":1749513600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T00:00:00Z","timestamp":1749513600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["787703"],"award-info":[{"award-number":["787703"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["NRF-2018R1A5A1059921"],"award-info":[{"award-number":["NRF-2018R1A5A1059921"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"publisher","award":["EP\/V026801\/2"],"award-info":[{"award-number":["EP\/V026801\/2"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["NRF-2020R1A2C1013629"],"award-info":[{"award-number":["NRF-2020R1A2C1013629"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["No.2021-0-01001"],"award-info":[{"award-number":["No.2021-0-01001"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004358","name":"Samsung","doi-asserted-by":"publisher","award":["No. IO201210-07969-01"],"award-info":[{"award-number":["No. IO201210-07969-01"]}],"id":[{"id":"10.13039\/100004358","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>With the increased popularity of Deep Neural Networks (DNNs), increases also the need for tools to assist developers in the DNN implementation, testing and debugging process. Several approaches have been proposed that automatically analyse and localise potential faults in DNNs under test. In this work, we evaluate and compare existing state-of-the-art fault localisation techniques, which operate based on both dynamic and static analysis of the DNN. The evaluation is performed on a benchmark consisting of both real faults obtained from bug reporting platforms and faulty models produced by a mutation tool. Our findings indicate that the usage of a single, specific ground truth (e.g. the human-defined one) for the evaluation of DNN fault localisation tools results in pretty low performance (maximum average recall of 0.33 and precision of 0.21). However, such figures increase when considering alternative, equivalent patches that exist for a given faulty DNN. The results indicate that <jats:sc>DeepFD<\/jats:sc> is the most effective tool, achieving an average recall of 0.55 and a precision of 0.37 on our benchmark.<\/jats:p>","DOI":"10.1007\/s10664-025-10657-7","type":"journal-article","created":{"date-parts":[[2025,6,10]],"date-time":"2025-06-10T09:43:15Z","timestamp":1749548595000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["An empirical study of fault localisation techniques for deep neural networks"],"prefix":"10.1007","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3037-8368","authenticated-orcid":false,"given":"Nargiz","family":"Humbatova","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinhan","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gunel","family":"Jahangirova","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shin","family":"Yoo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paolo","family":"Tonella","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,6,10]]},"reference":[{"key":"10657_CR1","unstructured":"Abadi, M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Man\u00e9 D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Vi\u00e9gas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Available at https:\/\/www.tensorflow.org"},{"key":"10657_CR2","doi-asserted-by":"crossref","unstructured":"Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175\u2013185. [Online]. Available http:\/\/www.jstor.org\/stable\/2685209","DOI":"10.1080\/00031305.1992.10475879"},{"key":"10657_CR3","doi-asserted-by":"crossref","unstructured":"Baker W,\u00a0O\u2019Connor M, Shahamiri SR,\u00a0Terragni V (2022) Detect, fix, and verify tensorflow api misuses. In: International conference on software analysis, evolution and reengineering, pp 1\u20135","DOI":"10.1109\/SANER53432.2022.00110"},{"key":"10657_CR4","doi-asserted-by":"crossref","unstructured":"Breiman L (2017) Classification and regression trees. Routledge","DOI":"10.1201\/9781315139470"},{"key":"10657_CR5","doi-asserted-by":"crossref","unstructured":"Cao J,\u00a0Li M,\u00a0Chen X,\u00a0Wen M,\u00a0Tian Y,\u00a0Wu B, Cheung S-C (2021) Replication package of deepfd. https:\/\/github.com\/ArabelaTso\/DeepFD","DOI":"10.1145\/3510003.3510099"},{"key":"10657_CR6","doi-asserted-by":"publisher","unstructured":"Cao J,\u00a0Li M,\u00a0Chen X,\u00a0Wen M,\u00a0Tian Y,\u00a0Wu B, Cheung S-C (2022) Deepfd: automated fault diagnosis and localization for deep learning programs.\u201d In: Proceedings of the 44th international conference on software engineering, ser. ICSE \u201922. New York, USA: Association for Computing Machinery, pp 573\u2013585. [Online]. Available https:\/\/doi.org\/10.1145\/3510003.3510099","DOI":"10.1145\/3510003.3510099"},{"key":"10657_CR7","unstructured":"Chollet F, et al (2015) Keras. Available at https:\/\/keras.io"},{"key":"10657_CR8","doi-asserted-by":"publisher","first-page":"1269","DOI":"10.1613\/jair.1.13643","volume":"74","author":"AI Cowen-Rivers","year":"2022","unstructured":"Cowen-Rivers AI, Lyu W, Tutunov R, Wang Z, Grosnit A, Griffiths RR, Maraval AM, Jianye H, Wang J, Peters J et al (2022) Hebo: pushing the limits of sample-efficient hyper-parameter optimisation. J Artif Intell Res 74:1269\u20131349","journal-title":"J Artif Intell Res"},{"key":"10657_CR9","unstructured":"DeepLearning.ai (2024) Five important AI programming languages. Available at https:\/\/www.deeplearning.ai\/blog\/five-important-ai-programming-languages\/"},{"key":"10657_CR10","unstructured":"Falkner S,\u00a0Klein A,\u00a0Hutter F (2018) Bohb: robust and efficient hyperparameter optimization at scale. In: International conference on machine learning. PMLR, pp 1437\u20131446"},{"key":"10657_CR11","unstructured":"Hale J (2018) Deep learning framework power scores. Available at https:\/\/towardsdatascience.com\/deep-learning-framework-power-scores-2018-23607ddf297a"},{"key":"10657_CR12","doi-asserted-by":"crossref","unstructured":"Henriksen P,\u00a0Leofante F,\u00a0Lomuscio A (2022) Repairing misclassifications in neural networks using limited data. In: Proceedings of the 37th ACM\/SIGAPP symposium on applied computing, pp 1031\u20131038","DOI":"10.1145\/3477314.3507059"},{"key":"10657_CR13","doi-asserted-by":"crossref","unstructured":"Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, vol\u00a01. IEEE, pp 278\u2013282","DOI":"10.1109\/ICDAR.1995.598994"},{"key":"10657_CR14","doi-asserted-by":"crossref","unstructured":"Humbatova N,\u00a0Jahangirova G,\u00a0Bavota G,\u00a0Riccio V,\u00a0Stocco A,\u00a0Tonella P (2020) Taxonomy of real faults in deep learning systems. In: Proceedings of the 41st international conference on software engineering, ICSE","DOI":"10.1145\/3377811.3380395"},{"key":"10657_CR15","unstructured":"Humbatova N,\u00a0Jahangirova G,\u00a0Tonella P (2020) DeepCrime: mutation Testing tool based on real faults. Available at https:\/\/github.com\/dlfaults\/deepcrime"},{"key":"10657_CR16","doi-asserted-by":"publisher","unstructured":"Humbatova N,\u00a0Jahangirova G,\u00a0Tonella P (2021) Deepcrime: Mutation testing of deep learning systems based on real faults. In: Proceedings of the 30th ACM SIGSOFT international symposium on software testing and analysis, ser. ISSTA 2021. New York, USA: Association for Computing Machinery, pp 67\u201378. [Online]. Available https:\/\/doi.org\/10.1145\/3460319.3464825","DOI":"10.1145\/3460319.3464825"},{"key":"10657_CR17","unstructured":"Humbatova N, Kim J, Jahangirova G, Yoo S, Tonella P (2023) Empirical comparison of fault localisation techniques for DNNs (replication package). Available at https:\/\/zenodo.org\/doi\/10.5281\/zenodo.10387015"},{"key":"10657_CR18","unstructured":"Jahangirova G,\u00a0Humbatova N,\u00a0Kim J,\u00a0Yoo S,\u00a0Tonella P (2024) Real faults in deep learning fault benchmarks: how real are they? arXiv:2412.16336"},{"key":"10657_CR19","doi-asserted-by":"publisher","unstructured":"Jahangirova G,\u00a0Tonella P (2020) An empirical evaluation of mutation operators for deep learning systems. In: IEEE International Conference on software testing, verification and validation, ser. ICST\u201920. 1em plus IEEE, p 12. [Online]. Available https:\/\/doi.org\/10.1109\/ICST46399.2020.00018","DOI":"10.1109\/ICST46399.2020.00018"},{"key":"10657_CR20","doi-asserted-by":"crossref","unstructured":"Kim J,\u00a0Humbatova N,\u00a0Jahangirova G,\u00a0Tonella P,\u00a0Yoo S (2023) Repairing dnn architecture: are we there yet? In: 2023 IEEE Conference on Software Testing, Verification and Validation (ICST)","DOI":"10.1109\/ICST57152.2023.00030"},{"key":"10657_CR21","unstructured":"LeCun Y, Bottou L, Bengio Y, Haffner P (2020) Lenet architecture: a complete guide. Available at https:\/\/www.kaggle.com\/code\/blurredmachine\/lenet-architecture-a-complete-guide"},{"issue":"3","key":"10657_CR22","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1007\/s10664-023-10291-1","volume":"28","author":"MM Morovati","year":"2023","unstructured":"Morovati MM, Nikanjam A, Khomh F, Jiang ZM (2023) Bugs in machine learning-based systems: a faultload benchmark. Empir Softw Eng 28(3):62","journal-title":"Empir Softw Eng"},{"issue":"1","key":"10657_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3470006","volume":"31","author":"A Nikanjam","year":"2021","unstructured":"Nikanjam A, Braiek HB, Morovati MM, Khomh F (2021) Automatic fault detection for deep learning programs using graph transformations. ACM Trans Softw Eng Methodol (TOSEM) 31(1):1\u201327","journal-title":"ACM Trans Softw Eng Methodol (TOSEM)"},{"key":"10657_CR24","unstructured":"Nikanjam A, Braiek HB, Morovati MM,\u00a0Khomh F (2021) Replication package of Neuralint. Available at https:\/\/github.com\/neuralint\/neuralint"},{"key":"10657_CR25","doi-asserted-by":"publisher","unstructured":"Renzullo J,\u00a0Weimer W,\u00a0Moses M,\u00a0Forrest S (2018) Neutrality and epistasis in program space. In: Proceedings of the 4th international workshop on genetic improvement workshop, ser. GI \u201918. New York, USA: Association for Computing Machinery, pp 1\u20138. [Online]. Available https:\/\/doi.org\/10.1145\/3194810.3194812","DOI":"10.1145\/3194810.3194812"},{"key":"10657_CR26","unstructured":"Schoop E,\u00a0Huang F,\u00a0Hartmann B (2021) Replication package of umlaut. Available at https:\/\/github.com\/BerkeleyHCI\/umlaut"},{"key":"10657_CR27","doi-asserted-by":"crossref","unstructured":"Schoop E,\u00a0Huang F,\u00a0Hartmann B (2021) Umlaut: debugging deep learning programs using program structure and model behavior. In: Proceedings of the 2021 CHI conference on human factors in computing systems, pp 1\u201316","DOI":"10.1145\/3411764.3445538"},{"key":"10657_CR28","doi-asserted-by":"publisher","unstructured":"Sohn J,\u00a0Kang S,\u00a0Yoo S (2022) Arachne: search based repair of deep neural networks. ACM Trans Softw Eng Methodol. [Online]. Available https:\/\/doi.org\/10.1145\/3563210","DOI":"10.1145\/3563210"},{"key":"10657_CR29","doi-asserted-by":"crossref","unstructured":"Sotoudeh M, Thakur AV (2021) Provable repair of deep neural networks. In: Proceedings of the 42nd ACM SIGPLAN international conference on programming language design and implementation, pp 588\u2013603","DOI":"10.1145\/3453483.3454064"},{"key":"10657_CR30","doi-asserted-by":"crossref","unstructured":"Sun B,\u00a0Sun J, Pham LH,\u00a0Shi J (2022) Causality-based neural network repair. In: Proceedings of the 44th international conference on software engineering, pp 338\u2013349","DOI":"10.1145\/3510003.3510080"},{"key":"10657_CR31","doi-asserted-by":"crossref","unstructured":"Usman M,\u00a0Gopinath D,\u00a0Sun Y,\u00a0Noller Y, P\u0103s\u0103reanu CS (2021) Nn repair: constraint-based repair of neural network classifiers. In: International conference on computer aided verification. Springer, pp 3\u201325","DOI":"10.1007\/978-3-030-81685-8_1"},{"key":"10657_CR32","doi-asserted-by":"crossref","unstructured":"Wardat M, Cruz BD,\u00a0Le W,\u00a0Rajan H (2021) Replication package of DeepDiagnosis. Available at https:\/\/github.com\/deepdiagnosis\/icse2022","DOI":"10.1145\/3510003.3510071"},{"key":"10657_CR33","doi-asserted-by":"crossref","unstructured":"Wardat M, Cruz BD,\u00a0Le W,\u00a0Rajan H (2022) DeepDiagnosis: automatically diagnosing faults and recommending actionable fixes in deep learning programs. In: Proceedings of the 44th international conference on software engineering, pp 561\u2013572","DOI":"10.1145\/3510003.3510071"},{"key":"10657_CR34","doi-asserted-by":"crossref","unstructured":"Wardat M,\u00a0Le W,\u00a0Rajan H (2021) Deeplocalize: fault localization for deep neural networks. In: 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE). Los Alamitos, CA, USA: IEEE Computer Society, pp 251\u2013262. [Online]. Available: https:\/\/doi.ieeecomputersociety.org\/10.1109\/ICSE43902.2021.00034","DOI":"10.1109\/ICSE43902.2021.00034"},{"key":"10657_CR35","doi-asserted-by":"publisher","first-page":"107027","DOI":"10.1016\/j.infsof.2022.107027","volume":"151","author":"D Wu","year":"2022","unstructured":"Wu D, Shen B, Chen Y, Jiang H, Qiao L (2022) Automatically repairing tensor shape faults in deep learning programs. Inf Softw Technol 151:107027","journal-title":"Inf Softw Technol"},{"key":"10657_CR36","doi-asserted-by":"crossref","unstructured":"Wu H,\u00a0Li Z,\u00a0Cui Z,\u00a0Liu J (2022) Genmunn: a mutation-based approach to repair deep neural network models. Int J Model Simul Sci Comput p 2341008","DOI":"10.1142\/S1793962323410088"},{"key":"10657_CR37","doi-asserted-by":"crossref","unstructured":"Wu D, Shen B, Chen Y, Jiang H, Qiao L (2021) Tensfa: detecting and repairing tensor shape faults in deep learning systems. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, pp 11\u201321","DOI":"10.1109\/ISSRE52982.2021.00014"},{"key":"10657_CR38","doi-asserted-by":"crossref","unstructured":"Zhang H,\u00a0Chan W (2019) Apricot: a weight-adaptation approach to fixing deep learning models. In: 2019 34th IEEE\/ACM International conference on Automated Software Engineering (ASE), pp 376\u2013387","DOI":"10.1109\/ASE.2019.00043"},{"key":"10657_CR39","doi-asserted-by":"crossref","unstructured":"Zhang Y,\u00a0Chen Y, Cheung S-C,\u00a0Xiong Y,\u00a0Zhang L (2018) An empirical study on tensorflow program bugs. In: Proceedings of the 27th ACM SIGSOFT international symposium on software testing and analysis, ser. ISSTA 2018. New York, USA: ACM, pp 129\u2013140. [Online]. Available http:\/\/doi.acm.org\/10.1145\/3213846.3213866","DOI":"10.1145\/3213846.3213866"},{"key":"10657_CR40","doi-asserted-by":"crossref","unstructured":"Zhang X,\u00a0Zhai J,\u00a0Ma S,\u00a0Shen C (2021) Autotrainer: an automatic dnn training problem detection and repair system. In: 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), pp 359\u2013371","DOI":"10.1109\/ICSE43902.2021.00043"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10657-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-025-10657-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-025-10657-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,13]],"date-time":"2025-09-13T08:55:40Z","timestamp":1757753740000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-025-10657-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,10]]},"references-count":40,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["10657"],"URL":"https:\/\/doi.org\/10.1007\/s10664-025-10657-7","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,10]]},"assertion":[{"value":"3 April 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 June 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"124"}}