{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T17:18:09Z","timestamp":1778606289200,"version":"3.51.4"},"publisher-location":"Cham","reference-count":24,"publisher":"Springer International Publishing","isbn-type":[{"value":"9783030816841","type":"print"},{"value":"9783030816858","type":"electronic"}],"license":[{"start":{"date-parts":[[2021,1,1]],"date-time":"2021-01-01T00:00:00Z","timestamp":1609459200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,15]],"date-time":"2021-07-15T00:00:00Z","timestamp":1626307200000},"content-version":"vor","delay-in-days":195,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We present <jats:sc>NNrepair<\/jats:sc>, a constraint-based technique for repairing neural network classifiers. The technique aims to fix the logic of the network at an <jats:italic>intermediate layer<\/jats:italic> or at the <jats:italic>last layer<\/jats:italic>. <jats:sc>NNrepair<\/jats:sc> first uses <jats:italic>fault localization<\/jats:italic> to find potentially faulty network parameters (such as the <jats:italic>weights<\/jats:italic>) and then performs <jats:italic>repair<\/jats:italic> using <jats:italic>constraint solving<\/jats:italic> to apply small modifications to the parameters to remedy the defects. We present novel strategies to enable precise yet efficient repair such as inferring correctness specifications to act as oracles for intermediate layer repair, and generation of <jats:italic>experts<\/jats:italic> for each class. We demonstrate the technique in the context of three different scenarios: (1) Improving the <jats:italic>overall accuracy<\/jats:italic> of a model, (2) Fixing security vulnerabilities caused by <jats:italic>poisoning<\/jats:italic> of training data and (3) Improving the <jats:italic>robustness<\/jats:italic> of the network against <jats:italic>adversarial<\/jats:italic> attacks. Our evaluation on MNIST and CIFAR-10 models shows that <jats:sc>NNrepair<\/jats:sc> can improve the accuracy by 45.56% points on poisoned data and 10.40% points on adversarial data. <jats:sc>NNrepair<\/jats:sc> also provides small improvement in the overall accuracy of models, without requiring new data or re-training.<\/jats:p>","DOI":"10.1007\/978-3-030-81685-8_1","type":"book-chapter","created":{"date-parts":[[2021,7,17]],"date-time":"2021-07-17T00:02:35Z","timestamp":1626480155000},"page":"3-25","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["NNrepair: Constraint-Based Repair of\u00a0Neural Network Classifiers"],"prefix":"10.1007","author":[{"given":"Muhammad","family":"Usman","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Divya","family":"Gopinath","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Youcheng","family":"Sun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yannic","family":"Noller","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Corina S.","family":"P\u0103s\u0103reanu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,7,15]]},"reference":[{"key":"1_CR1","unstructured":"Ensemble learning methods for deep learning neural networks. https:\/\/machinelearningmastery.com\/ensemble-methods-for-deep-learning-neural-networks"},{"key":"1_CR2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/978-3-030-16722-6_10","volume-title":"Fundamental Approaches to Software Engineering","author":"HF Eniser","year":"2019","unstructured":"Eniser, H.F., Gerasimou, S., Sen, A.: DeepFault: fault localization for deep neural networks. In: H\u00e4hnle, R., van der Aalst, W. (eds.) FASE 2019. LNCS, vol. 11424, pp. 171\u2013191. Springer, Cham (2019). https:\/\/doi.org\/10.1007\/978-3-030-16722-6_10"},{"key":"1_CR3","volume-title":"Deep Learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)"},{"key":"1_CR4","doi-asserted-by":"crossref","unstructured":"Gopinath, D., Converse, H., Pasareanu, C., Taly, A.: Property inference for deep neural networks. In: 34th International Conference on Automated Software Engineering (ASE), pp. 797\u2013809. IEEE (2019)","DOI":"10.1109\/ASE.2019.00079"},{"issue":"12","key":"1_CR5","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/3318162","volume":"62","author":"CL Goues","year":"2019","unstructured":"Goues, C.L., Pradel, M., Roychoudhury, A.: Automated program repair. Commun. ACM 62(12), 56\u201365 (2019). https:\/\/doi.org\/10.1145\/3318162","journal-title":"Commun. ACM"},{"key":"1_CR6","doi-asserted-by":"publisher","first-page":"47230","DOI":"10.1109\/ACCESS.2019.2909068","volume":"7","author":"T Gu","year":"2019","unstructured":"Gu, T., Liu, K., Dolan-Gavitt, B., Garg, S.: BadNets: evaluating backdooring attacks on deep neural networks. IEEE Access 7, 47230\u201347244 (2019). https:\/\/doi.org\/10.1109\/ACCESS.2019.2909068","journal-title":"IEEE Access"},{"key":"1_CR7","doi-asserted-by":"crossref","unstructured":"Huang, L., Joseph, A.D., Nelson, B., Rubinstein, B.I., Tygar, J.D.: Adversarial machine learning. In: 4th Workshop on Security and Artificial Intelligence, pp. 43\u201358. ACM (2011)","DOI":"10.1145\/2046684.2046692"},{"key":"1_CR8","doi-asserted-by":"publisher","first-page":"100270","DOI":"10.1016\/j.cosrev.2020.100270","volume":"37","author":"X Huang","year":"2020","unstructured":"Huang, X., et al.: A survey of safety and trustworthiness of deep neural networks: verification, testing, adversarial attack and defence, and interpretability. Comput..Sci. Rev. 37, 100270 (2020)","journal-title":"Comput..Sci. Rev."},{"key":"1_CR9","doi-asserted-by":"crossref","unstructured":"Islam, M.J., Pan, R., Nguyen, G., Rajan, H.: Repairing deep neural networks: Fix patterns and challenges. In: 42nd International Conference on Software Engineering (ICSE) (2020)","DOI":"10.1145\/3377811.3380378"},{"issue":"6245","key":"1_CR10","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1126\/science.aaa8415","volume":"349","author":"MI Jordan","year":"2015","unstructured":"Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science 349(6245), 255\u2013260 (2015)","journal-title":"Science"},{"key":"1_CR11","doi-asserted-by":"crossref","unstructured":"Liu, Y., et al.: Trojaning attack on neural networks. In: 25th Annual Network and Distributed System Security Symposium (NDSS) (2018)","DOI":"10.14722\/ndss.2018.23291"},{"key":"1_CR12","doi-asserted-by":"crossref","unstructured":"Ma, S., Liu, Y., Lee, W.C., Zhang, X., Grama, A.: MODE: automated neural network model debugging via state differential analysis and input selection. In: 26th Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 175\u2013186. ACM (2018)","DOI":"10.1145\/3236024.3236082"},{"issue":"1","key":"1_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3105906","volume":"51","author":"M Monperrus","year":"2018","unstructured":"Monperrus, M.: Automatic software repair: a bibliography. ACM Comput. Surv. 51(1), 1\u201324 (2018). https:\/\/doi.org\/10.1145\/3105906","journal-title":"ACM Comput. Surv."},{"key":"1_CR14","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/978-3-540-78800-3_24","volume-title":"Tools and Algorithms for the Construction and Analysis of Systems","author":"L de Moura","year":"2008","unstructured":"de Moura, L., Bj\u00f8rner, N.: Z3: an efficient SMT solver. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 337\u2013340. Springer, Heidelberg (2008). https:\/\/doi.org\/10.1007\/978-3-540-78800-3_24"},{"key":"1_CR15","doi-asserted-by":"crossref","unstructured":"Papernot, N., McDaniel, P.D., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: EuroS&P (2016)","DOI":"10.1109\/EuroSP.2016.36"},{"issue":"3","key":"1_CR16","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1007\/s10515-013-0122-2","volume":"20","author":"CS P\u0103s\u0103reanu","year":"2013","unstructured":"P\u0103s\u0103reanu, C.S., Visser, W., Bushnell, D.H., Geldenhuys, J., Mehlitz, P.C., Rungta, N.: Symbolic pathfinder: integrating symbolic execution with model checking for java bytecode analysis. Autom. Softw. Eng. 20(3), 391\u2013425 (2013). https:\/\/doi.org\/10.1007\/s10515-013-0122-2","journal-title":"Autom. Softw. Eng."},{"key":"1_CR17","doi-asserted-by":"crossref","unstructured":"Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. In: ESEC\/SIGSOFT FSE (2005)","DOI":"10.21236\/ADA482657"},{"key":"1_CR18","unstructured":"Sohn, J., Kang, S., Yoo, S.: Search based repair of deep neural networks. arXiv preprint arXiv:1912.12463 (2019)"},{"key":"1_CR19","unstructured":"Sotoudeh, M., Thakur, A.V.: Correcting deep neural networks with small, generalizing patches. In: Workshop on Safety and Robustness in Decision Making (2019)"},{"key":"1_CR20","unstructured":"Szegedy, C., et al.: Intriguing properties of neural networks, Technical report (2013). http:\/\/arxiv.org\/abs\/1312.6199"},{"key":"1_CR21","doi-asserted-by":"publisher","unstructured":"Usman, M., Noller, Y., P\u0103s\u0103reanu, C.S., Sun, Y., Gopinath, D.: Neurospf: a tool for the symbolic analysis of neural networks. In: 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 25\u201328 (2021). https:\/\/doi.org\/10.1109\/ICSE-Companion52605.2021.00027","DOI":"10.1109\/ICSE-Companion52605.2021.00027"},{"key":"1_CR22","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1145\/1735223.1735249","volume":"53","author":"W Weimer","year":"2010","unstructured":"Weimer, W., Forrest, S., Le Goues, C., Nguyen, T.: Automatic program repair with evolutionary computation. Commun. ACM 53, 109\u2013116 (2010). https:\/\/doi.org\/10.1145\/1735223.1735249","journal-title":"Commun. ACM"},{"key":"1_CR23","unstructured":"Wong, E., Kolter, J.Z.: Provable defenses against adversarial examples via the convex outer adversarial polytope. In: 35th International Conference on Machine Learning (ICML), Stockholmsm\u00e4ssan, Stockholm, Sweden, pp. 5283\u20135292. PMLR (2018). http:\/\/proceedings.mlr.press\/v80\/wong18a.html"},{"key":"1_CR24","doi-asserted-by":"crossref","unstructured":"Zhang, H., Chan, W.: Apricot: a weight-adaptation approach to fixing deep learning models. In: 34th International Conference on Automated Software Engineering (ASE), pp. 376\u2013387. IEEE (2019)","DOI":"10.1109\/ASE.2019.00043"}],"container-title":["Lecture Notes in Computer Science","Computer Aided Verification"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-030-81685-8_1","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,17]],"date-time":"2021-07-17T00:02:54Z","timestamp":1626480174000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-030-81685-8_1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021]]},"ISBN":["9783030816841","9783030816858"],"references-count":24,"URL":"https:\/\/doi.org\/10.1007\/978-3-030-81685-8_1","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021]]},"assertion":[{"value":"15 July 2021","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"CAV","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Computer Aided Verification","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2021","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"20 July 2021","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"23 July 2021","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"33","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"cav2021","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"http:\/\/i-cav.org\/2021\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Double-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"EasyChair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"290","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"63","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"22% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"12","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"Yes","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"16 tool papers and 5 invited papers are also included.","order":10,"name":"additional_info_on_review_process","label":"Additional Info on Review Process","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}