{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T21:04:52Z","timestamp":1777496692936,"version":"3.51.4"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"PLDI","license":[{"start":{"date-parts":[[2023,6,6]],"date-time":"2023-06-06T00:00:00Z","timestamp":1686009600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-2048123"],"award-info":[{"award-number":["CCF-2048123"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000015","name":"DOE U.S. Department of Energy","doi-asserted-by":"publisher","award":["DE-SC0022285"],"award-info":[{"award-number":["DE-SC0022285"]}],"id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2023,6,6]]},"abstract":"<jats:p>Deep neural networks (DNNs) are becoming increasingly important components  \nof software, and are considered the state-of-the-art solution for a number  \nof problems, such as image recognition. However, DNNs are far from  \ninfallible, and incorrect behavior of DNNs can have disastrous real-world  \nconsequences. This paper addresses the problem of architecture-preserving  \nV-polytope provable repair of DNNs.  \nA V-polytope defines a convex bounded polytope using its vertex representation.  \nV-polytope provable repair guarantees that the repaired DNN  \nsatisfies the given specification on the infinite set of points in the given V-polytope.  \nAn architecture-preserving repair only modifies the parameters of the DNN, without  \nmodifying its architecture. The repair has the flexibility to  \nmodify multiple layers of the DNN, and runs in polynomial time.  \nIt supports DNNs with activation functions that have some linear pieces,  \nas well as fully-connected, convolutional, pooling and residual layers.  \nTo the best our knowledge, this is the first provable repair approach that  \nhas all of these features.  \nWe implement our approach in a tool called APRNN. Using  \nMNIST, ImageNet, and ACAS Xu DNNs, we show that  \nit has better efficiency, scalability, and generalization  \ncompared to PRDNN and REASSURE, prior provable repair methods that are  \nnot architecture preserving.<\/jats:p>","DOI":"10.1145\/3591238","type":"journal-article","created":{"date-parts":[[2023,6,6]],"date-time":"2023-06-06T20:06:24Z","timestamp":1686081984000},"page":"443-467","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Architecture-Preserving Provable Repair of Deep Neural Networks"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4047-699X","authenticated-orcid":false,"given":"Zhe","family":"Tao","sequence":"first","affiliation":[{"name":"University of California at Davis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-1506-2853","authenticated-orcid":false,"given":"Stephanie","family":"Nawas","sequence":"additional","affiliation":[{"name":"University of California at Davis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8593-2972","authenticated-orcid":false,"given":"Jacqueline","family":"Mitchell","sequence":"additional","affiliation":[{"name":"University of California at Davis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3166-1517","authenticated-orcid":false,"given":"Aditya V.","family":"Thakur","sequence":"additional","affiliation":[{"name":"University of California at Davis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,6,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/262839.262861"},{"key":"e_1_2_1_2_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown , Benjamin Mann , Nick Ryder , Melanie Subbiah , Jared Kaplan , Prafulla Dhariwal , Arvind Neelakantan , Pranav Shyam , Girish Sastry , Amanda Askell , Sandhini Agarwal , Ariel Herbert-Voss , Gretchen Krueger , Tom Henighan , Rewon Child , Aditya Ramesh , Daniel M. Ziegler , Jeffrey Wu , Clemens Winter , Christopher Hesse , Mark Chen , Eric Sigler , Mateusz Litwin , Scott Gray , Benjamin Chess , Jack Clark , Christopher Berner , Sam McCandlish , Alec Radford , Ilya Sutskever , and Dario Amodei . 2020 . Language Models are Few-Shot Learners . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 , NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html"},{"key":"e_1_2_1_3_1","unstructured":"Jia Deng Alex Berg Sanjeev Satheesh Hao Su Aditya Khosla and Fei-Fei Li. 2012. Imagenet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012). https:\/\/www.image-net.org\/challenges\/LSVRC\/2012\/ \t\t\t\t  Jia Deng Alex Berg Sanjeev Satheesh Hao Su Aditya Khosla and Fei-Fei Li. 2012. Imagenet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012). https:\/\/www.image-net.org\/challenges\/LSVRC\/2012\/"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/n19-1423"},{"key":"e_1_2_1_5_1","volume-title":"9th International Conference on Learning Representations, ICLR 2021","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy , Lucas Beyer , Alexander Kolesnikov , Dirk Weissenborn , Xiaohua Zhai , Thomas Unterthiner , Mostafa Dehghani , Matthias Minderer , Georg Heigold , Sylvain Gelly , Jakob Uszkoreit , and Neil Houlsby . 2021 . An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale . In 9th International Conference on Learning Representations, ICLR 2021 , Virtual Event, Austria , May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=YicbFdNTTy Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=YicbFdNTTy"},{"key":"e_1_2_1_6_1","volume-title":"The Tenth International Conference on Learning Representations, ICLR 2022","author":"Ferrari Claudio","year":"2022","unstructured":"Claudio Ferrari , Mark Niklas M\u00fcller , Nikola Jovanovic , and Martin T. Vechev . 2022. Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound . In The Tenth International Conference on Learning Representations, ICLR 2022 , Virtual Event , April 25-29, 2022 . OpenReview.net. https:\/\/openreview.net\/forum?id=l_amHf1oaK Claudio Ferrari, Mark Niklas M\u00fcller, Nikola Jovanovic, and Martin T. Vechev. 2022. Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Bound. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https:\/\/openreview.net\/forum?id=l_amHf1oaK"},{"key":"e_1_2_1_7_1","volume-title":"Sound and Complete Neural Network Repair with Minimality and Locality Guarantees. In The Tenth International Conference on Learning Representations, ICLR 2022","author":"Fu Feisi","year":"2022","unstructured":"Feisi Fu and Wenchao Li . 2022 . Sound and Complete Neural Network Repair with Minimality and Locality Guarantees. In The Tenth International Conference on Learning Representations, ICLR 2022 , Virtual Event , April 25-29, 2022. OpenReview.net. https:\/\/openreview.net\/forum?id=xS8AMYiEav3 Feisi Fu and Wenchao Li. 2022. Sound and Complete Neural Network Repair with Minimality and Locality Guarantees. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net. https:\/\/openreview.net\/forum?id=xS8AMYiEav3"},{"key":"e_1_2_1_8_1","unstructured":"Feisi Fu and Wenchao Li. 2023. REASSURE. https:\/\/github.com\/BU-DEPEND-Lab\/REASSURE\/tree\/4781c8e781913598632da922fc8e41561f85e854\/ICLR \t\t\t\t  Feisi Fu and Wenchao Li. 2023. REASSURE. https:\/\/github.com\/BU-DEPEND-Lab\/REASSURE\/tree\/4781c8e781913598632da922fc8e41561f85e854\/ICLR"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.29007\/699q"},{"key":"e_1_2_1_10_1","volume-title":"Deep Learning","author":"Goodfellow Ian","unstructured":"Ian Goodfellow , Yoshua Bengio , and Aaron Courville . 2016. Deep Learning . MIT Press . http:\/\/www.deeplearningbook.org Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http:\/\/www.deeplearningbook.org"},{"key":"e_1_2_1_11_1","unstructured":"Gurobi Optimization LLC. 2022. Gurobi Optimizer Reference Manual. https:\/\/www.gurobi.com \t\t\t\t  Gurobi Optimization LLC. 2022. Gurobi Optimizer Reference Manual. https:\/\/www.gurobi.com"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_13_1","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Hendrycks Dan","year":"2019","unstructured":"Dan Hendrycks and Thomas G. Dietterich . 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations . In 7th International Conference on Learning Representations, ICLR 2019 , New Orleans, LA, USA , May 6-9, 2019 . OpenReview.net. https:\/\/openreview.net\/forum?id=HJz6tiCqYm Dan Hendrycks and Thomas G. Dietterich. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=HJz6tiCqYm"},{"key":"e_1_2_1_14_1","volume-title":"Natural Adversarial Examples. CoRR, abs\/1907.07174","author":"Hendrycks Dan","year":"2019","unstructured":"Dan Hendrycks , Kevin Zhao , Steven Basart , Jacob Steinhardt , and Dawn Song . 2019. Natural Adversarial Examples. CoRR, abs\/1907.07174 ( 2019 ), arXiv:1907.07174. arxiv:1907.07174 Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. 2019. Natural Adversarial Examples. CoRR, abs\/1907.07174 (2019), arXiv:1907.07174. arxiv:1907.07174"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00140"},{"key":"e_1_2_1_16_1","volume-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 1MB model size. CoRR, abs\/1602.07360","author":"Iandola Forrest N.","year":"2016","unstructured":"Forrest N. Iandola , Matthew W. Moskewicz , Khalid Ashraf , Song Han , William J. Dally , and Kurt Keutzer . 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 1MB model size. CoRR, abs\/1602.07360 ( 2016 ), arXiv:1602.07360. arxiv:1602.07360 Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 1MB model size. CoRR, abs\/1602.07360 (2016), arXiv:1602.07360. arxiv:1602.07360"},{"key":"e_1_2_1_17_1","volume-title":"Owen","author":"Julian Kyle D.","year":"2018","unstructured":"Kyle D. Julian , Mykel J. Kochenderfer , and Michael P . Owen . 2018 . Deep Neural Network Compression for Aircraft Collision Avoidance Systems. CoRR , abs\/1810.04240 (2018), arXiv:1810.04240. arxiv:1810.04240 Kyle D. Julian, Mykel J. Kochenderfer, and Michael P. Owen. 2018. Deep Neural Network Compression for Aircraft Collision Avoidance Systems. CoRR, abs\/1810.04240 (2018), arXiv:1810.04240. arxiv:1810.04240"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18)","author":"Kemker Ronald","year":"2018","unstructured":"Ronald Kemker , Marc McClure , Angelina Abitino , Tyler L. Hayes , and Christopher Kanan . 2018 . Measuring Catastrophic Forgetting in Neural Networks . In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18) , the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA , February 2-7, 2018, Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 3390\u20133398. https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI18\/paper\/view\/16410 Ronald Kemker, Marc McClure, Angelina Abitino, Tyler L. Hayes, and Christopher Kanan. 2018. Measuring Catastrophic Forgetting in Neural Networks. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 3390\u20133398. https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI18\/paper\/view\/16410"},{"key":"e_1_2_1_20_1","unstructured":"Leonid Genrikhovich Khachiyan. 1979. A polynomial algorithm in linear programming. In Doklady Akademii Nauk. 244. \t\t\t\t  Leonid Genrikhovich Khachiyan. 1979. A polynomial algorithm in linear programming. In Doklady Akademii Nauk. 244."},{"key":"e_1_2_1_21_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States, Peter L. Bartlett, Fernando C. N. Pereira, Christopher J. C. Burges, L\u00e9on Bottou, and Kilian Q. Weinberger (Eds .). 1106\u20131114. https:\/\/proceedings.neurips.cc\/paper\/2012\/hash\/c399862d3b9d6b76c8436e924a68c45b-Abstract.html Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012. Proceedings of a meeting held December 3-6, 2012, Lake Tahoe, Nevada, United States, Peter L. Bartlett, Fernando C. N. Pereira, Christopher J. C. Burges, L\u00e9on Bottou, and Kilian Q. Weinberger (Eds.). 1106\u20131114. https:\/\/proceedings.neurips.cc\/paper\/2012\/hash\/c399862d3b9d6b76c8436e924a68c45b-Abstract.html"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.34727\/2021\/isbn.978-3-85448-046-4_27"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 38th International Conference on Machine Learning, ICML 2021","volume":"6222","author":"Leino Klas","year":"2021","unstructured":"Klas Leino , Zifan Wang , and Matt Fredrikson . 2021 . Globally-Robust Neural Networks . In Proceedings of the 38th International Conference on Machine Learning, ICML 2021 , 18-24 July 2021, Virtual Event, Marina Meila and Tong Zhang (Eds.) (Proceedings of Machine Learning Research , Vol. 139). PMLR, 6212\u2013 6222 . http:\/\/proceedings.mlr.press\/v139\/leino21a.html Klas Leino, Zifan Wang, and Matt Fredrikson. 2021. Globally-Robust Neural Networks. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, Marina Meila and Tong Zhang (Eds.) (Proceedings of Machine Learning Research, Vol. 139). PMLR, 6212\u20136222. http:\/\/proceedings.mlr.press\/v139\/leino21a.html"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.34727\/2020\/isbn.978-3-85448-042-6_22"},{"key":"e_1_2_1_25_1","volume-title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs\/1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs\/1907.11692 ( 2019 ), arXiv:1907.11692. arxiv:1907.11692 Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, abs\/1907.11692 (2019), arXiv:1907.11692. arxiv:1907.11692"},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Soenksen LR Kassis T Conover ST Marti-Fuster B Birkenfeld JS Tucker-Schwartz J Naseem A Stavert RR Kim CC Senna MM Avil\u00e9s-Izquierdo J Collins JJ Barzilay R and Gray ML. 2021. Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images.. Science Translational Medicine 13 581 (2021). \t\t\t\t  Soenksen LR Kassis T Conover ST Marti-Fuster B Birkenfeld JS Tucker-Schwartz J Naseem A Stavert RR Kim CC Senna MM Avil\u00e9s-Izquierdo J Collins JJ Barzilay R and Gray ML. 2021. Using deep learning for dermatologist-level detection of suspicious pigmented skin lesions from wide-field images.. Science Translational Medicine 13 581 (2021).","DOI":"10.1126\/scitranslmed.abb3652"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmoneco.2021.07.004"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan","author":"Mirman Matthew","year":"2018","unstructured":"Matthew Mirman , Timon Gehr , and Martin T. Vechev . 2018. Differentiable Abstract Interpretation for Provably Robust Neural Networks . In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan , Stockholm, Sweden , July 10-15, 2018 , Jennifer G. Dy and Andreas Krause (Eds.) (Proceedings of Machine Learning Research, Vol. 80). PMLR, 3575\u20133583. http:\/\/proceedings.mlr.press\/v80\/mirman18b.html Matthew Mirman, Timon Gehr, and Martin T. Vechev. 2018. Differentiable Abstract Interpretation for Provably Robust Neural Networks. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan, Stockholm, Sweden, July 10-15, 2018, Jennifer G. Dy and Andreas Krause (Eds.) (Proceedings of Machine Learning Research, Vol. 80). PMLR, 3575\u20133583. http:\/\/proceedings.mlr.press\/v80\/mirman18b.html"},{"key":"e_1_2_1_29_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=0DcZxeWfOPt","author":"Mitchell Eric","year":"2022","unstructured":"Eric Mitchell , Charles Lin , Antoine Bosselut , Chelsea Finn , and Christopher D Manning . 2022 . Fast Model Editing at Scale . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=0DcZxeWfOPt Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022. Fast Model Editing at Scale. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=0DcZxeWfOPt"},{"key":"e_1_2_1_30_1","volume-title":"International Conference on Machine Learning, ICML 2022","volume":"15831","author":"Mitchell Eric","year":"2022","unstructured":"Eric Mitchell , Charles Lin , Antoine Bosselut , Christopher D. Manning , and Chelsea Finn . 2022 . Memory-Based Model Editing at Scale . In International Conference on Machine Learning, ICML 2022 , 17-23 July 2022, Baltimore, Maryland, USA, Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesv\u00e1ri, Gang Niu, and Sivan Sabato (Eds.) (Proceedings of Machine Learning Research , Vol. 162). PMLR, 15817\u2013 15831 . https:\/\/proceedings.mlr.press\/v162\/mitchell22a.html Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D. Manning, and Chelsea Finn. 2022. Memory-Based Model Editing at Scale. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesv\u00e1ri, Gang Niu, and Sivan Sabato (Eds.) (Proceedings of Machine Learning Research, Vol. 162). PMLR, 15817\u201315831. https:\/\/proceedings.mlr.press\/v162\/mitchell22a.html"},{"key":"e_1_2_1_31_1","volume-title":"MNIST-C: A Robustness Benchmark for Computer Vision. CoRR, abs\/1906.02337","author":"Mu Norman","year":"2019","unstructured":"Norman Mu and Justin Gilmer . 2019. MNIST-C: A Robustness Benchmark for Computer Vision. CoRR, abs\/1906.02337 ( 2019 ), arXiv:1906.02337. arxiv:1906.02337 Norman Mu and Justin Gilmer. 2019. MNIST-C: A Robustness Benchmark for Computer Vision. CoRR, abs\/1906.02337 (2019), arXiv:1906.02337. arxiv:1906.02337"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3498704"},{"key":"e_1_2_1_33_1","volume-title":"Training language models to follow instructions with human feedback. CoRR, abs\/2203.02155","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang , Jeff Wu , Xu Jiang , Diogo Almeida , Carroll L. Wainwright , Pamela Mishkin , Chong Zhang , Sandhini Agarwal , Katarina Slama , Alex Ray , John Schulman , Jacob Hilton , Fraser Kelton , Luke Miller , Maddie Simens , Amanda Askell , Peter Welinder , Paul F. Christiano , Jan Leike , and Ryan Lowe . 2022. Training language models to follow instructions with human feedback. CoRR, abs\/2203.02155 ( 2022 ), https:\/\/doi.org\/10.48550\/arXiv.2203.02155 arXiv:2203.02155. 10.48550\/arXiv.2203.02155 Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. CoRR, abs\/2203.02155 (2022), https:\/\/doi.org\/10.48550\/arXiv.2203.02155 arXiv:2203.02155."},{"key":"e_1_2_1_34_1","volume-title":"Scaling the Convex Barrier with Active Sets. In 9th International Conference on Learning Representations, ICLR 2021","author":"Palma Alessandro De","year":"2021","unstructured":"Alessandro De Palma , Harkirat S. Behl , Rudy Bunel , Philip H. S. Torr , and M. Pawan Kumar . 2021 . Scaling the Convex Barrier with Active Sets. In 9th International Conference on Learning Representations, ICLR 2021 , Virtual Event, Austria , May 3-7, 2021 . OpenReview.net. https:\/\/openreview.net\/forum?id=uQfOy7LrlTR Alessandro De Palma, Harkirat S. Behl, Rudy Bunel, Philip H. S. Torr, and M. Pawan Kumar. 2021. Scaling the Convex Barrier with Active Sets. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=uQfOy7LrlTR"},{"key":"e_1_2_1_35_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas K\u00f6pf , Edward Z. Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019. PyTorch: An Imperative Style , High-Performance Deep Learning Library . 8024\u20138035. https:\/\/proceedings.neurips.cc\/paper\/ 2019 \/hash\/bdbca288fee7f92f2bfa9f7012727740-Abstract.html Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K\u00f6pf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. 8024\u20138035. https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/bdbca288fee7f92f2bfa9f7012727740-Abstract.html"},{"key":"e_1_2_1_36_1","volume-title":"3rd International Conference on Learning Representations, ICLR","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman . 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition . In 3rd International Conference on Learning Representations, ICLR 2015 , San Diego, CA , USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds .). arxiv:1409.1556 Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). arxiv:1409.1556"},{"key":"e_1_2_1_37_1","unstructured":"Gagandeep Singh. 2019. ETH Robustness Analyzer for Neural Networks (ERAN). https:\/\/github.com\/eth-sri\/eran \t\t\t\t  Gagandeep Singh. 2019. ETH Robustness Analyzer for Neural Networks (ERAN). https:\/\/github.com\/eth-sri\/eran"},{"key":"e_1_2_1_38_1","volume-title":"Fast and Effective Robustness Certification. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018","author":"Singh Gagandeep","year":"2018","unstructured":"Gagandeep Singh , Timon Gehr , Matthew Mirman , Markus P\u00fcschel , and Martin T. Vechev . 2018 . Fast and Effective Robustness Certification. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 , NeurIPS 2018 , December 3-8, 2018, Montr\u00e9al, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 10825\u201310836. https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/f2f446980d8e971ef3da97af089481c3-Abstract.html Gagandeep Singh, Timon Gehr, Matthew Mirman, Markus P\u00fcschel, and Martin T. Vechev. 2018. Fast and Effective Robustness Certification. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montr\u00e9al, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 10825\u201310836. https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/f2f446980d8e971ef3da97af089481c3-Abstract.html"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290354"},{"key":"e_1_2_1_40_1","volume-title":"Editable Neural Networks. In 8th International Conference on Learning Representations, ICLR 2020","author":"Sinitsin Anton","year":"2020","unstructured":"Anton Sinitsin , Vsevolod Plokhotnyuk , Dmitry V. Pyrkin , Sergei Popov , and Artem Babenko . 2020 . Editable Neural Networks. In 8th International Conference on Learning Representations, ICLR 2020 , Addis Ababa, Ethiopia , April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=HJedXaEtvS Anton Sinitsin, Vsevolod Plokhotnyuk, Dmitry V. Pyrkin, Sergei Popov, and Artem Babenko. 2020. Editable Neural Networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=HJedXaEtvS"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-023-00695-1"},{"key":"e_1_2_1_42_1","volume-title":"Generalizing Patches. In NeurIPS 2019 Workshop on Safety and Robustness in Decision Making.","author":"Sotoudeh Matthew","unstructured":"Matthew Sotoudeh and Aditya V. Thakur . 2019. Correcting Deep Neural Networks with Small , Generalizing Patches. In NeurIPS 2019 Workshop on Safety and Robustness in Decision Making. Matthew Sotoudeh and Aditya V. Thakur. 2019. Correcting Deep Neural Networks with Small, Generalizing Patches. In NeurIPS 2019 Workshop on Safety and Robustness in Decision Making."},{"key":"e_1_2_1_43_1","volume-title":"Thakur","author":"Sotoudeh Matthew","year":"2021","unstructured":"Matthew Sotoudeh and Aditya V . Thakur . 2021 . PRDNN. https:\/\/github.com\/95616ARG\/PRDNN Matthew Sotoudeh and Aditya V. Thakur. 2021. PRDNN. https:\/\/github.com\/95616ARG\/PRDNN"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454064"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72013-1_15"},{"key":"e_1_2_1_46_1","volume-title":"Katherine Yelick, and David Brown.","author":"Stevens Rick","year":"2020","unstructured":"Rick Stevens , Valerie Taylor , Jeff Nichols , Arthur Barney Maccabe , Katherine Yelick, and David Brown. 2020 . AI for Science. Argonne National Lab.(ANL), Argonne, IL (United States) . Rick Stevens, Valerie Taylor, Jeff Nichols, Arthur Barney Maccabe, Katherine Yelick, and David Brown. 2020. AI for Science. Argonne National Lab.(ANL), Argonne, IL (United States)."},{"key":"e_1_2_1_47_1","volume-title":"Thakur","author":"Tao Zhe","year":"2023","unstructured":"Zhe Tao , Stephanie Nawas , Jacqueline Mitchell , and Aditya V . Thakur . 2023 . Architecture-Preserving Provable Repair of Deep Neural Networks. CoRR , abs\/2304.03496 (2023), arXiv:2304.03496. arxiv:2304.03496 Zhe Tao, Stephanie Nawas, Jacqueline Mitchell, and Aditya V. Thakur. 2023. Architecture-Preserving Provable Repair of Deep Neural Networks. CoRR, abs\/2304.03496 (2023), arXiv:2304.03496. arxiv:2304.03496"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.7807290"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-81685-8_1"},{"key":"e_1_2_1_50_1","volume-title":"Efficient Formal Safety Analysis of Neural Networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018","author":"Wang Shiqi","year":"2018","unstructured":"Shiqi Wang , Kexin Pei , Justin Whitehouse , Junfeng Yang , and Suman Jana . 2018 . Efficient Formal Safety Analysis of Neural Networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 , NeurIPS 2018, December 3-8, 2018, Montr\u00e9al, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 6369\u20136379. https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/2ecd2bd94734e5dd392d8678bc64cdab-Abstract.html Shiqi Wang, Kexin Pei, Justin Whitehouse, Junfeng Yang, and Suman Jana. 2018. Efficient Formal Safety Analysis of Neural Networks. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montr\u00e9al, Canada, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicol\u00f2 Cesa-Bianchi, and Roman Garnett (Eds.). 6369\u20136379. https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/2ecd2bd94734e5dd392d8678bc64cdab-Abstract.html"},{"key":"e_1_2_1_51_1","volume-title":"Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers. In 9th International Conference on Learning Representations, ICLR 2021","author":"Xu Kaidi","year":"2021","unstructured":"Kaidi Xu , Huan Zhang , Shiqi Wang , Yihan Wang , Suman Jana , Xue Lin , and Cho-Jui Hsieh . 2021 . Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers. In 9th International Conference on Learning Representations, ICLR 2021 , Virtual Event, Austria , May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=nVZtXBI6LNn Kaidi Xu, Huan Zhang, Shiqi Wang, Yihan Wang, Suman Jana, Xue Lin, and Cho-Jui Hsieh. 2021. Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifiers. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=nVZtXBI6LNn"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591238","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3591238","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:46Z","timestamp":1750178866000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3591238"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,6]]},"references-count":51,"journal-issue":{"issue":"PLDI","published-print":{"date-parts":[[2023,6,6]]}},"alternative-id":["10.1145\/3591238"],"URL":"https:\/\/doi.org\/10.1145\/3591238","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,6]]},"assertion":[{"value":"2023-06-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}