{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:12:32Z","timestamp":1781367152750,"version":"3.54.1"},"reference-count":49,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T00:00:00Z","timestamp":1691971200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T00:00:00Z","timestamp":1691971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001782","name":"University of Melbourne","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001782","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Vis"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>It has been shown that neural networks are vulnerable to adversarial attacks: adding well-crafted, imperceptible perturbations to their input can modify their output. Adversarial training is one of the most effective approaches to training robust models against such attacks. Unfortunately, this method is much slower than vanilla training of neural networks since it needs to construct adversarial examples for the entire training data at every iteration. By leveraging the theory of coreset selection, we show how selecting a small subset of training data provides a principled approach to reducing the time complexity of robust training. To this end, we first provide convergence guarantees for adversarial coreset selection. In particular, we show that the convergence bound is directly related to how well our coresets can approximate the gradient computed over the entire training data. Motivated by our theoretical analysis, we propose using this gradient approximation error as our adversarial coreset selection objective to reduce the training set size effectively. Once built, we run adversarial training over this subset of the training data. Unlike existing methods, our approach can be adapted to a wide variety of training objectives, including TRADES, <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\ell _p$$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:msub>\n                    <mml:mi>\u2113<\/mml:mi>\n                    <mml:mi>p<\/mml:mi>\n                  <\/mml:msub>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-PGD, and Perceptual Adversarial Training. We conduct extensive experiments to demonstrate that our approach speeds up adversarial training by 2\u20133 times while experiencing a slight degradation in the clean and robust accuracy.<\/jats:p>","DOI":"10.1007\/s11263-023-01860-4","type":"journal-article","created":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T13:02:10Z","timestamp":1692018130000},"page":"3307-3331","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Adversarial Coreset Selection for Efficient Robust Training"],"prefix":"10.1007","volume":"131","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9418-1487","authenticated-orcid":false,"given":"Hadi M.","family":"Dolatabadi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0885-0643","authenticated-orcid":false,"given":"Sarah M.","family":"Erfani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4388-0517","authenticated-orcid":false,"given":"Christopher","family":"Leckie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,8,14]]},"reference":[{"issue":"1","key":"1860_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-021-00419-9","volume":"8","author":"A Adadi","year":"2021","unstructured":"Adadi, A. (2021). A survey on data-efficient algorithms in big data era. Journal of Big Data, 8(1), 1\u201354.","journal-title":"Journal of Big Data"},{"key":"1860_CR2","unstructured":"Andriushchenko, M., & Flammarion, N. (2020). Understanding and improving fast adversarial training. In Proceedings of the advances in neural information processing systems 33: Annual conference on neural information processing systems."},{"key":"1860_CR3","doi-asserted-by":"crossref","unstructured":"Biggio, B., Corona, I., & Maiorca, D., et\u00a0al. (2013). Evasion attacks against machine learning at test time. In Proceedings of the European conference on machine learning and knowledge discovery in databases (ECML-PKDD), pp. 387\u2013402.","DOI":"10.1007\/978-3-642-40994-3_25"},{"key":"1860_CR4","unstructured":"Campbell, T., & Broderick, T. (2018). Bayesian coreset construction via greedy iterative geodesic ascent. In Proceedings of the 35th international conference on machine learning (ICML), pp. 697\u2013705."},{"key":"1860_CR5","unstructured":"Croce, F., & Hein, M. (2020). Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the 37th international conference on machine learning (ICML) 2020, pp. 2206\u20132216."},{"key":"1860_CR6","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-46092-0","volume-title":"The theory of max-min and its application to weapons allocation problems","author":"JM Danskin","year":"1967","unstructured":"Danskin, J. M. (1967). The theory of max-min and its application to weapons allocation problems (Vol. 5). Springer Science & Business Media."},{"key":"1860_CR7","unstructured":"Elenberg, E. R., Khanna, R., & Dimakis, A. G., et al. (2016). Restricted strong convexity implies weak submodularity. CoRR abs\/1612.00804, arXiv:1612.00804."},{"key":"1860_CR8","doi-asserted-by":"crossref","unstructured":"Eykholt, K., Evtimov, I., & Fernandes, E., et\u00a0al. (2018). Robust physical-world attacks on deep learning visual classification. In Proceeding of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 1625\u20131634.","DOI":"10.1109\/CVPR.2018.00175"},{"key":"1860_CR9","unstructured":"Feldman, D. (2020). Introduction to core-sets: An updated survey. CoRR abs\/2011.09384. arXiv:2011.09384."},{"key":"1860_CR10","unstructured":"Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In Proceedings of the 3rd international conference on learning representations (ICLR)."},{"key":"1860_CR11","doi-asserted-by":"crossref","unstructured":"Har-Peled, S., & Mazumdar, S. (2004). On coresets for k-means and k-median clustering. In Proceedings of the 36th annual ACM symposium on theory of computing (STOC), pp. 291\u2013300.","DOI":"10.1145\/1007352.1007400"},{"key":"1860_CR12","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., & Ren, S., et\u00a0al. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 770\u2013778.","DOI":"10.1109\/CVPR.2016.90"},{"key":"1860_CR13","unstructured":"de\u00a0Jorge\u00a0Aranda, P., Bibi, A., & Volpi, R., et\u00a0al. (2022). Make some noise: Reliable and efficient single-step adversarial training. In Proceedings of the advances in neural information processing systems 35: Annual conference on neural information processing systems (NeurIPS)."},{"key":"1860_CR14","unstructured":"Kang, D., Sun, Y., & Hendrycks, D., et\u00a0al. (2019). Testing robustness against unforeseen adversaries. CoRR abs\/1908.08016, arXiv:1908.08016."},{"key":"1860_CR15","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., & Aittala, M., et\u00a0al. (2020). Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 8107\u20138116.","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"1860_CR16","unstructured":"Katharopoulos, A., & Fleuret, F. (2018). Not all samples are created equal: Deep learning with importance sampling. In Proceedings of the 35th international conference on machine learning (ICML), pp 2530\u20132539."},{"key":"1860_CR17","unstructured":"Killamsetty, K., Sivasubramanian, D., & Ramakrishnan, G., et\u00a0al. (2021a). GRAD-MATCH: gradient matching based data subset selection for efficient deep model training. In Proceedings of the 38th international conference on machine learning (ICML), pp. 5464\u20135474."},{"key":"1860_CR18","doi-asserted-by":"crossref","unstructured":"Killamsetty, K., Sivasubramanian, D., & Ramakrishnan, G., et\u00a0al. (2021b). GLISTER: Generalization based data subset selection for efficient and robust learning. In Proceedings of the 35th AAAI conference on artificial intelligence, pp. 8110\u20138118.","DOI":"10.1609\/aaai.v35i9.16988"},{"key":"1860_CR19","unstructured":"Killamsetty, K., Zhao, X., & Chen, F., et\u00a0al. (2021c). RETRIEVE: Coreset selection for efficient and robust semi-supervised learning. In Advances in neural information processing systems 34: Annual conference on neural information processing systems (NeurIPS), pp. 14488\u201314501."},{"key":"1860_CR20","unstructured":"Kolter, Z., & Madry, A. (2018). Adversarial robustness: Theory and practice. In Tutorial in the advances in neural information processing systems 31: Annual conference on neural information processing systems (NeurIPS). https:\/\/adversarial-ml-tutorial.org\/."},{"key":"1860_CR21","unstructured":"Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images. Master\u2019s thesis, Department of Computer Science, University of Toronto."},{"key":"1860_CR22","unstructured":"Laidlaw, C., & Feizi, S. (2019). Functional adversarial attacks. In Proceedings of the advances in neural information processing systems 32: Annual conference on neural information processing systems (NeurIPS), pp. 10408\u201310418."},{"key":"1860_CR23","unstructured":"Laidlaw, C., Singla, S., & Feizi, S. (2021). Perceptual adversarial robustness: Defense against unseen threat models. In Proceedings of the 9th international conference on learning representations (ICLR)."},{"key":"1860_CR24","doi-asserted-by":"crossref","unstructured":"Liu, Y., Ma, X., & Bailey, J., et\u00a0al. (2020). Reflection backdoor: A natural backdoor attack on deep neural networks. In Proceedings of the 16th European conference on computer vision (ECCV), pp. 182\u2013199.","DOI":"10.1007\/978-3-030-58607-2_11"},{"issue":"107","key":"1860_CR25","first-page":"332","volume":"110","author":"X Ma","year":"2021","unstructured":"Ma, X., Niu, Y., Gu, L., et al. (2021). Understanding adversarial attacks on deep learning based medical image analysis systems. Pattern Recognition, 110(107), 332.","journal-title":"Pattern Recognition"},{"key":"1860_CR26","unstructured":"Madry, A., Makelov, A., & Schmidt, L., et\u00a0al. (2018). Towards deep learning models resistant to adversarial attacks. In Proceedings of the 6th international conference on learning representations (ICLR)."},{"key":"1860_CR27","doi-asserted-by":"crossref","unstructured":"Minoux, M. (1978). Accelerated greedy algorithms for maximizing submodular set functions. In Optimization techniques (pp. 234\u2013243). Springer.","DOI":"10.1007\/BFb0006528"},{"key":"1860_CR28","unstructured":"Mirzasoleiman, B., Bilmes, J. A., & Leskovec, J. (2020a). Coresets for data-efficient training of machine learning models. In Proceedings of the 37th international conference on machine learning (ICML), pp. 6950\u20136960."},{"key":"1860_CR29","unstructured":"Mirzasoleiman, B., Cao, K., & Leskovec, J. (2020b). Coresets for robust training of deep neural networks against noisy labels. In Proceedings of the advances in neural information processing systems 33: annual conference on neural information processing systems (NeurIPS)."},{"issue":"1","key":"1860_CR30","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/BF01588971","volume":"14","author":"GL Nemhauser","year":"1978","unstructured":"Nemhauser, G. L., Wolsey, L. A., & Fisher, M. L. (1978). An analysis of approximations for maximizing submodular set functions: I. Mathematical Programming, 14(1), 265\u2013294.","journal-title":"Mathematical Programming"},{"key":"1860_CR31","unstructured":"Netzer, Y., Wang, T., & Coates, A., et\u00a0al. (2011). Reading digits in natural images with unsupervised feature learning. In NeurIPS workshop on deep learning and unsupervised feature learning."},{"key":"1860_CR32","doi-asserted-by":"crossref","unstructured":"Pati, Y. C., Rezaiifar, R., & Krishnaprasad, P. S. (1993). Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th asilomar conference on signals, systems and computers, pp. 40\u201344.","DOI":"10.1109\/ACSSC.1993.342465"},{"key":"1860_CR33","unstructured":"Qin, C., Martens, J., & Gowal, S., et\u00a0al. (2019). Adversarial robustness through local linearization. In Proceedings of the advances in neural information processing systems 32: Annual conference on neural information processing systems (NeurIPS)."},{"issue":"3","key":"1860_CR34","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky, O., Deng, J., Su, H., et al. (2015). ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211\u2013252.","journal-title":"International Journal of Computer Vision"},{"key":"1860_CR35","doi-asserted-by":"crossref","unstructured":"Schwartz, R., Dodge, J., Smith, N. A., et al. (2020). Green AI. Communication of the ACM, 63(12), 54\u201363.","DOI":"10.1145\/3381831"},{"key":"1860_CR36","doi-asserted-by":"crossref","unstructured":"Smith, L. N. (2017). Cyclical learning rates for training neural networks. In Proceedings of the IEEE winter conference on applications of computer vision (WACV), pp. 464\u2013472","DOI":"10.1109\/WACV.2017.58"},{"key":"1860_CR37","doi-asserted-by":"crossref","unstructured":"Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th conference of the association for computational linguistics ACL, pp. 3645\u20133650.","DOI":"10.18653\/v1\/P19-1355"},{"key":"1860_CR38","unstructured":"Szegedy, C., Zaremba, W., & Sutskever, I., et\u00a0al. (2014). Intriguing properties of neural networks. In Proceedings of the 2nd international conference on learning representations (ICLR)."},{"key":"1860_CR39","unstructured":"Tram\u00e8r, F., Kurakin, A., & Papernot, N., et\u00a0al. (2018). Ensemble adversarial training: Attacks and defenses. In Proceedings of the 6th international conference on learning representations (ICLR)."},{"key":"1860_CR40","unstructured":"Tsipras, D., Santurkar, S., & Engstrom, L., et\u00a0al. (2019). Robustness may be at odds with accuracy. In Proceedings of the 7th international conference on learning representations (ICLR)."},{"key":"1860_CR41","unstructured":"Vahdat, A., & Kautz, J. (2020). NVAE: A deep hierarchical variational autoencoder. In Proceedings of the advances in neural information processing systems 33: Annual conference on neural information processing systems (NeurIPS)."},{"key":"1860_CR42","unstructured":"Wei, K., Iyer, R., & Bilmes, J. (2015). Submodularity in data subset selection and active learning. In Proceedings of the 32nd international conference on machine learning (ICML), pp. 1954\u20131963."},{"issue":"4","key":"1860_CR43","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1007\/BF02579435","volume":"2","author":"LA Wolsey","year":"1982","unstructured":"Wolsey, L. A. (1982). An analysis of the greedy algorithm for the submodular set covering problem. Combinatorica, 2(4), 385\u2013393.","journal-title":"Combinatorica"},{"key":"1860_CR44","unstructured":"Wong, E., Rice, L., & Kolter, J. Z. (2020). Fast is better than free: Revisiting adversarial training. In Proceedings of the 8th international conference on learning representations (ICLR)."},{"key":"1860_CR45","unstructured":"Wu, Y., Kirillov, A., & Massa, F., et\u00a0al. (2019). Detectron2. https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"1860_CR46","unstructured":"Xiao, C., Zhu, J., & Li, B., et\u00a0al. (2018). Spatially transformed adversarial examples. In Proceedings of the 6th international conference on learning representations (ICLR)."},{"key":"1860_CR47","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., & Komodakis, N. (2016). Wide residual networks. In Proceedings of the British machine vision conference (BMVC).","DOI":"10.5244\/C.30.87"},{"key":"1860_CR48","unstructured":"Zhang, H., Yu, Y., & Jiao, J., et\u00a0al. (2019). Theoretically principled trade-off between robustness and accuracy. In Proceedings of the 36th international conference on machine learning (ICML), pp. 7472\u20137482"},{"key":"1860_CR49","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., & Efros, A. A., et\u00a0al. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 586\u2013595","DOI":"10.1109\/CVPR.2018.00068"}],"container-title":["International Journal of Computer Vision"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-023-01860-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11263-023-01860-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11263-023-01860-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,27]],"date-time":"2023-10-27T14:09:53Z","timestamp":1698415793000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11263-023-01860-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,14]]},"references-count":49,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["1860"],"URL":"https:\/\/doi.org\/10.1007\/s11263-023-01860-4","relation":{},"ISSN":["0920-5691","1573-1405"],"issn-type":[{"value":"0920-5691","type":"print"},{"value":"1573-1405","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,14]]},"assertion":[{"value":"2 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}