{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T03:54:16Z","timestamp":1768535656771,"version":"3.49.0"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"9","license":[{"start":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:00:00Z","timestamp":1689292800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:00:00Z","timestamp":1689292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100008982","name":"QNRF","doi-asserted-by":"crossref","award":["NPRP11C-1229-170007"],"award-info":[{"award-number":["NPRP11C-1229-170007"]}],"id":[{"id":"10.13039\/100008982","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100019687","name":"Hamad bin Khalifa University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100019687","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Adversarial training has been empirically shown to be more prone to overfitting than standard training. The exact underlying reasons are still not fully understood. In this paper, we identify one cause of overfitting related to current practices of generating adversarial examples from misclassified samples. We show that, following current practice, adversarial examples from misclassified samples results in harder-to-classify samples than the original ones. This leads to a complex adjustment of the decision boundary during training and hence overfitting. To mitigate this issue, we propose A3T, an accuracy aware AT method that generate adversarial example differently for misclassified and correctly classified samples. We show that our approach achieves better generalization while maintaining comparable robustness to state-of-the-art AT methods on a wide range of computer vision, natural language processing, and tabular tasks.<\/jats:p>","DOI":"10.1007\/s10994-023-06341-w","type":"journal-article","created":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T19:01:45Z","timestamp":1689361305000},"page":"3191-3210","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["A3T: accuracy aware adversarial training"],"prefix":"10.1007","volume":"112","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9300-6564","authenticated-orcid":false,"given":"Enes","family":"Altinisik","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Safa","family":"Messaoud","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Husrev Taha","family":"Sencar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sanjay","family":"Chawla","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,7,14]]},"reference":[{"key":"6341_CR1","doi-asserted-by":"crossref","unstructured":"Altinisik, E., Sajjad, H., Sencar, H. T., Messaoud, S., & Chawla, S. (2022). Impact of adversarial training on robustness and generalizability of language models. arXiv:2211.05523.","DOI":"10.18653\/v1\/2023.findings-acl.496"},{"key":"6341_CR2","unstructured":"Balaji, Y., Goldstein, T., & Hoffman, J. (2019). Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets. arXiv:1910.08051."},{"key":"6341_CR3","unstructured":"Chen, T., Zhang, Z., Liu, S., Chang, S., & Wang, Z. (2021). Robust overfitting may be mitigated by properly learned smoothening. In ICLR."},{"key":"6341_CR4","doi-asserted-by":"crossref","unstructured":"Cheng, M., Lei, Q., Chen, P., Dhillon, I. S., & Hsieh, C. (2022). CAT: customized adversarial training for improved robustness. In IJCAI (pp. 673\u2013679).","DOI":"10.24963\/ijcai.2022\/95"},{"key":"6341_CR5","unstructured":"Croce, F., Andriushchenko, M., Sehwag, V., & Debenedetti, E., Flammarion, N., Chiang, M., Mittal, P., & Hein, M. (2021). Robustbench: A standardized adversarial robustness benchmark. In NeurIPS."},{"key":"6341_CR6","unstructured":"Ding, G. W., Sharma, Y., Lui, K. Y. C., & Huang, R. (2020). MMA training: Direct input space margin maximization through adversarial training. In ICLR."},{"key":"6341_CR7","unstructured":"Dong, Y., Xu, K., Yang, X., Pang, T., & Deng, Z., Su, H., & Zhu, J. (2022). Exploring memorization in adversarial training. In ICLR."},{"key":"6341_CR8","unstructured":"Engstrom, L., Ilyas, A., & Athalye, A. (2018). Evaluating and understanding the robustness of adversarial logit pairing. arXiv:1807.10272."},{"key":"6341_CR9","doi-asserted-by":"crossref","unstructured":"Fawzi, A., Balog, M., Huang, A., Hubert, T., Romera-Paredes, B., Barekatain, M., Novikov, A., Ruiz, R., & Schrittwieser, J., Swirszcz, G., & Silver, D. (2022). Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610, 47\u201353.","DOI":"10.1038\/s41586-022-05172-4"},{"key":"6341_CR10","unstructured":"Gan, Z., Chen, Y., Li, L., Zhu, C., Cheng, Y., & Liu, J. (2020). Large-scale adversarial training for vision-and-language representation learning. In NeurIPS."},{"key":"6341_CR11","unstructured":"Gan, Z., Chen, Y., Li, L., Zhu, C., Cheng, Y., & Liu, J. (2020). Large-scale adversarial training for vision-and-language representation learning. In NeurIPS."},{"key":"6341_CR12","unstructured":"Goodfellow, I. J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. In ICLR."},{"key":"6341_CR13","unstructured":"Gowal, S., Qin, C., Uesato, J., Mann, T. A., & Kohli, P. (2020). Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv:2010.03593."},{"key":"6341_CR14","unstructured":"He, P., Liu, X., Gao, J., & Chen, W. (2021). Deberta: Decoding-enhanced bert with disentangled attention. In ICLR."},{"key":"6341_CR15","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV (pp. 1026\u20131034).","DOI":"10.1109\/ICCV.2015.123"},{"key":"6341_CR16","unstructured":"Huang, L., Zhang, C., & Zhang, H. (2020). Self-adaptive training: Beyond empirical risk minimization. In NeurIPS."},{"key":"6341_CR17","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1038\/s41586-021-03819-2","volume":"596","author":"J Jumper","year":"2021","unstructured":"Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., \u017d\u00eddek, A., Potapenko, A., & Bridgland, A. (2021). Highly accurate protein structure prediction with alphafold. Nature, 596, 583\u2013589.","journal-title":"Nature"},{"key":"6341_CR18","unstructured":"Kohavi, R., & Becker, B. (1996). Adult Dataset. In UCI Machine Learning Repository."},{"key":"6341_CR19","doi-asserted-by":"crossref","unstructured":"Kurakin, A., Goodfellow, I. J., & Bengio, S. (2017). Adversarial examples in the physical world. In ICLR.","DOI":"10.1201\/9781351251389-8"},{"key":"6341_CR20","unstructured":"Kurakin, A., Goodfellow, I. J., & Bengio, S. (2017). Adversarial machine learning at scale. In ICLR."},{"key":"6341_CR21","unstructured":"Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. In ICLR."},{"key":"6341_CR22","unstructured":"Matlab: Compare Deep Learning Networks for Credit Default Prediction\u2014MATLAB & Simulink. Retrieved September 27, 2022, from https:\/\/www.mathworks.com\/help\/deeplearning\/ug\/compare-deep-learning-networks-for-credit-default-prediction.html."},{"key":"6341_CR23","unstructured":"Miyato, T., Maeda, S.-I., Koyama, M., Nakae, K., & Ishii, S. (2015). Distributional smoothing with virtual adversarial training. arXiv:1507.00677."},{"issue":"8","key":"6341_CR24","doi-asserted-by":"publisher","first-page":"1979","DOI":"10.1109\/TPAMI.2018.2858821","volume":"41","author":"T Miyato","year":"2019","unstructured":"Miyato, T., Maeda, S., Koyama, M., & Ishii, S. (2019). Virtual adversarial training: A regularization method for supervised and semi-supervised learning. TPAMI, 41(8), 1979\u20131993.","journal-title":"TPAMI"},{"key":"6341_CR25","unstructured":"Pang, T., Yang, X., Dong, Y., Su, H., & Zhu, J. (2021). Bag of tricks for adversarial training. In ICLR."},{"key":"6341_CR26","doi-asserted-by":"crossref","unstructured":"Papernot, N., McDaniel, P. D., Goodfellow, I. J., Jha, S., Celik, Z. B., & Swami, A. (2017). Practical black-box attacks against machine learning. In AsiaCCS (pp. 506\u2013519).","DOI":"10.1145\/3052973.3053009"},{"key":"6341_CR27","doi-asserted-by":"crossref","unstructured":"Pozzolo, A. D., Caelen, O., Johnson, R. A., & Bontempi, G. (2015). Calibrating probability with undersampling for unbalanced classification. In SSCI (pp. 159\u2013166).","DOI":"10.1109\/SSCI.2015.33"},{"key":"6341_CR28","first-page":"8821","volume":"139","author":"A Ramesh","year":"2021","unstructured":"Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021). Zero-shot text-to-image generation. ICML, 139, 8821\u20138831.","journal-title":"ICML"},{"key":"6341_CR29","first-page":"8093","volume":"119","author":"L Rice","year":"2020","unstructured":"Rice, L., Wong, E., & Kolter, J. Z. (2020). Overfitting in adversarially robust deep learning. ICML, 119, 8093\u20138104.","journal-title":"ICML"},{"key":"6341_CR30","unstructured":"Schmidt, L., Santurkar, S., Tsipras, D., Talwar, K., & Madry, A. (2018). Adversarially robust generalization requires more data. In NeurIPS (pp. 5019\u20135031)."},{"key":"6341_CR31","doi-asserted-by":"crossref","unstructured":"Sch\u00f6nherr, L., Kohls, K., Zeiler, S., Holz, T., & Kolossa, D. (2019). Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding. In NDSS.","DOI":"10.14722\/ndss.2019.23288"},{"key":"6341_CR32","unstructured":"Stutz, D. (2022). Understanding and improving robustness and uncertainty estimation in deep learning. In Saarland University."},{"key":"6341_CR33","unstructured":"Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., & Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. In ICLR."},{"key":"6341_CR34","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., & Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. In NeurIPS (pp. 5998\u20136008)."},{"key":"6341_CR35","doi-asserted-by":"crossref","unstructured":"Wang, J., & Zhang, H. (2019). Bilateral adversarial training: Towards fast training of more robust models against adversarial attacks. In ICCV (pp. 6628\u20136637).","DOI":"10.1109\/ICCV.2019.00673"},{"key":"6341_CR36","doi-asserted-by":"crossref","unstructured":"Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019). GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR.","DOI":"10.18653\/v1\/W18-5446"},{"key":"6341_CR37","unstructured":"Wang, Y., Zou, D., Yi, J., & Bailey, J., Ma, X., & Gu, Q. (2020). Improving adversarial robustness requires revisiting misclassified examples. In ICLR."},{"key":"6341_CR38","first-page":"6586","volume":"97","author":"Y Wang","year":"2019","unstructured":"Wang, Y., Ma, X., Bailey, J., Yi, J., Zhou, B., & Gu, Q. (2019). On the convergence and robustness of adversarial training. ICML, 97, 6586\u20136595.","journal-title":"ICML"},{"key":"6341_CR39","unstructured":"Wu, D., Xia, S., & Wang, Y. (2020). Adversarial weight perturbation helps robust generalization. In NeurIPS."},{"key":"6341_CR40","first-page":"25595","volume":"162","author":"C Yu","year":"2022","unstructured":"Yu, C., Han, B., Shen, L., Yu, J., Gong, C., & Liu, T. (2022). Understanding robust overfitting of adversarial training and beyond. ICML, 162, 25595\u201325610.","journal-title":"ICML"},{"key":"6341_CR41","unstructured":"Zhang, S., Gao, H., Zhang, T., Zhou, Y., & Wu, Z. (2022). Alleviating robust overfitting of adversarial training with consistency regularization. arXiv:2205.11744."},{"key":"6341_CR42","first-page":"7472","volume":"97","author":"H Zhang","year":"2019","unstructured":"Zhang, H., Yu, Y., Jiao, J., Xing, E. P., El Ghaoui, L., & Jordan, M. (2019). Theoretically principled trade-off between robustness and accuracy. ICML, 97, 7472\u20137482.","journal-title":"ICML"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06341-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-023-06341-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-023-06341-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,10]],"date-time":"2023-08-10T17:20:23Z","timestamp":1691688023000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-023-06341-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,14]]},"references-count":42,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["6341"],"URL":"https:\/\/doi.org\/10.1007\/s10994-023-06341-w","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,14]]},"assertion":[{"value":"29 November 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 April 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 July 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Enes Altinisik has received research support from QNRF. All authors are employees of Qatar Computing Research Institute (QCRI\/HBKU).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to participate"}},{"value":"Not applicable.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}