{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T07:49:14Z","timestamp":1776844154159,"version":"3.51.2"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"National Science and Technology Council","award":["113-2622-E-007-018"],"award-info":[{"award-number":["113-2622-E-007-018"]}]},{"name":"National Center for High-Performance Computing, National Institutes of Applied Research (NIAR), Taiwan"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Cyber-Phys. Syst."],"published-print":{"date-parts":[[2026,4,30]]},"abstract":"<jats:p>Adversarial training is a widely adopted strategy to bolster the robustness of neural network models against adversarial attacks. This article revisits the fundamental assumptions underlying image classification and suggests that representing data as one-hot labels is a key factor that leads to vulnerabilities. However, in real-world datasets, data ambiguity often arises, with samples exhibiting characteristics of multiple classes, rendering one-hot label representations imprecise. To address this, we introduce a novel approach, Low-Temperature Distillation (LTD), designed to refine label representations. Unlike previous approaches, LTD incorporates a relatively low temperature in the teacher model, while maintaining a fixed temperature for the student model during both training and inference. This strategy not only refines assumptions about data distribution but also strengthens model robustness and avoids the gradient masking problem commonly encountered in defensive distillation. Experimental results demonstrate the efficacy of the proposed method when combined with existing frameworks, achieving robust accuracy rates of 58.19%, 31.13%, and 42.08% on the CIFAR-10, CIFAR-100, and ImageNet datasets, respectively, without the need for additional data.<\/jats:p>","DOI":"10.1145\/3778246","type":"journal-article","created":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T09:19:35Z","timestamp":1764235175000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["LTD: Low Temperature Distillation for Gradient Masking-free Adversarial Training"],"prefix":"10.1145","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1780-8383","authenticated-orcid":false,"given":"Erh-Chung","family":"Chen","sequence":"first","affiliation":[{"name":"National Tsing Hua University, Hsinchu City, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3940-4478","authenticated-orcid":false,"given":"Che-Rung","family":"Lee","sequence":"additional","affiliation":[{"name":"National Tsing Hua University, Hsinchu City, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,4,21]]},"reference":[{"key":"e_1_3_1_2_2","volume-title":"Proceedings of the ICML 2021 Workshop on Adversarial Machine Learning","author":"Addepalli Sravanti","year":"2021","unstructured":"Sravanti Addepalli, Samyak Jain, Gaurang Sriramanan, Shivangi Khare, and Venkatesh Babu Radhakrishnan. 2021. Towards achieving adversarial robustness beyond perceptual limits. In Proceedings of the ICML 2021 Workshop on Adversarial Machine Learning. Retrieved from https:\/\/openreview.net\/forum?id=SHB_znlW5G7"},{"key":"e_1_3_1_3_2","first-page":"12214","article-title":"Are labels required for improving adversarial robustness","volume":"32","author":"Alayrac Jean-Baptiste","year":"2019","unstructured":"Jean-Baptiste Alayrac, Jonathan Uesato, Po-Sen Huang, Alhussein Fawzi, Robert Stanforth, and Pushmeet Kohli. 2019. Are labels required for improving adversarial robustness? In Advances in Neural Information Processing Systems, Vol. 32, 12214\u201312223.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_4_2","first-page":"20014","article-title":"XCiT: Cross-covariance image transformers","volume":"34","author":"Ali Alaaeldin","year":"2021","unstructured":"Alaaeldin Ali, Hugo Touvron, Mathilde Caron, Piotr Bojanowski, Matthijs Douze, Armand Joulin, Ivan Laptev, Natalia Neverova, Gabriel Synnaeve, Jakob Verbeek, et\u00a0al. 2021. XCiT: Cross-covariance image transformers. In Advances in Neural Information Processing Systems, Vol. 34, 20014\u201320027.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58592-1_29"},{"key":"e_1_3_1_6_2","first-page":"274","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Athalye Anish","year":"2018","unstructured":"Anish Athalye, Nicholas Carlini, and David Wagner. 2018. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. In Proceedings of the International Conference on Machine Learning. PMLR, 274\u2013283."},{"key":"e_1_3_1_7_2","unstructured":"Lucas Beyer Olivier J. H\u00e9naff Alexander Kolesnikov Xiaohua Zhai and A\u00e4ron van den Oord. 2020. Are we done with ImageNet? arXiv:2006.07159. Retrieved from https:\/\/arxiv.org\/abs\/2006.07159"},{"key":"e_1_3_1_8_2","unstructured":"Nicholas Carlini Anish Athalye Nicolas Papernot Wieland Brendel Jonas Rauber Dimitris Tsipras Ian Goodfellow Aleksander Madry and Alexey Kurakin. 2019. On evaluating adversarial robustness. arXiv:1902.06705. Retrieved from https:\/\/arxiv.org\/abs\/1902.06705"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_1_10_2","first-page":"11192","article-title":"Unlabeled data improves adversarial robustness","volume":"32","author":"Carmon Yair","year":"2019","unstructured":"Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C. Duchi, and Percy S. Liang. 2019. Unlabeled data improves adversarial robustness. In Advances in Neural Information Processing Systems, Vol. 32, 11192\u201311203.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAI.2023.3297086"},{"key":"e_1_3_1_12_2","volume-title":"Proceedings of the Asian Conference on Computer Vision","author":"Chen Erh-Chung","year":"2020","unstructured":"Erh-Chung Chen and Che-Rung Lee. 2020. Towards fast and robust adversarial training for image classification. In Proceedings of the Asian Conference on Computer Vision."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140448"},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Chen Tianlong","year":"2021","unstructured":"Tianlong Chen, Zhenyu Zhang, Sijia Liu, Shiyu Chang, and Zhangyang Wang. 2021. Robust overfitting may be mitigated by properly learned smoothening. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_1_15_2","unstructured":"Xinyun Chen Chang Liu Bo Li Kimberly Lu and Dawn Song. 2017. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv:1712.05526. Retrieved from https:\/\/arxiv.org\/abs\/1712.05526"},{"key":"e_1_3_1_16_2","volume-title":"Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)","author":"Croce Francesco","year":"2021","unstructured":"Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. 2021. RobustBench: A standardized adversarial robustness benchmark. In Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). Retrieved from https:\/\/openreview.net\/forum?id=SSKZPJCt7B"},{"key":"e_1_3_1_17_2","first-page":"2206","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Croce Francesco","year":"2020","unstructured":"Francesco Croce and Matthias Hein. 2020. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In Proceedings of the International Conference on Machine Learning. PMLR, 2206\u20132216."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01543"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/SaTML54575.2023.00024"},{"key":"e_1_3_1_20_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171\u20134186."},{"key":"e_1_3_1_21_2","unstructured":"Logan Engstrom Andrew Ilyas Hadi Salman Shibani Santurkar and Dimitris Tsipras. 2019. Robustness (Python Library). Retrieved from https:\/\/github.com\/MadryLab\/robustness"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00175"},{"key":"e_1_3_1_23_2","unstructured":"Jianping Gou Baosheng Yu Stephen John Maybank and Dacheng Tao. 2020. Knowledge distillation: A survey. arXiv:2006.05525. Retrieved from https:\/\/arxiv.org\/abs\/2006.05525"},{"key":"e_1_3_1_24_2","unstructured":"Sven Gowal Chongli Qin Jonathan Uesato Timothy Mann and Pushmeet Kohli. 2020. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv:2010.03593. Retrieved from https:\/\/arxiv.org\/abs\/2010.03593"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1002\/rob.21918"},{"key":"e_1_3_1_26_2","first-page":"11137","article-title":"Image captioning: Transforming objects into words","volume":"32","author":"Herdade Simao","year":"2019","unstructured":"Simao Herdade, Armin Kappeler, Kofi Boakye, and Joao Soares. 2019. Image captioning: Transforming objects into words. In Advances in Neural Information Processing Systems, Vol. 32, 11137\u201311147.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_27_2","unstructured":"Geoffrey Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. Retrieved from http:\/\/arxiv.org\/abs\/1503.02531"},{"key":"e_1_3_1_28_2","first-page":"1097","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, Vol. 25, 1097\u20131105.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_29_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917)","author":"Kurakin Alexey","year":"2017","unstructured":"Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2017. Adversarial machine learning at scale. In Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=BJm4T4Kgx"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781351251389-8"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3048120"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00035"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00336"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_1_35_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=rJzIBfZAb"},{"key":"e_1_3_1_36_2","first-page":"18599","article-title":"When adversarial training meets vision transformers: Recipes from training to architecture","volume":"35","author":"Mo Yichuan","year":"2022","unstructured":"Yichuan Mo, Dongxian Wu, Yifei Wang, Yiwen Guo, and Yisen Wang. 2022. When adversarial training meets vision transformers: Recipes from training to architecture. In Advances in Neural Information Processing Systems, Vol. 35, 18599\u201318611.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_37_2","first-page":"4694","article-title":"When does label smoothing help","volume":"32","author":"M\u00fcller Rafael","year":"2019","unstructured":"Rafael M\u00fcller, Simon Kornblith, and Geoffrey E. Hinton. 2019. When does label smoothing help? In Advances in Neural Information Processing Systems, Vol. 32, 4694\u20134703.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_38_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Pang Tianyu","year":"2021","unstructured":"Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. 2021. Bag of tricks for adversarial training. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=Xb8xvrtB8Ce"},{"key":"e_1_3_1_39_2","first-page":"7779","article-title":"Boosting adversarial training with hypersphere embedding","volume":"33","author":"Pang Tianyu","year":"2020","unstructured":"Tianyu Pang, Xiao Yang, Yinpeng Dong, Kun Xu, Jun Zhu, and Hang Su. 2020. Boosting adversarial training with hypersphere embedding. In Advances in Neural Information Processing Systems, Vol. 33, 7779\u20137792.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.41"},{"key":"e_1_3_1_41_2","first-page":"8093","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Rice Leslie","year":"2020","unstructured":"Leslie Rice, Eric Wong, and Zico Kolter. 2020. Overfitting in adversarially robust deep learning. In Proceedings of the International Conference on Machine Learning. PMLR, 8093\u20138104."},{"key":"e_1_3_1_42_2","first-page":"3533","article-title":"Do adversarially robust ImageNet models transfer better","volume":"33","author":"Salman Hadi","year":"2020","unstructured":"Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. 2020. Do adversarially robust ImageNet models transfer better? In Advances in Neural Information Processing Systems, Vol. 33, 3533\u20133545.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_43_2","first-page":"3358","article-title":"Adversarial training for free!","volume":"32","author":"Shafahi Ali","year":"2019","unstructured":"Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S. Davis, Gavin Taylor, and Tom Goldstein. 2019. Adversarial training for free! In Advances in Neural Information Processing Systems, Vol. 32, 3358\u20133369.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3152527"},{"key":"e_1_3_1_45_2","first-page":"9155","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Stutz David","year":"2020","unstructured":"David Stutz, Matthias Hein, and Bernt Schiele. 2020. Confidence-calibrated adversarial training: Generalizing to unseen attacks. In Proceedings of the International Conference on Machine Learning. PMLR, 9155\u20139166."},{"key":"e_1_3_1_46_2","first-page":"1","volume-title":"Proceedings of the 2nd International Conference on Learning Representations (ICLR \u201914)","author":"Szegedy Christian","year":"2014","unstructured":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In Proceedings of the 2nd International Conference on Learning Representations (ICLR \u201914). Yoshua Bengio and Yann LeCun (Eds.), ICLR, 1\u201310."},{"key":"e_1_3_1_47_2","first-page":"9625","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Tsipras Dimitris","year":"2020","unstructured":"Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. 2020. From ImageNet to image classification: Contextualizing progress on benchmarks. In Proceedings of the International Conference on Machine Learning. PMLR, 9625\u20139635."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00721"},{"key":"e_1_3_1_49_2","first-page":"36246","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Wang Zekai","year":"2023","unstructured":"Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, and Shuicheng Yan. 2023. Better diffusion models further improve adversarial training. In Proceedings of the International Conference on Machine Learning. PMLR, 36246\u201336263."},{"key":"e_1_3_1_50_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Wong Eric","year":"2020","unstructured":"Eric Wong, Leslie Rice, and J. Zico Kolter. 2020. Fast is better than free: Revisiting adversarial training. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=BJx040EFvH"},{"key":"e_1_3_1_51_2","first-page":"2958","article-title":"Adversarial weight perturbation helps robust generalization","volume":"33","author":"Wu Dongxian","year":"2020","unstructured":"Dongxian Wu, Shu-Tao Xia, and Yisen Wang. 2020. Adversarial weight perturbation helps robust generalization. In Advances in Neural Information Processing Systems, Vol. 33, 2958\u20132969.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319535.3354209"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00237"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.5244\/C.30.87"},{"key":"e_1_3_1_55_2","first-page":"7472","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhang Hongyang","year":"2019","unstructured":"Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. 2019. Theoretically principled trade-off between robustness and accuracy. In Proceedings of the International Conference on Machine Learning. PMLR, 7472\u20137482."},{"key":"e_1_3_1_56_2","first-page":"11278","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhang Jingfeng","year":"2020","unstructured":"Jingfeng Zhang, Xilie Xu, Bo Han, Gang Niu, Lizhen Cui, Masashi Sugiyama, and Mohan Kankanhalli. 2020. Attacks which do not kill training make adversarial learning stronger. In Proceedings of the International Conference on Machine Learning. PMLR, 11278\u201311287."},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1253"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2989782"},{"key":"e_1_3_1_59_2","unstructured":"Jianing Zhu Jiangchao Yao Bo Han Jingfeng Zhang Tongliang Liu Gang Niu Jingren Zhou Jianliang Xu and Hongxia Yang. 2021. Reliable adversarial distillation with unreliable teachers. arXiv:2106.04928. Retrieved from https:\/\/arxiv.org\/abs\/2106.04928"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01613"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01498"}],"container-title":["ACM Transactions on Cyber-Physical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3778246","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T06:35:48Z","timestamp":1776839748000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3778246"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,21]]},"references-count":60,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4,30]]}},"alternative-id":["10.1145\/3778246"],"URL":"https:\/\/doi.org\/10.1145\/3778246","relation":{},"ISSN":["2378-962X","2378-9638"],"issn-type":[{"value":"2378-962X","type":"print"},{"value":"2378-9638","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,21]]},"assertion":[{"value":"2024-11-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-17","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-04-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}