{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T20:45:17Z","timestamp":1775594717152,"version":"3.50.1"},"reference-count":289,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2025,2,10]],"date-time":"2025-02-10T00:00:00Z","timestamp":1739145600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EU\u2019s H2020","award":["101016233"],"award-info":[{"award-number":["101016233"]}]},{"name":"EU\u2019s H2020 research and innovation programme under the Marie Sk\u0142odowska\u2010Curie","award":["955990"],"award-info":[{"award-number":["955990"]}]},{"name":"ICAI GENIUS lab of the research program ROBUST","award":["KICH3.LTP.20.006"],"award-info":[{"award-number":["KICH3.LTP.20.006"]}]},{"DOI":"10.13039\/501100003246","name":"Dutch Research Council","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>\n            Despite the impressive performance of Artificial Intelligence (AI) systems, their robustness remains elusive and constitutes a key issue that impedes large-scale adoption.\n            <jats:styled-content style=\"color:#000000\">Besides, robustness is interpreted differently across domains and contexts of AI<\/jats:styled-content>\n            . In this work, we systematically survey recent progress to provide a reconciled terminology of concepts around AI robustness. We introduce three taxonomies to organize and describe the literature both from a fundamental and applied point of view:\n            <jats:styled-content style=\"color:#000000\">(1) methods and approaches that address robustness in different phases of the machine learning pipeline; (2) methods improving robustness in specific model architectures, tasks, and systems; and in addition, (3) methodologies and insights around evaluating the robustness of AI systems, particularly the tradeoffs with other trustworthiness properties.<\/jats:styled-content>\n            Finally, we identify and discuss research gaps and opportunities and give an outlook on the field. We highlight the central role of humans in evaluating and enhancing AI robustness, considering the necessary knowledge\n            <jats:styled-content style=\"color:#000000\">they<\/jats:styled-content>\n            can provide, and discuss the need for better understanding practices and developing supportive tools in the future.\n          <\/jats:p>","DOI":"10.1145\/3665926","type":"journal-article","created":{"date-parts":[[2024,5,27]],"date-time":"2024-05-27T11:56:48Z","timestamp":1716811008000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":34,"title":["A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1681-5859","authenticated-orcid":false,"given":"Andrea","family":"Tocchetti","sequence":"first","affiliation":[{"name":"Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4200-1664","authenticated-orcid":false,"given":"Lorenzo","family":"Corti","sequence":"additional","affiliation":[{"name":"TU Delft, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2725-5305","authenticated-orcid":false,"given":"Agathe","family":"Balayn","sequence":"additional","affiliation":[{"name":"TU Delft, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9685-4873","authenticated-orcid":false,"given":"Mireia","family":"Yurrita","sequence":"additional","affiliation":[{"name":"TU Delft, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0139-1061","authenticated-orcid":false,"given":"Philip","family":"Lippmann","sequence":"additional","affiliation":[{"name":"TU Delft, Delft, Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8753-2434","authenticated-orcid":false,"given":"Marco","family":"Brambilla","sequence":"additional","affiliation":[{"name":"Dipartimento di Elettronica e Informazione, Politecnico di Milano, Milano, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0350-0313","authenticated-orcid":false,"given":"Jie","family":"Yang","sequence":"additional","affiliation":[{"name":"TU Delft, Delft, Netherlands"}]}],"member":"320","published-online":{"date-parts":[[2025,2,10]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"119","volume-title":"CNS","author":"Abdelaty Maged","year":"2021","unstructured":"Maged Abdelaty, Sandra Scott-Hayward, Roberto Doriguzzi-Corin, and Domenico Siracusa. 2021. GADoT: GAN-based adversarial training for robust DDoS attack detection. In CNS. IEEE, 119\u2013127."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2021.3097296"},{"key":"e_1_3_2_4_2","first-page":"2114","volume-title":"UAI","author":"Agarwal Chirag","year":"2021","unstructured":"Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitnik. 2021. Towards a unified framework for fair and stable graph representation learning. In UAI. PMLR, 2114\u20132124."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2016.2643678"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","unstructured":"David Alvarez-Melis and Tommi S. Jaakkola. 2018. On the Robustness of Interpretability Methods. arXiv:1806.08049. [cs.LG]. DOI:10.48550\/ARXIV.1806.08049","DOI":"10.48550\/ARXIV.1806.08049"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/HSI.2018.8430788"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445736"},{"key":"e_1_3_2_9_2","first-page":"1","article-title":"Detection and robustness evaluation of android malware classifiers","author":"Anupama M. L.","year":"2021","unstructured":"M. L. Anupama, P. Vinod, Corrado Aaron Visaggio, M. A. Arya, Josna Philomina, Rincy Raphael, Anson Pinhero, K. S. Ajith, and P. Mathiyalagan. 2021. Detection and robustness evaluation of android malware classifiers. Journal of Computer Virology and Hacking Techniques 18, 3 (2021), 1\u201324.","journal-title":"Journal of Computer Virology and Hacking Techniques"},{"key":"e_1_3_2_10_2","unstructured":"Elahe Arani Fahad Sarfraz and Bahram Zonooz. 2020. Adversarial concurrent training: Optimizing robustness and accuracy trade-off of deep neural networks. In 31st British Machine Vision Conference (BMVC\u201920) BMVA Press. Retrieved from https:\/\/www.bmvc2020-conference.com\/assets\/papers\/0859.pdf"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/AITEST49225.2020.00009"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i7.16734"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","unstructured":"Shriya Atmakuri Tejas Chheda Dinesh Kandula Nishant Yadav Taesung Lee and Hessel Tuinhof. 2022. Robustness of Explanation Methods for NLP Models. arXiv:2206.12284. [cs.CL]. DOI:10.48550\/ARXIV.2206.12284","DOI":"10.48550\/ARXIV.2206.12284"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","unstructured":"Yang Bai Yuyuan Zeng Yong Jiang Shu-Tao Xia Xingjun Ma and Yisen Wang. 2022. Improving Adversarial Robustness via Channel-wise Activation Suppressing. arXiv:2103.08307. [cs.LG]. DOI:10.48550\/ARXIV.2103.08307","DOI":"10.48550\/ARXIV.2103.08307"},{"key":"e_1_3_2_16_2","first-page":"5644","volume-title":"NeurIPS","author":"Bajaj Mohit","year":"2021","unstructured":"Mohit Bajaj, Lingyang Chu, Zi Yu Xue, Jian Pei, Lanjun Wang, Peter Cho-Ho Lam, and Yong Zhang. 2021. Robust counterfactual explanations on graph neural networks. In NeurIPS. Vol. 34, Curran Associates, Inc., 5644\u20135655. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/2c8c3a57383c63caef6724343eb62257-Paper.pdf"},{"key":"e_1_3_2_17_2","first-page":"1709","volume-title":"WWW.","author":"Balayn Agathe","year":"2022","unstructured":"Agathe Balayn, Gaole He, Andrea Hu, Jie Yang, and Ujwal Gadiraju. 2022. Ready player one! Eliciting diverse knowledge using a configurable game. In WWW.1709\u20131719."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517474"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581555"},{"key":"e_1_3_2_20_2","first-page":"1937","volume-title":"WWW.","author":"Balayn Agathe","year":"2021","unstructured":"Agathe Balayn, Panagiotis Soilis, Christoph Lofi, Jie Yang, and Alessandro Bozzon. 2021. What do you mean? Interpreting image classification with crowdsourced concept extraction and analysis. In WWW.1937\u20131948."},{"key":"e_1_3_2_21_2","volume-title":"NeurIPS","author":"Balunovic Mislav","year":"2019","unstructured":"Mislav Balunovic, Maximilian Baader, Gagandeep Singh, Timon Gehr, and Martin Vechev. 2019. Certifying geometric robustness of neural networks. In NeurIPS. Vol. 32, Curran Associates, Inc. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2019\/file\/f7fa6aca028e7ff4ef62d75ed025fe76-Paper.pdf"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33012429"},{"key":"e_1_3_2_23_2","series-title":"PMLR","first-page":"3732","volume-title":"AIStats","author":"Bar Oshrat","year":"2022","unstructured":"Oshrat Bar, Amnon Drory, and Raja Giryes. 2022. A spectral perspective of DNN robustness to label noise. In AIStats(PMLR, Vol. 151). PMLR, 3732\u20133752. Retrieved from DOI:https:\/\/proceedings.mlr.press\/v151\/bar22a.html"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-021-00423-x"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157391"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.463"},{"key":"e_1_3_2_27_2","first-page":"325","volume-title":"NeurIPS","author":"Benz Philipp","year":"2021","unstructured":"Philipp Benz, Chaoning Zhang, Adil Karjauv, and In So Kweon. 2021. Robustness may be at odds with fairness: An empirical study on class-wise accuracy. In NeurIPS. PMLR, 325\u2013342."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CISS.2018.8362326"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3501965"},{"key":"e_1_3_2_30_2","article-title":"Certifiable robustness to graph perturbations","author":"Bojchevski Aleksandar","year":"2019","unstructured":"Aleksandar Bojchevski and S. G\u00fcnnemann. 2019. Certifiable robustness to graph perturbations. In NeurIPS .","journal-title":"NeurIPS"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","unstructured":"Cristian Bucilua Rich Caruana and Alexandru Niculescu-Mizil. 2006. Model compression. In KDD \u201906. ACM 535\u2013541. DOI:10.1145\/1150402.1150464","DOI":"10.1145\/1150402.1150464"},{"key":"e_1_3_2_33_2","first-page":"77","volume-title":"FAccT","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In FAccT. PMLR, 77\u201391."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00255"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3479569"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2022.102843"},{"key":"e_1_3_2_37_2","first-page":"15602","volume-title":"NeurIPS","author":"Carbone Ginevra","year":"2020","unstructured":"Ginevra Carbone, Matthew Wicker, Luca Laurenti, A. Patane, L. Bortolussi, and Guido Sanguinetti. 2020. Robustness of Bayesian neural networks to gradient-based attacks. In NeurIPS. Vol. 33, Curran Associates, 15602\u201315613. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/b3f61131b6eceeb2b14835fa648a48ff-Paper.pdf"},{"key":"e_1_3_2_38_2","unstructured":"Nicholas Carlini A. Athalye N. Papernot W. Brendel Jonas Rauber Dimitris Tsipras Ian Goodfellow Aleksander Madry and Alexey Kurakin. 2019. On evaluating adversarial robustness. arXiv:1902.06705. [cs.LG]. Retrieved from https:\/\/arxiv.org\/abs\/1902.06705"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8080832"},{"key":"e_1_3_2_41_2","unstructured":"Alvin Chan Yi Tay Yew-Soon Ong and Jie Fu. 2020. Jacobian adversarially regularized networks for robustness. In 8th International Conference on Learning Representations (ICLR\u201920) OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=Hke0V1rKPS"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01496"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","unstructured":"Howard Chen Jacqueline He Karthik Narasimhan and Danqi Chen. 2022. Can rationalization improve robustness? In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 3792\u20133805. DOI:10.48550\/ARXIV.2204.11790","DOI":"10.48550\/ARXIV.2204.11790"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"Pin-Yu Chen Yash Sharma Huan Zhang Jinfeng Yi and Cho-Jui Hsieh. 2018. EAD: Elastic-net attacks to deep neural networks via adversarial examples. Proceedings of the AAAI Conference on Artificial Intelligence 32 1 (2018). DOI:10.1609\/aaai.v32i1.11302","DOI":"10.1609\/aaai.v32i1.11302"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-10925-7_4"},{"key":"e_1_3_2_46_2","first-page":"52","volume-title":"ECML\/KDD","author":"Chen Shang-Tse","year":"2018","unstructured":"Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Polo Chau. 2018. ShapeShifter: Robust physical adversarial attack on faster R-CNN object detector. In ECML\/KDD. Springer, 52\u201368."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01635"},{"key":"e_1_3_2_48_2","first-page":"19314","volume-title":"NeurIPS","author":"Chen Yu","year":"2020","unstructured":"Yu Chen, Lingfei Wu, and Mohammed Zaki. 2020. Iterative deep graph learning for graph neural networks: Better and robust node embeddings. In NeurIPS. Vol. 33, Curran Associates, Inc., 19314\u201319326. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/e05c7ba4e087beea9410929698dc41a6-Paper.pdf"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","unstructured":"Minhao Cheng Pin-Yu Chen Sijia Liu Shiyu Chang Cho-Jui Hsieh and Payel Das. 2021. Self-progressing robust training. In Proceedings of the AAAI Conference on Artificial Intelligence 7107\u20137115. DOI:10.48550\/ARXIV.2012.11769","DOI":"10.48550\/ARXIV.2012.11769"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","unstructured":"Minhao Cheng Qi Lei Pin-Yu Chen Inderjit Dhillon and Cho-Jui Hsieh. 2022. CAT: Customized adversarial training for improved robustness. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI\u201922) Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization 673\u2013679. DOI:10.24963\/ijcai.2022\/95","DOI":"10.24963\/ijcai.2022\/95"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/211"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2022.3160283"},{"key":"e_1_3_2_53_2","unstructured":"Jeremy Cohen Elan Rosenfeld and Zico Kolter. 2019. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th International Conference on Machine Learning Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). Proceedings of Machine Learning Research Vol. 97. PMLR 1310\u20131320. https:\/\/proceedings.mlr.press\/v97\/cohen19c.html"},{"key":"e_1_3_2_54_2","first-page":"1311","volume-title":"IJCAI","author":"Console Luca","year":"1989","unstructured":"Luca Console, Daniele Theseider Dupre, and Pietro Torasso. 1989. A theory of diagnosis for incomplete causal models.. In IJCAI. 1311\u20131317."},{"key":"e_1_3_2_55_2","volume-title":"NeurIPS","author":"Croce Francesco","year":"2021","unstructured":"Francesco Croce, M. Andriushchenko, V. Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang, Prateek Mittal, and Matthias Hein. 2021. RobustBench: A standardized adversarial robustness benchmark. In NeurIPS. Vol. 1. Retrieved from DOI:https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/file\/a3c65c2974270fd093ee8a9bf8ae7d0b-Paper-round2.pdf"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","unstructured":"Wesley Hanwen Deng Manish Nagireddy Michelle Seng Ah Lee Jatinder Singh Zhiwei Steven Wu Kenneth Holstein and Haiyi Zhu. 2022. Exploring how machine learning practitioners (try to) use fairness toolkits. In FAccT \u201922. DOI:10.1145\/3531146.3533113","DOI":"10.1145\/3531146.3533113"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","unstructured":"Zhun Deng Linjun Zhang Amirata Ghorbani and James Zou. 2021. Improving adversarial robustness via unlabeled out-of-domain data. In International Conference on Artificial Intelligence and Statistics PMLR 2845\u20132853. DOI:10.48550\/ARXIV.2006.08476","DOI":"10.48550\/ARXIV.2006.08476"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00022"},{"key":"e_1_3_2_59_2","first-page":"6478","volume-title":"NeurIPS","author":"Ding Frances","year":"2021","unstructured":"Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring adult: New datasets for fair machine learning. In NeurIPS. Vol. 34, Curran Associates, Inc., 6478\u20136490. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/32e54441e6382a7fbacbbbaf3c450059-Paper.pdf"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","unstructured":"Andrea Dittadi Samuele Papa Michele De Vita Bernhard Sch\u00f6lkopf Ole Winther and Francesco Locatello. 2022. Generalization and robustness implications in object-centric learning. In International Conference on Machine Learning PMLR 5221\u20135285. DOI:10.48550\/ARXIV.2107.00637","DOI":"10.48550\/ARXIV.2107.00637"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2021.108194"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","unstructured":"Minjing Dong Yanxi Li Yunhe Wang and Chang Xu. 2020. Adversarially robust neural architectures. arXiv preprint arXiv:2009.00902 (2020). DOI:10.48550\/ARXIV.2009.00902","DOI":"10.48550\/ARXIV.2009.00902"},{"key":"e_1_3_2_63_2","unstructured":"Qingxiu Dong Lei Li Damai Dai Ce Zheng Zhiyong Wu Baobao Chang Xu Sun Jingjing Xu Lei Li and Zhifang Sui. 2023. A Survey on In-context Learning. arxiv:2301.00234 [cs.CL]. Retrieved from https:\/\/arxiv.org\/abs\/2301.00234"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","unstructured":"Yinpeng Dong Qi-An Fu X. Yang T. Pang H. Su Zihao Xiao and Jun Zhu. 2020. Benchmarking adversarial robustness on image classification. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 321\u2013331. DOI:10.48550\/ARXIV.1912.11852","DOI":"10.48550\/ARXIV.1912.11852"},{"key":"e_1_3_2_65_2","unstructured":"Nathan Drenkow Numair Sani Ilya Shpitser and Mathias Unberath. 2022. A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?arxiv:2112.00639 [cs.CV]. Retrieved from https:\/\/arxiv.org\/abs\/2112.00639"},{"key":"e_1_3_2_66_2","first-page":"516","volume-title":"CCS","author":"Du Tianyu","year":"2021","unstructured":"Tianyu Du, Shouling Ji, Lujia Shen, Yao Zhang, Jinfeng Li, Jie Shi, Chengfang Fang, Jianwei Yin, Raheem Beyah, and Ting Wang. 2021. Cert-RNN: Towards certifying the robustness of recurrent neural networks. In CCS. 516\u2013534."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9533725"},{"key":"e_1_3_2_68_2","volume-title":"ICLR","author":"Dvijotham Krishnamurthy (Dj)","year":"2020","unstructured":"Krishnamurthy (Dj) Dvijotham, Jamie Hayes, Borja Balle, Zico Kolter, Chongli Qin, Andras Gyorgy, Kai Xiao, Sven Gowal, and Pushmeet Kohli. 2020. A framework for robustness certification of smoothed classifiers using f-divergences. In ICLR. Retrieved from DOI:https:\/\/openreview.net\/forum?id=SJlKrkSFPH"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/305"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2499149.2499168"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0055-y"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","unstructured":"James Fox and Sivasankaran Rajamanickam. 2019. How robust are graph neural networks to structural noise? arXiv preprint arXiv:1912.10206 (2019). DOI:10.48550\/ARXIV.1912.10206","DOI":"10.48550\/ARXIV.1912.10206"},{"key":"e_1_3_2_73_2","first-page":"1081","volume-title":"Big Data","author":"Freitas Scott","year":"2020","unstructured":"Scott Freitas, Shang-Tse Chen, Zijie J. Wang, and Duen Horng Chau. 2020. Unmask: Adversarial detection and defense through robust feature alignment. In Big Data. IEEE, 1081\u20131088."},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","unstructured":"Ji Gao Beilun Wang Zeming Lin Weilin Xu and Yanjun Qi. 2017. Masking deep neural network models for robustness against adversarial samples. arXiv preprint arXiv:1702.06763 (2017). DOI:10.48550\/ARXIV.1702.06763","DOI":"10.48550\/ARXIV.1702.06763"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380415"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458723"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00058"},{"key":"e_1_3_2_78_2","first-page":"7637","volume-title":"NeurIPS","author":"Geisler Simon","year":"2021","unstructured":"Simon Geisler, Tobias Schmidt, Hakan \u015eirin, Daniel Z\u00fcgner, Aleksandar Bojchevski, and Stephan G\u00fcnnemann. 2021. Robustness of graph neural networks at scale. In NeurIPS. Vol. 34, Curran Associates, Inc., 7637\u20137649. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/3ea2db50e62ceefceaf70a9d9a56a6f4-Paper.pdf"},{"key":"e_1_3_2_79_2","volume-title":"NeurIPS","year":"2019","unstructured":"Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. 2019. Towards automatic concept-based explanations. In NeurIPS."},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2018.8461907"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","unstructured":"Tejas Gokhale Swaroop Mishra Man Luo Bhavdeep Sachdeva and Chitta Baral. 2022. Generalized but not robust? Comparing the effects of data modification methods on out-of-domain generalization and adversarial robustness. In Findings of the Association for Computational Linguistics (ACL\u201922) Smaranda Muresan Preslav Nakov and Aline Villavicencio (Eds.). Association for Computational Linguistics Dublin Ireland 2705\u20132718. DOI:10.18653\/v1\/2022.findings-acl.213","DOI":"10.18653\/v1\/2022.findings-acl.213"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3134599"},{"key":"e_1_3_2_83_2","unstructured":"Dou Goodman Hao Xin Wang Yang Wu Yuesheng Xiong Junfeng and Zhang Huan. 2020. Advbox: A toolbox to generate adversarial examples that fool neural networks. arXiv:2001.05574. [cs.LG]. Retrieved from https:\/\/arxiv.org\/abs\/2001.05574"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01090-4_1"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-022-00831-6"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00398"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.822"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","unstructured":"C. Haase-Schutz R. Stal H. Hertlein and B. Sick. 2021. Iterative label improvement: Robust training by confidence based filtering and dataset partitioning. In 25th International Conference on Pattern Recognition (ICPR\u201921). IEEE Computer Society Los Alamitos CA USA 9483\u20139490. DOI:10.1109\/ICPR48806.2021.9411918","DOI":"10.1109\/ICPR48806.2021.9411918"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","unstructured":"Kilian Hendrickx Lorenzo Perini Dries Van der Plas Wannes Meert and Jesse Davis. 2024. Machine learning with a reject option: A survey. Machine Learning 113 5 (2024) 3073\u20133110. DOI:10.1007\/s10994-024-06534-x","DOI":"10.1007\/s10994-024-06534-x"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","unstructured":"Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking Neural Network Robustness to Common Corruptions and Perturbations. arXiv:1903.12261. [cs.LG]. DOI:10.48550\/ARXIV.1903.12261","DOI":"10.48550\/ARXIV.1903.12261"},{"key":"e_1_3_2_92_2","unstructured":"Patrick Henriksen Kerstin Hammernik Daniel Rueckert and Alessio Lomuscio. 2021. Bias field robustness verification of large neural image classifiers. In 32nd British Machine Vision Conference (BMVC\u201921). BMVA Press 202. Retrieved 22-June-2022 from https:\/\/www.bmvc2021-virtualconference.com\/assets\/papers\/1291.pdf"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","unstructured":"Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531. [stat.ML]. DOI:10.48550\/ARXIV.1503.02531","DOI":"10.48550\/ARXIV.1503.02531"},{"issue":"337","key":"e_1_3_2_94_2","first-page":"2","article-title":"Human-centered concept explanations for neural networks","volume":"342","author":"Hitzler P.","year":"2022","unstructured":"P. Hitzler and M. K. Sarker. 2022. Human-centered concept explanations for neural networks. Neuro-Symbolic Artificial Intelligence: The State of the Art 342, 337 (2022), 2.","journal-title":"Neuro-Symbolic Artificial Intelligence: The State of the Art"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300809"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300830"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3392878"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","unstructured":"Ramtin Hosseini Xingyi Yang and Pengtao Xie. 2021. DSRNA: Differentiable search of robust neural architectures. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201921). Computer Vision Foundation\/IEEE 6196\u20136205. DOI:10.1109\/CVPR46437.2021.00613","DOI":"10.1109\/CVPR46437.2021.00613"},{"key":"e_1_3_2_99_2","first-page":"2955","volume-title":"WWW","year":"2020","unstructured":"Xiao Hu, Haobo Wang, Anirudh Vegesana, Somesh Dube, Kaiwen Yu, Gore Kao, Shuo-Han Chen, Yung-Hsiang Lu, George K. Thiruvathukal, and Ming Yin. 2020. Crowdsourcing detection of sampling biases in image datasets. In WWW. 2955\u20132961."},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1243"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-11915-1_31"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00057"},{"key":"e_1_3_2_103_2","first-page":"10558","volume-title":"NeurIPS","author":"Jeong Jongheon","year":"2020","unstructured":"Jongheon Jeong and Jinwoo Shin. 2020. Consistency regularization for certified robustness of smoothed classifiers. In NeurIPS. Vol. 33, Curran Associates, Inc., 10558\u201310570. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/77330e1330ae2b086e5bfcae50d9ffae-Paper.pdf"},{"key":"e_1_3_2_104_2","unstructured":"Malhar Jere Maghav Kumar and Farinaz Koushanfar. 2020. A singular value perspective on model robustness. arXiv:2012.03516. [cs.CV]. Retrieved from https:\/\/arxiv.org\/abs\/2012.03516"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6311"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","unstructured":"Jonghoon Jin Aysegul Dundar and Eugenio Culurciello. 2015. Robust Convolutional Neural Networks under Adversarial Noise. arXiv:1511.06306. [cs.LG]. DOI:10.48550\/ARXIV.1511.06306","DOI":"10.48550\/ARXIV.1511.06306"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403049"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","unstructured":"Takuhiro Kaneko Yoshitaka Ushiku and Tatsuya Harada. 2019. Label-noise robust generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201919). Computer Vision Foundation\/IEEE Long Beach CA USA 2467\u20132476. DOI:10.1109\/CVPR.2019.00257","DOI":"10.1109\/CVPR.2019.00257"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10796-021-10219-4"},{"key":"e_1_3_2_111_2","volume-title":"ICML","year":"2018","unstructured":"Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Vi\u00e9gas, and Rory Sayres. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors. In ICML."},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00168"},{"key":"e_1_3_2_113_2","first-page":"5637","volume-title":"ICML","author":"Koh Pang Wei","year":"2021","unstructured":"Pang Wei Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, Weihua Hu, Michihiro Yasunaga, R. L. Phillips, Irena Gao, Tony Lee, Etiene David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2021. Wilds: A benchmark of in-the-wild distribution shifts. In ICML. PMLR, 5637\u20135664."},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-020-01401-3"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1145\/3476052"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21353"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.266"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.5555\/3524938.3525460"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","unstructured":"Alfred Laugros Alice Caplier and Matthieu Ospici. 2020. Addressing neural network robustness with mixup and targeted labeling adversarial training. In Computer Vision - ECCV 2020 Workshops - Glasgow Adrien Bartoli and Andrea Fusiello (Eds.). Lecture Notes in Computer Science Springer 178\u2013195. DOI:10.1007\/978-3-030-68238-5_14","DOI":"10.1007\/978-3-030-68238-5_14"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3048120"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445261"},{"key":"e_1_3_2_122_2","first-page":"6212","volume-title":"ICML","author":"Leino Klas","year":"2021","unstructured":"Klas Leino, Z. Wang, and M. Fredrikson. 2021. Globally-robust neural networks. In ICML. Vol. 139, PMLR, 6212\u20136222. Retrieved from DOI:https:\/\/proceedings.mlr.press\/v139\/leino21a.html"},{"key":"e_1_3_2_123_2","unstructured":"Alexander Levine and Soheil Feizi. 2021. Improved deterministic smoothing for L \\(_1\\) certified robustness. In Proceedings of the 38th International Conference on Machine Learning (ICML\u201921) Marina Meila and Tong Zhang (Eds.). Vol. 139 PMLR 6254\u20136264. http:\/\/proceedings.mlr.press\/v139\/levine21a.html"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21342"},{"key":"e_1_3_2_125_2","first-page":"27249","volume-title":"NeurIPS","author":"Li Dongyue","year":"2021","unstructured":"Dongyue Li and Hongyang Zhang. 2021. Improved regularization and robustness for fine-tuning in neural networks. In NeurIPS. Vol. 34, Curran Associates, Inc., 27249\u201327262. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/e4a93f0332b2519177ed55741ea4e5e7-Paper.pdf"},{"key":"e_1_3_2_126_2","first-page":"1381","volume-title":"USENIX","author":"Li Jinfeng","year":"2020","unstructured":"Jinfeng Li, Tianyu Du, Shouling Ji, Rong Zhang, Quan Lu, Min Yang, and Ting Wang. 2020. \\(\\lbrace\\) TextShield \\(\\rbrace\\) : Robust text classification based on multimodal embedding and neural machine translation. In USENIX. 1381\u20131398."},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/654"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","unstructured":"Xin Li Xiangrui Li Deng Pan and Dongxiao Zhu. 2021. Improving adversarial robustness via probabilistically compact loss with logit Cconstraints. In Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI\u201921) Thirty-Third Conference on Innovative Applications of Artificial Intelligence (IAAI\u201921) The Eleventh Symposium on Educational Advances in Artificial Intelligence (EAAI\u201921) Virtual Event February 2-9 2021 AAAI Press 8482\u20138490. DOI:10.1609\/AAAI.V35I10.17030","DOI":"10.1609\/AAAI.V35I10.17030"},{"key":"e_1_3_2_129_2","first-page":"29578","volume-title":"NeurIPS","author":"Li Yanxi","year":"2021","unstructured":"Yanxi Li, Zhaohui Yang, Yunhe Wang, and Chang Xu. 2021. Neural architecture dilation for adversarial robustness. In NeurIPS. Vol. 34, Curran Associates, Inc., 29578\u201329589. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2021\/file\/f7664060cc52bc6f3d620bcedc94a4b6-Paper.pdf"},{"key":"e_1_3_2_130_2","unstructured":"Zhimin Li Shusen Liu Xin Yu Kailkhura Bhavya Jie Cao Diffenderfer James Daniel Peer-Timo Bremer and Valerio Pascucci. 2023. \u201cUnderstanding Robustness Lottery\u201d: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches."},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376590"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380306"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2022.3207878"},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(97)00011-7"},{"key":"e_1_3_2_135_2","unstructured":"Divyam Madaan Jinwoo Shin and Sung Ju Hwang. 2021. Learning to generate noise for multi-attack robustness. In Proceedings of the 38th International Conference on Machine Learning (ICML\u201921) Vol. 139 PMLR 7279\u20137289. Retrieved from http:\/\/proceedings.mlr.press\/v139\/madaan21a.html"},{"key":"e_1_3_2_136_2","volume-title":"ICLR","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In ICLR. OpenReview.net. Retrieved from DOI:https:\/\/openreview.net\/forum?id=rJzIBfZAb"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","unstructured":"Ravi Mangal Aditya V. Nori and Alessandro Orso. 2019. Robustness of neural networks: a probabilistic and practical approach. In Proceedings of the 41st International Conference on Software Engineering: New Ideas and Emerging Results ICSE (NIER) 2019 Montreal QC Canada May 29-31 2019 IEEE\/ACM 93\u201396. DOI:10.1109\/ICSE-NIER.2019.00032","DOI":"10.1109\/ICSE-NIER.2019.00032"},{"key":"e_1_3_2_138_2","unstructured":"Chengzhi Mao Ziyuan Zhong Junfeng Yang Carl Vondrick and Baishakhi Ray. 2019. Metric learning for adversarial robustness. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 NeurIPS 2019 December 8-14 2019 Vancouver BC Canada 478\u2013489. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/c24cd76e1ce41366a4bbe8a49b02a028-Abstract.html"},{"key":"e_1_3_2_139_2","unstructured":"Gary Marcus. 2020. The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence."},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00190"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1145\/3457607"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1145\/2666652.2666656"},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460700"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/MLSP.2017.8168163"},{"key":"e_1_3_2_145_2","first-page":"7721","volume-title":"ICML","author":"Miller John P.","year":"2021","unstructured":"John P. Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. 2021. Accuracy on the line: On the strong correlation between out-of-distribution and in-distribution generalization. In ICML. PMLR, 7721\u20137735."},{"key":"e_1_3_2_146_2","first-page":"3578","volume-title":"ICML","author":"Mirman Matthew","year":"2018","unstructured":"Matthew Mirman, Timon Gehr, and Martin Vechev. 2018. Differentiable abstract interpretation for provably robust neural networks. In ICML. Vol. 80, PMLR, 3578\u20133586. Retrieved from DOI:https:\/\/proceedings.mlr.press\/v80\/mirman18b.html"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287596"},{"key":"e_1_3_2_148_2","doi-asserted-by":"publisher","unstructured":"Jisoo Mok Byunggook Na Hyeokjun Choe and Sungroh Yoon. 2021. AdvRush: Searching for adversarially robust neural architectures. In 2021 IEEE\/CVF International Conference on Computer Vision ICCV 2021 Montreal QC Canada October 10-17 2021 IEEE 12302\u201312312. DOI:10.1109\/ICCV48922.2021.01210","DOI":"10.1109\/ICCV48922.2021.01210"},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.rineng.2021.100225"},{"key":"e_1_3_2_150_2","unstructured":"Seyed-Mohsen Moosavi-Dezfooli Ashish Shrivastava and Oncel Tuzel. 2019. Divide Denoise and Defend against Adversarial Attacks."},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","unstructured":"Milad Moradi and Matthias Samwald. 2021. Evaluating the robustness of neural language models to input perturbations. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing EMNLP 2021 Virtual Event\/Punta Cana Dominican Republic 7-11 November 2021 Association for Computational Linguistics 1558\u20131570. DOI:10.18653\/V1\/2021.EMNLP-MAIN.117","DOI":"10.18653\/V1\/2021.EMNLP-MAIN.117"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2978474"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510209"},{"key":"e_1_3_2_154_2","first-page":"10947","volume-title":"AAAI","author":"Nanda Vedant","year":"2022","unstructured":"Vedant Nanda, Till Speicher, John P. Dickerson, Krishna P. Gummadi, and Muhammad Bilal Zafar. 2022. Unifying model explainability and robustness for joint text classification and rationale extraction. In AAAI. Vol. 36, 10947\u201310955."},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450658"},{"key":"e_1_3_2_156_2","doi-asserted-by":"publisher","unstructured":"Muzammal Naseer Salman H. Khan Munawar Hayat Fahad Shahbaz Khan and Fatih Porikli. 2020. A Self-supervised Approach for Adversarial Robustness. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition CVPR 2020 Seattle WA USA June 13-19 2020 Computer Vision Foundation\/IEEE 259\u2013268. DOI:10.1109\/CVPR42600.2020.00034","DOI":"10.1109\/CVPR42600.2020.00034"},{"key":"e_1_3_2_157_2","unstructured":"Behnam Neyshabur Srinadh Bhojanapalli David McAllester and Nati Srebro. 2017. Exploring Generalization in deep learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017 December 4-9 2017 Long Beach CA USA 5947\u20135956. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/10ce03a1ed01077e3e289f3e53c72813-Abstract.html"},{"key":"e_1_3_2_158_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i10.17106"},{"key":"e_1_3_2_159_2","first-page":"3327","volume-title":"CogSci","author":"Nobandegani Ardavan Salehi","year":"2019","unstructured":"Ardavan Salehi Nobandegani, Kevin da Silva Castanheira, Timothy O\u2019Donnell, and Thomas R. Shultz. 2019. On robustness: An undervalued dimension of human rationality. In CogSci. 3327."},{"key":"e_1_3_2_160_2","volume-title":"NeurIPS","author":"Northcutt Curtis G.","year":"2021","unstructured":"Curtis G. Northcutt, Anish Athalye, and Jonas Mueller. 2021. Pervasive label errors in test sets destabilize machine learning benchmarks. In NeurIPS."},{"key":"e_1_3_2_161_2","unstructured":"Mehdi Nourelahi Lars Kotthoff Peijie Chen and Anh Nguyen. 2023. How explainable are adversarially-robust CNNs?"},{"key":"e_1_3_2_162_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10633"},{"key":"e_1_3_2_163_2","unstructured":"Tianyu Pang Min Lin Xiao Yang Jun Zhu and Shuicheng Yan. 2022. Robustness and accuracy could be reconcilable by (Proper) definition. In International Conference on Machine Learning ICML 2022 17-23 July 2022 Baltimore Maryland USA (Proceedings of Machine Learning Research) PMLR 17258\u201317277. Retrieved from https:\/\/proceedings.mlr.press\/v162\/pang22a.html"},{"key":"e_1_3_2_164_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01479"},{"key":"e_1_3_2_165_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.41"},{"key":"e_1_3_2_166_2","unstructured":"Magdalini Paschali Sailesh Conjeti Fernando Navarro and Nassir Navab. 2018. Generalizability vs. Robustness: Adversarial Examples for Medical Imaging."},{"key":"e_1_3_2_167_2","first-page":"4574","volume-title":"AISTATS","author":"Pawelczyk Martin","year":"2022","unstructured":"Martin Pawelczyk, Chirag Agarwal, Shalmali Joshi, Sohini Upadhyay, and Himabindu Lakkaraju. 2022. Exploring counterfactual explanations through the lens of adversarial examples: A theoretical and empirical analysis. In AISTATS. Vol. 151, PMLR, 4574\u20134594. Retrieved from DOI:https:\/\/proceedings.mlr.press\/v151\/pawelczyk22a.html"},{"key":"e_1_3_2_168_2","doi-asserted-by":"publisher","unstructured":"Joshua C. Peterson Ruairidh M. Battleday Thomas L. Griffiths and Olga Russakovsky. 2019. Human uncertainty makes classification more robust. In 2019 IEEE\/CVF International Conference on Computer Vision ICCV 2019 Seoul Korea (South) October 27 - November 2 2019 IEEE 9616\u20139625. DOI:10.1109\/ICCV.2019.00971","DOI":"10.1109\/ICCV.2019.00971"},{"key":"e_1_3_2_169_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1337"},{"key":"e_1_3_2_170_2","doi-asserted-by":"publisher","unstructured":"Maura Pintor Daniele Angioni Angelo Sotgiu Luca Demetrio Ambra Demontis Battista Biggio and Fabio Roli. 2023. ImageNet-Patch: A dataset for benchmarking machine learning robustness against adversarial patches. Pattern Recognit. 134 (2023) 109064. DOI:10.1016\/J.PATCOG.2022.109064","DOI":"10.1016\/J.PATCOG.2022.109064"},{"key":"e_1_3_2_171_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449205"},{"key":"e_1_3_2_172_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.294"},{"key":"e_1_3_2_173_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1561"},{"key":"e_1_3_2_174_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.specom.2019.08.006"},{"key":"e_1_3_2_175_2","doi-asserted-by":"publisher","unstructured":"R. Hamon H. Junklewitz and J. I. Sanchez Martin. 2020. Robustness and explainability of artificial intelligence. KJ- 1407 NA-30040-EN-N (online) (2020). Publications Office. DOI:10.2760\/57493. (online).","DOI":"10.2760\/57493"},{"key":"e_1_3_2_176_2","unstructured":"Aditi Raghunathan Sang Michael Xie Fanny Yang John C. Duchi and Percy Liang. 2020. Understanding and Mitigating the Tradeoff between Robustness and Accuracy. In Proceedings of the 37th International Conference on Machine Learning ICML 2020 13-18 July 2020 Virtual Event (Proceedings of Machine Learning Research) PMLR 7909\u20137919. Retrieved from http:\/\/proceedings.mlr.press\/v119\/raghunathan20a.html"},{"key":"e_1_3_2_177_2","doi-asserted-by":"publisher","DOI":"10.1145\/3449081"},{"key":"e_1_3_2_178_2","doi-asserted-by":"publisher","unstructured":"Vikas C. Raykar Shipeng Yu Linda H. Zhao Gerardo Hermosillo Valadez Charles Florin Luca Bogoni and Linda Moy. 2010. Learning from crowds. J. Mach. Learn. Res. 11 (2010) 1297\u20131322. DOI:10.5555\/1756006.1859894","DOI":"10.5555\/1756006.1859894"},{"key":"e_1_3_2_179_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17135"},{"key":"e_1_3_2_180_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N16-3020"},{"key":"e_1_3_2_181_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445604"},{"key":"e_1_3_2_182_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-5833-1"},{"key":"e_1_3_2_183_2","first-page":"111","volume-title":"STAIRS 2016","author":"Rodosthenous Christos","year":"2016","unstructured":"Christos Rodosthenous and Loizos Michael. 2016. A hybrid approach to commonsense knowledge acquisition. In STAIRS 2016. IOS Press, 111\u2013122."},{"key":"e_1_3_2_184_2","doi-asserted-by":"publisher","DOI":"10.14569\/IJACSA.2018.090131"},{"key":"e_1_3_2_185_2","doi-asserted-by":"publisher","unstructured":"Andras Rozsa Manuel G\u00fcnther and Terrance E. Boult. 2018. Towards robust deep neural networks with bANG. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV\u201918) Lake Tahoe NV USA March 12-15 2018 IEEE Computer Society 803\u2013811. DOI:10.1109\/WACV.2018.00093","DOI":"10.1109\/WACV.2018.00093"},{"key":"e_1_3_2_186_2","doi-asserted-by":"publisher","unstructured":"Wenjie Ruan Min Wu Youcheng Sun Xiaowei Huang Daniel Kroening and Marta Kwiatkowska. 2019. Global robustness evaluation of deep neural networks with provable guarantees for the hamming distance. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI\u201919) Macao China August 10-16 2019 ijcai.org 5944\u20135952. DOI:10.24963\/IJCAI.2019\/824","DOI":"10.24963\/IJCAI.2019\/824"},{"key":"e_1_3_2_187_2","doi-asserted-by":"publisher","unstructured":"Evgenia Rusak Lukas Schott Roland S. Zimmermann Julian Bitterwolf Oliver Bringmann Matthias Bethge and Wieland Brendel. 2020. A Simple way to make neural networks robust against diverse image corruptions. In Computer Vision - ECCV 2020-16th European Conference Glasgow UK August 23-28 2020 Proceedings Part III (Lecture Notes in Computer Science) Springer 53\u201369. DOI:10.1007\/978-3-030-58580-8_4","DOI":"10.1007\/978-3-030-58580-8_4"},{"key":"e_1_3_2_188_2","first-page":"1","volume-title":"2021 CHI","author":"Sambasivan Nithya","year":"2021","unstructured":"Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M. Aroyo. 2021. \u201cEveryone wants to do the model work, not the data work\u201d: Data cascades in high-stakes AI. In 2021 CHI. 1\u201315."},{"key":"e_1_3_2_189_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6_1"},{"key":"e_1_3_2_190_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1179"},{"key":"e_1_3_2_191_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-021-09609-8"},{"key":"e_1_3_2_192_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1454"},{"key":"e_1_3_2_193_2","unstructured":"Vikash Sehwag Saeed Mahloujifar Tinashe Handina Sihui Dai Chong Xiang Mung Chiang and Prateek Mittal. 2022. Robust learning meets generative models: Can proxy distributions improve adversarial robustness? In The Tenth International Conference on Learning Representations ICLR 2022 Virtual Event April 25-29 2022 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=WVX0NNVBBkV"},{"key":"e_1_3_2_194_2","unstructured":"Shreya Shankar Rolando Garcia Joseph M. Hellerstein and Aditya G. Parameswaran. 2022. Operationalizing machine learning: An interview study. arXiv:2209.09125. Retrieved from https:\/\/arxiv.org\/abs\/2209.09125"},{"key":"e_1_3_2_195_2","doi-asserted-by":"publisher","unstructured":"Vaishaal Shankar Achal Dave Rebecca Roelofs Deva Ramanan Benjamin Recht and Ludwig Schmidt. 2021. Do image classifiers generalize across time? In 2021 IEEE\/CVF International Conference on Computer Vision ICCV 2021 Montreal QC Canada October 10-17 2021 IEEE 9641\u20139649. DOI:10.1109\/ICCV48922.2021.00952","DOI":"10.1109\/ICCV48922.2021.00952"},{"key":"e_1_3_2_196_2","unstructured":"Rulin Shao Zhouxing Shi Jinfeng Yi Pin-Yu Chen and Cho-Jui Hsieh. 2022. On the adversarial robustness of vision transformers. Trans. Mach. Learn. Res. 2022 (2022). Retrieved from https:\/\/openreview.net\/forum?id=lE7K4n1Esk"},{"key":"e_1_3_2_197_2","first-page":"882","volume-title":"WWW.","author":"Noorian Shahin Sharifi","year":"2022","unstructured":"Shahin Sharifi Noorian, S. Qiu, U. Gadiraju, J. Yang, and Alessandro Bozzon. 2022. What should you know? a human-in-the-loop approach to unknown unknowns characterization in image recognition. In WWW.882\u2013892."},{"key":"e_1_3_2_198_2","doi-asserted-by":"publisher","unstructured":"Saima Sharmin Nitin Rathi Priyadarshini Panda and Kaushik Roy. 2020. Inherent adversarial robustness of deep spiking neural networks: Effects of discrete input encoding and non-linear activations. In Computer Vision - ECCV 2020-16th European Conference Glasgow UK August 23-28 2020 Proceedings Part XXIX (Lecture Notes in Computer Science) Springer 399\u2013414. DOI:10.1007\/978-3-030-58526-6_24","DOI":"10.1007\/978-3-030-58526-6_24"},{"key":"e_1_3_2_199_2","unstructured":"Max W. Shen. 2022. Trust in AI: Interpretability is not necessary or sufficient while black-box interaction is necessary and sufficient."},{"key":"e_1_3_2_200_2","unstructured":"Jiashuo Liu Zheyan Shen Yue He Xingxuan Zhang Renzhe Xu Han Yu and Peng Cui. 2023. Towards Out-Of-distribution generalization: A survey."},{"key":"e_1_3_2_201_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395352.3402618"},{"key":"e_1_3_2_202_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.320"},{"key":"e_1_3_2_203_2","volume-title":"ICLR","author":"Simonyan K.","year":"2014","unstructured":"K. Simonyan, A. Vedaldi, and A. Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In ICLR."},{"key":"e_1_3_2_204_2","volume-title":"ICLR","author":"Singh Gagandeep","year":"2019","unstructured":"Gagandeep Singh, Timon Gehr, Markus P\u00fcschel, and Martin T. Vechev. 2019. Boosting robustness certification of neural networks. In ICLR."},{"key":"e_1_3_2_205_2","doi-asserted-by":"publisher","DOI":"10.1108\/BIJ-02-2022-0112"},{"key":"e_1_3_2_206_2","volume-title":"ICLR","author":"Singla Sahil","year":"2022","unstructured":"Sahil Singla, Surbhi Singla, and Soheil Feizi. 2022. Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100. In ICLR. Retrieved from DOI:https:\/\/openreview.net\/forum?id=tD7eCtaSkR"},{"key":"e_1_3_2_207_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00607"},{"key":"e_1_3_2_208_2","doi-asserted-by":"publisher","unstructured":"Dylan Slack Sophie Hilgard Emily Jia Sameer Singh and Himabindu Lakkaraju. 2020. Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods. In AIES\u201920: AAAI\/ACM Conference on AI Ethics and Society New York NY USA February 7-8 2020 ACM 180\u2013186. DOI:10.1145\/3375627.3375830","DOI":"10.1145\/3375627.3375830"},{"key":"e_1_3_2_209_2","unstructured":"Carol J. Smith. 2019. Designing trustworthy AI: A human-machine teaming framework to guide development. arXiv:1910.03515. Retrieved from https:\/\/arxiv.org\/abs\/1910.03515"},{"key":"e_1_3_2_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372870"},{"key":"e_1_3_2_211_2","unstructured":"Chang Song Elias Fallon and Hai Li. 2021. Improving Adversarial Robustness in Weight-quantized Neural Networks."},{"key":"e_1_3_2_212_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21386"},{"key":"e_1_3_2_213_2","unstructured":"Matthew Staib and Stefanie Jegelka. 2017. Distributionally robust deep learning as a generalization of adversarial training. In NIPS workshop on Machine Learning and Computer Security 4."},{"key":"e_1_3_2_214_2","first-page":"3008","volume-title":"NIPS","author":"Stiennon Nisan","year":"2020","unstructured":"Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F. Christiano. 2020. Learning to summarize with human feedback. In NIPS. Vol. 33. Curran Associates, Inc., 3008\u20133021. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/1f89885d556929e98d3ef9b86448f951-Paper.pdf"},{"key":"e_1_3_2_215_2","doi-asserted-by":"crossref","DOI":"10.1007\/s10115-013-0679-x","article-title":"Explaining prediction models and individual predictions with feature contributions","author":"\u0160trumbelj E.","year":"2014","unstructured":"E. \u0160trumbelj and I. Kononenko. 2014. Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems (2014).","journal-title":"Knowledge and Information Systems"},{"key":"e_1_3_2_216_2","volume-title":"ICML","author":"Stutz David","year":"2020","unstructured":"David Stutz, Matthias Hein, and Bernt Schiele. 2020. Confidence-calibrated adversarial training: Generalizing to unseen attacks. In ICML 119 (2020), 9155\u20139166."},{"key":"e_1_3_2_217_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01258-8_39"},{"key":"e_1_3_2_218_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jjimei.2023.100170"},{"key":"e_1_3_2_219_2","unstructured":"Ke Sun Zhanxing Zhu and Zhouchen Lin. 2019. Enhancing the Robustness of Deep Neural Networks by Boundary Conditional GAN."},{"key":"e_1_3_2_220_2","doi-asserted-by":"publisher","unstructured":"Weidi Sun Yuteng Lu Xiyue Zhang and Meng Sun. 2022. DeepGlobal: A framework for global robustness verification of feedforward neural networks. J. Syst. Archit. 128 (2022) 102582. DOI:10.1016\/J.SYSARC.2022.102582","DOI":"10.1016\/J.SYSARC.2022.102582"},{"key":"e_1_3_2_221_2","volume-title":"ICML","year":"2017","unstructured":"Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In ICML."},{"key":"e_1_3_2_222_2","unstructured":"Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian J. Goodfellow and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations ICLR 2014 Banff AB Canada April 14-16 2014 Conference Track Proceedings. Retrieved from http:\/\/arxiv.org\/abs\/1312.6199"},{"key":"e_1_3_2_223_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i8.20817"},{"key":"e_1_3_2_224_2","unstructured":"Shiyu Tang Ruihao Gong Yan Wang Aishan Liu Jiakai Wang Xinyun Chen Fengwei Yu Xianglong Liu Dawn Song Alan Yuille Philip H. S. Torr and Dacheng Tao. 2022. RobustART: Benchmarking Robustness on Architecture Design and Training Techniques."},{"key":"e_1_3_2_225_2","first-page":"18583","volume-title":"NeurIPS","author":"Taori Rohan","year":"2020","unstructured":"Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. In NeurIPS. Vol. 33, Curran Associates, 18583\u201318599. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/d8330f857a17c53d217014ee776bfd50-Paper.pdf"},{"key":"e_1_3_2_226_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2020.103550"},{"key":"e_1_3_2_227_2","unstructured":"Dang Duy Thang and Toshihiro Matsui. 2019. Image transformation can make neural networks more robust against adversarial examples."},{"key":"e_1_3_2_228_2","unstructured":"Vincent Tjeng Kai Yuanqing Xiao and Russ Tedrake. 2019. Evaluating robustness of neural networks with mixed integer programming. In 7th International Conference on Learning Representations ICLR 2019 New Orleans LA USA May 6-9 2019 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=HyGIdiRqtm"},{"key":"e_1_3_2_229_2","unstructured":"Yu-Lin Tsai Chia-Yi Hsu Chia-Mu Yu and Pin-Yu Chen. 2021. Formalizing generalization and adversarial robustness of neural networks to weight perturbations. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021 NeurIPS 2021 December 6-14 2021 virtual 19692\u201319704. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/a3ab4ff8fa4deed2e3bae3a5077675f0-Abstract.html"},{"key":"e_1_3_2_230_2","volume-title":"ICLR","author":"Tsipras Dimitris","year":"2018","unstructured":"Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. 2018. Robustness may be at odds with accuracy. In ICLR."},{"key":"e_1_3_2_231_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3455381"},{"key":"e_1_3_2_232_2","unstructured":"Meet P. Vadera Satya Narayan Shukla Brian Jalaian and Benjamin M. Marlin. 2020. Assessing the adversarial robustness of monte carlo and distillation methods for deep bayesian neural network classification."},{"key":"e_1_3_2_233_2","doi-asserted-by":"crossref","unstructured":"Pratik Vaishnavi Tianji Cong Kevin Eykholt Atul Prakash and Amir Rahmati. 2019. Can Attention Masks Improve Adversarial Robustness?","DOI":"10.1007\/978-3-030-62144-5_2"},{"key":"e_1_3_2_234_2","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5274"},{"issue":"193","key":"e_1_3_2_235_2","first-page":"1","article-title":"Making better use of the crowd: How crowdsourcing can advance machine learning research","volume":"18","author":"Vaughan Jennifer Wortman","year":"2018","unstructured":"Jennifer Wortman Vaughan. 2018. Making better use of the crowd: How crowdsourcing can advance machine learning research. Journal of Machine Learning Research 18, 193 (2018), 1\u201346.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_236_2","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174014"},{"key":"e_1_3_2_237_2","doi-asserted-by":"publisher","DOI":"10.1145\/3194770.3194776"},{"key":"e_1_3_2_238_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2022.103840"},{"key":"e_1_3_2_239_2","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124784"},{"key":"e_1_3_2_240_2","doi-asserted-by":"publisher","unstructured":"Binghui Wang Jinyuan Jia Xiaoyu Cao and Neil Zhenqiang Gong. 2021. Certified robustness of graph neural networks against adversarial structural perturbation. In KDD\u201921: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Virtual Event Singapore August 14-18 2021 ACM 1645\u20131653. DOI:10.1145\/3447548.3467295","DOI":"10.1145\/3447548.3467295"},{"key":"e_1_3_2_241_2","unstructured":"Huaxia Wang and Chun-Nam Yu. 2019. A direct approach to robust deep learning using adversarial networks. In 7th International Conference on Learning Representations ICLR 2019 New Orleans LA USA May 6-9 2019 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=S1lIMn05F7"},{"key":"e_1_3_2_242_2","volume-title":"ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models","year":"2023","unstructured":"Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Weirong Ye, Xiubo Geng, Binxing Jiao, Yue Zhang, and Xingxu Xie. 2023. On the robustness of ChatGPT: An adversarial and out-of-distribution perspective. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models."},{"key":"e_1_3_2_243_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00249"},{"key":"e_1_3_2_244_2","unstructured":"Lijie Wang Hao Liu Shuyuan Peng Hongxuan Tang Xinyan Xiao Ying Chen Hua Wu and Haifeng Wang. 2021. DuTrust: A Sentiment Analysis Dataset for Trustworthiness Evaluation."},{"key":"e_1_3_2_245_2","unstructured":"Serena Lutong Wang Wenshuo Guo Harikrishna Narasimhan Andrew Cotter Maya R. Gupta and Michael I. Jordan. 2020. Robust optimization for fairness with noisy protected groups. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 NeurIPS 2020 December 6-12 2020 virtual. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/37d097caf1299d9aa79c2c2b843d2d78-Abstract.html"},{"key":"e_1_3_2_246_2","volume-title":"ICLR","author":"Wang Yisen","year":"2020","unstructured":"Yisen Wang, Difan Zou, Jinfeng Yi, James Bailey, Xingjun Ma, and Quanquan Gu. 2020. Improving adversarial robustness requires revisiting misclassified examples. In ICLR."},{"key":"e_1_3_2_247_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i16.17651"},{"key":"e_1_3_2_248_2","unstructured":"Stefan Webb Tom Rainforth Yee Whye Teh and M. Pawan Kumar. 2019. A statistical approach to assessing neural network robustness. In 7th International Conference on Learning Representations ICLR 2019 New Orleans LA USA May 6-9 2019 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=S1xcx3C5FX"},{"key":"e_1_3_2_249_2","first-page":"24824","volume-title":"NeurIPS","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS. Vol. 35, 24824\u201324837. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/file\/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf"},{"key":"e_1_3_2_250_2","unstructured":"Tsui-Wei Weng Huan Zhang Pin-Yu Chen Jinfeng Yi Dong Su Yupeng Gao Cho-Jui Hsieh and Luca Daniel. 2018. Evaluating the robustness of neural networks: An extreme value theory approach. In 6th International Conference on Learning Representations ICLR 2018 Vancouver BC Canada April 30 - May 3 2018 Conference Track Proceedings OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=BkUHlMZ0b"},{"key":"e_1_3_2_251_2","doi-asserted-by":"publisher","DOI":"10.1109\/BIGCOMP.2019.8679132"},{"key":"e_1_3_2_252_2","unstructured":"Eric Wong and J. Zico Kolter. 2021. Learning perturbation sets for robust machine learning. In 9th International Conference on Learning Representations ICLR 2021 Virtual Event Austria May 3-7 2021 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=MIDckA56aD"},{"key":"e_1_3_2_253_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0104-6"},{"key":"e_1_3_2_254_2","doi-asserted-by":"publisher","unstructured":"Chenwang Wu Defu Lian Yong Ge Zhihao Zhu Enhong Chen and Senchao Yuan. 2021. Fight fire with fire: Towards robust recommender systems via adversarial poisoning training. InSIGIR \u201921. ACM 1074\u20131083. DOI:10.1145\/3404835.3462914","DOI":"10.1145\/3404835.3462914"},{"key":"e_1_3_2_255_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i13.17388"},{"key":"e_1_3_2_256_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jisa.2022.103121"},{"key":"e_1_3_2_257_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2022.01.039"},{"key":"e_1_3_2_258_2","first-page":"11492","volume-title":"ICML","author":"Xu Han","year":"2021","unstructured":"Han Xu, Xiaorui Liu, Yaxin Li, Anil Jain, and Jiliang Tang. 2021. To be robust or to be fair: Towards fairness in adversarial training. In ICML. Vol. 139, PMLR, 11492\u201311501. Retrieved from DOI:https:\/\/proceedings.mlr.press\/v139\/xu21b.html"},{"key":"e_1_3_2_259_2","volume-title":"NeurIPS","author":"Xu Yilun","year":"2019","unstructured":"Yilun Xu, Peng Cao, Yuqing Kong, and Yizhou Wang. 2019. L_DMI: A novel information-theoretic loss function for training deep nets robust to label noise. In NeurIPS. Vol. 32, Curran Associates, Inc. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper\/2019\/file\/8a1ee9f2b7abe6e88d1a479ab6a42c5e-Paper.pdf"},{"key":"e_1_3_2_260_2","first-page":"1161","volume-title":"ICML","year":"2011","unstructured":"Yan Yan, Romer Rosales, Glenn Fung, and Jennifer G. Dy. 2011. Active learning from crowds. In ICML. 1161\u20131168."},{"key":"e_1_3_2_261_2","unstructured":"Ziang Yan Yiwen Guo and Changshui Zhang. 2018. Deep Defense: Training DNNs with improved adversarial robustness. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 NeurIPS 2018 December 3-8 2018 Montr\u00e9al Canada 417\u2013426. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/8f121ce07d74717e0b1f21d122e04521-Abstract.html"},{"key":"e_1_3_2_262_2","volume-title":"WWW","author":"Yang J.","year":"2018","unstructured":"J. Yang, T. Drake, A Damianou, and Y. Maarek. 2018. Leveraging crowdsourcing data for deep active learning. An application: Learning intents in Alexa. In WWW."},{"key":"e_1_3_2_263_2","first-page":"2158","volume-title":"WWW","year":"2019","unstructured":"Jie Yang, Alisa Smirnova, Dingqi Yang, Gianluca Demartini, Yuan Lu, and Philippe Cudre-Mauroux. 2019. Scalpel-CD: Leveraging crowdsourcing and deep probabilistic modeling for debugging noisy training data. In WWW. 2158\u20132168."},{"key":"e_1_3_2_264_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00165-021-00548-1"},{"key":"e_1_3_2_265_2","unstructured":"Yichen Yang Xiaosen Wang and Kun He. 2022. Robust textual embedding against word-level adversarial attacks. In Uncertainty in Artificial Intelligence Proceedings of the Thirty-Eighth Conference on Uncertainty in Artificial Intelligence UAI 2022 1-5 August 2022 Eindhoven The Netherlands (Proceedings of Machine Learning Research) PMLR 2214\u20132224. Retrieved from https:\/\/proceedings.mlr.press\/v180\/yang22c.html"},{"key":"e_1_3_2_266_2","doi-asserted-by":"publisher","DOI":"10.1587\/nolta.10.221"},{"key":"e_1_3_2_267_2","article-title":"Detection defense against adversarial attacks with saliency map","author":"Ye Dengpan","year":"2021","unstructured":"Dengpan Ye, Chuanxi Chen, Changrui Liu, Hao Wang, and Shunzhi Jiang. 2021. Detection defense against adversarial attacks with saliency map. International Journal of Intelligent Systems 37, 12 (2021), 10193\u201310210.","journal-title":"International Journal of Intelligent Systems"},{"key":"e_1_3_2_268_2","doi-asserted-by":"publisher","unstructured":"Fuxun Yu Zhuwei Qin Chenchen Liu Liang Zhao Yanzhi Wang and Xiang Chen. 2019. Interpreting and evaluating neural network robustness. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence IJCAI 2019 Macao China August 10-16 2019 ijcai.org 4199\u20134205. DOI:10.24963\/IJCAI.2019\/583","DOI":"10.24963\/IJCAI.2019\/583"},{"key":"e_1_3_2_269_2","doi-asserted-by":"publisher","unstructured":"Feng Yuan Lina Yao and Boualem Benatallah. 2019. Adversarial collaborative neural network for robust recommendation. In SIGIR\u201919. ACM New York NY USA 1065\u20131068. DOI:10.1145\/3331184.3331321","DOI":"10.1145\/3331184.3331321"},{"key":"e_1_3_2_270_2","unstructured":"Mikhail Yurochkin Amanda Bower and Yuekai Sun. 2020. Training individually fair ML models with Sensitive Subspace Robustness."},{"key":"e_1_3_2_271_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-013-1369-6"},{"key":"e_1_3_2_272_2","unstructured":"Runtian Zhai Tianle Cai Di He Chen Dan Kun He John Hopcroft and Liwei Wang. 2019. Adversarially Robust Generalization Just Requires More Unlabeled Data."},{"issue":"1","key":"e_1_3_2_273_2","first-page":"1","article-title":"How do data science workers collaborate? Roles, workflows, and tools","volume":"4","author":"Zhang Amy X.","year":"2020","unstructured":"Amy X. Zhang, Michael Muller, and Dakuo Wang. 2020. How do data science workers collaborate? Roles, workflows, and tools. ACM on Human-Computer Interaction 4, CSCW1 (2020), 1\u201323.","journal-title":"ACM on Human-Computer Interaction"},{"key":"e_1_3_2_274_2","doi-asserted-by":"publisher","DOI":"10.1109\/tip.2020.3042083"},{"key":"e_1_3_2_275_2","unstructured":"Huan Zhang Tsui-Wei Weng Pin-Yu Chen Cho-Jui Hsieh and Luca Daniel. 2018. Efficient neural network robustness certification with general activation functions. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 NeurIPS 2018 December 3-8 2018 Montr\u00e9al Canada 4944\u20134953. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2018\/hash\/d04863f100d59b3eb688a11f95b0ae60-Abstract.html"},{"key":"e_1_3_2_276_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2962027"},{"key":"e_1_3_2_277_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411983"},{"key":"e_1_3_2_278_2","first-page":"38629","volume-title":"Advances in Neural Information Processing Systems","volume":"35","author":"Zhang Marvin","year":"2022","unstructured":"Marvin Zhang, Sergey Levine, and Chelsea Finn. 2022. MEMO: Test time robustness via adaptation and augmentation. In Advances in Neural Information Processing Systems. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, Curran Associates, Inc., 38629\u201338642. Retrieved from DOI:https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/file\/fc28053a08f59fccb48b11f2e31e81c7-Paper-Conference.pdf"},{"key":"e_1_3_2_279_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-022-00338-w"},{"key":"e_1_3_2_280_2","volume-title":"CVPR","year":"2018","unstructured":"Quanshi Zhang, Ying Nian Wu, and Song-Chun Zhu. 2018. Interpretable convolutional neural networks. In CVPR."},{"key":"e_1_3_2_281_2","unstructured":"Xiao Zhang and David E. Evans. 2022. Understanding intrinsic robustness using label uncertainty. In The Tenth International Conference on Learning Representations ICLR 2022 Virtual Event April 25-29 2022 OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=6ET9SzlgNX"},{"key":"e_1_3_2_282_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.82"},{"key":"e_1_3_2_283_2","unstructured":"Long Zhao Ting Liu Xi Peng and Dimitris N. Metaxas. 2020. Maximum-entropy adversarial data augmentation for improved generalization and robustness. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 NeurIPS 2020 December 6-12 2020 virtual. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/a5bfc9e07964f8dddeb95fc584cd965d-Abstract.html"},{"key":"e_1_3_2_284_2","volume-title":"NeurIPS","author":"Zheng Qinkai","year":"2021","unstructured":"Qinkai Zheng, Xu Zou, Yuxiao Dong, Yukuo Cen, Da Yin, Jiarong Xu, Yang Yang, and Jie Tang. 2021. Graph robustness benchmark: Benchmarking the adversarial robustness of graph machine learning. In NeurIPS. Retrieved from DOI:https:\/\/openreview.net\/forum?id=NxWUnvwFV4"},{"key":"e_1_3_2_285_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.590"},{"key":"e_1_3_2_286_2","unstructured":"Yiqi Zhong Lei Wu Xianming Liu and Junjun Jiang. 2022. Exploiting the Potential of Datasets: A Data-Centric Approach for Model Robustness."},{"key":"e_1_3_2_287_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-5368"},{"key":"e_1_3_2_288_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01613"},{"key":"e_1_3_2_289_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22031241"},{"key":"e_1_3_2_290_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00950"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665926","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3665926","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:44:27Z","timestamp":1750290267000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665926"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,10]]},"references-count":289,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3665926"],"URL":"https:\/\/doi.org\/10.1145\/3665926","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,10]]},"assertion":[{"value":"2022-10-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}