{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,5]],"date-time":"2026-08-05T08:49:49Z","timestamp":1785919789889,"version":"3.56.0"},"publisher-location":"Cham","reference-count":43,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783032006417","type":"print"},{"value":"9783032006424","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,10]],"date-time":"2025-08-10T00:00:00Z","timestamp":1754784000000},"content-version":"vor","delay-in-days":221,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Neural networks are vulnerable to adversarial attacks. Existing robustness evaluation methods have notable limitations, which makes robustness assessment challenging. This work explores robustness evaluation techniques and identifies key factors, including distance metrics, loss functions, attack generation algorithms, attacker models, specificity, and computational resources. Building on those factors, a novel robustness metric for classification tasks is proposed. Our metric accounts for both, targeted and untargeted attacks across three attacker models, while incorporating accuracy and loss into a weighted aggregation. The scoring includes robustness-versus-perturbation and loss-versus-perturbation curves. Our robustness metric offers a more reliable evaluation and deeper insights into model vulnerability compared to previous approaches.<\/jats:p>","DOI":"10.1007\/978-3-032-00642-4_16","type":"book-chapter","created":{"date-parts":[[2025,8,9]],"date-time":"2025-08-09T06:59:24Z","timestamp":1754722764000},"page":"272-290","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Towards a\u00a0Metric to\u00a0Assess Neural Network Resilience Against Adversarial Samples"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-1892-4622","authenticated-orcid":false,"given":"Johannes","family":"Geier","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1594-2119","authenticated-orcid":false,"given":"Patrizia","family":"Heinl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,8,10]]},"reference":[{"key":"16_CR1","doi-asserted-by":"publisher","unstructured":"Abusnaina, A., et al.: Adversarial example detection using latent neighborhood graph. In: 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), pp. 7667\u20137676 (2021). https:\/\/doi.org\/10.1109\/ICCV48922.2021.00759","DOI":"10.1109\/ICCV48922.2021.00759"},{"key":"16_CR2","unstructured":"Athalye, A., Carlini, N., Wagner, D.: Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.\u00a080, pp. 274\u2013283 (2018), https:\/\/proceedings.mlr.press\/v80\/athalye18a.html"},{"key":"16_CR3","doi-asserted-by":"crossref","unstructured":"Barreno, M.A.: Evaluating the security of machine learning algorithms. Ph.D. thesis, University of California, Berkley (2008). https:\/\/www2.eecs.berkeley.edu\/Pubs\/TechRpts\/2008\/EECS-2008-63.pdf","DOI":"10.21236\/ADA519143"},{"key":"16_CR4","unstructured":"Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A.V., Criminisi, A.: Measuring neural net robustness with constraints. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 2621\u20132629. NIPS\u201916, Curran Associates Inc., Red Hook, NY, USA (2016)"},{"key":"16_CR5","doi-asserted-by":"publisher","unstructured":"Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: YOLOv4: optimal speed and accuracy of object detection (2020). https:\/\/doi.org\/10.48550\/arXiv.2004.10934","DOI":"10.48550\/arXiv.2004.10934"},{"key":"16_CR6","doi-asserted-by":"publisher","unstructured":"Carlini, N., et al.: On evaluating adversarial robustness (2019). https:\/\/doi.org\/10.48550\/arXiv.1902.06705","DOI":"10.48550\/arXiv.1902.06705"},{"key":"16_CR7","doi-asserted-by":"publisher","unstructured":"Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39\u201357 (2017). https:\/\/doi.org\/10.1109\/SP.2017.49","DOI":"10.1109\/SP.2017.49"},{"key":"16_CR8","doi-asserted-by":"publisher","unstructured":"Cho, J.H., Hurley, P.M., Xu, S.: Metrics and measurement of trustworthy systems. In: MILCOM 2016 - 2016 IEEE Military Communications Conference, pp. 1237\u20131242 (2016). https:\/\/doi.org\/10.1109\/MILCOM.2016.7795500","DOI":"10.1109\/MILCOM.2016.7795500"},{"key":"16_CR9","unstructured":"Cohen, J., Rosenfeld, E., Kolter, Z.: Certified adversarial robustness via randomized smoothing. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.\u00a097, pp. 1310\u20131320 (2019). https:\/\/proceedings.mlr.press\/v97\/cohen19c.html"},{"key":"16_CR10","doi-asserted-by":"publisher","unstructured":"Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2019). https:\/\/doi.org\/10.48550\/arXiv.1810.04805","DOI":"10.48550\/arXiv.1810.04805"},{"key":"16_CR11","doi-asserted-by":"publisher","unstructured":"Finlayson, S.G., Chung, H.W., Kohane, I.S., Beam, A.L.: Adversarial attacks against medical deep learning systems (2019). https:\/\/doi.org\/10.48550\/arXiv.1804.05296","DOI":"10.48550\/arXiv.1804.05296"},{"key":"16_CR12","unstructured":"FIRST: Common Vulnerability Scoring System version 4.0 Specification Document (2024). https:\/\/www.first.org\/cvss\/v4-0\/. Accessed 07 Feb 2025"},{"key":"16_CR13","unstructured":"Gilmer, J., Ford, N., Carlini, N., Cubuk, E.: Adversarial examples are a natural consequence of test error in noise. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.\u00a097, pp. 2280\u20132289 (2019). https:\/\/proceedings.mlr.press\/v97\/gilmer19a.html"},{"key":"16_CR14","doi-asserted-by":"publisher","unstructured":"Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2015). https:\/\/doi.org\/10.48550\/arXiv.1412.6572","DOI":"10.48550\/arXiv.1412.6572"},{"issue":"1","key":"16_CR15","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1029\/WR018i001p00014","volume":"18","author":"T Hashimoto","year":"1982","unstructured":"Hashimoto, T., Stedinger, J.R., Loucks, D.P.: Reliability, resiliency, and vulnerability criteria for water resource system performance evaluation. Water Resour. Res. 18(1), 14\u201320 (1982). https:\/\/doi.org\/10.1029\/WR018i001p00014","journal-title":"Water Resour. Res."},{"key":"16_CR16","doi-asserted-by":"crossref","unstructured":"Jansen, W.: Directions in security metrics research. Diane Publishing (2010). https:\/\/nvlpubs.nist.gov\/nistpubs\/legacy\/ir\/nistir7564.pdf","DOI":"10.6028\/NIST.IR.7564"},{"key":"16_CR17","doi-asserted-by":"publisher","unstructured":"Katz, G., Barrett, C., Dill, D.L., Julian, K., Kochenderfer, M.J.: Reluplex: an efficient SMT solver for verifying deep neural networks. In: Majumdar, R., Kun\u010dak, V. (eds.) Computer Aided Verification, pp. 97\u2013117. Springer International Publishing, Cham (2017). https:\/\/doi.org\/10.1007\/978-3-319-63387-9_5","DOI":"10.1007\/978-3-319-63387-9_5"},{"key":"16_CR18","doi-asserted-by":"publisher","unstructured":"Kaziakhmedov, E., Kireev, K., Melnikov, G., Pautov, M., Petiushko, A.: Real-world attack on MTCNN face detection system. In: 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON), pp. 0422\u20130427 (2019). https:\/\/doi.org\/10.1109\/SIBIRCON48586.2019.8958122","DOI":"10.1109\/SIBIRCON48586.2019.8958122"},{"key":"16_CR19","doi-asserted-by":"publisher","unstructured":"Kim, H.: Torchattacks: a pytorch repository for adversarial attacks (2020). https:\/\/doi.org\/10.48550\/arXiv.2010.01950","DOI":"10.48550\/arXiv.2010.01950"},{"key":"16_CR20","unstructured":"Krizhevsky, A.: Learning multiple layers of features from tiny images (2009). https:\/\/www.cs.utoronto.ca\/~kriz\/learning-features-2009-TR.pdf"},{"key":"16_CR21","unstructured":"Kurakin, A., Goodfellow, I.J., Bengio, S.: Adversarial machine learning at scale. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net (2017). https:\/\/openreview.net\/forum?id=BJm4T4Kgx"},{"key":"16_CR22","doi-asserted-by":"publisher","unstructured":"Lecuyer, M., Atlidakis, V., Geambasu, R., Hsu, D., Jana, S.: Certified robustness to adversarial examples with differential privacy. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 656\u2013672 (2019). https:\/\/doi.org\/10.1109\/SP.2019.00044","DOI":"10.1109\/SP.2019.00044"},{"key":"16_CR23","doi-asserted-by":"publisher","unstructured":"Liu, Y., Chen, X., Liu, C., Song, D.: Delving into transferable adversarial examples and black-box attacks (2017). https:\/\/doi.org\/10.48550\/arXiv.1611.02770","DOI":"10.48550\/arXiv.1611.02770"},{"key":"16_CR24","unstructured":"Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (2018). https:\/\/openreview.net\/forum?id=rJzIBfZAb"},{"issue":"3","key":"16_CR25","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1029\/2000WR900329","volume":"37","author":"HR Maier","year":"2001","unstructured":"Maier, H.R., Lence, B.J., Tolson, B.A., Foschi, R.O.: First-order reliability method for estimating reliability, vulnerability, and resilience. Water Resour. Res. 37(3), 779\u2013790 (2001)","journal-title":"Water Resour. Res."},{"issue":"2","key":"16_CR26","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1002\/2017EF000649","volume":"6","author":"C McPhail","year":"2018","unstructured":"McPhail, C., Maier, H.R., Kwakkel, J.H., Giuliani, M., Castelletti, A., Westra, S.: Robustness metrics: how are they calculated, when should they be used and why do they give different results? Earth\u2019s Fut. 6(2), 169\u2013191 (2018)","journal-title":"Earth\u2019s Fut."},{"key":"16_CR27","doi-asserted-by":"publisher","unstructured":"Moosavi-Dezfooli, S.M., Fawzi, A., Frossard, P.: Deepfool: A simple and accurate method to fool deep neural networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2574\u20132582 (2016). https:\/\/doi.org\/10.1109\/CVPR.2016.282","DOI":"10.1109\/CVPR.2016.282"},{"key":"16_CR28","doi-asserted-by":"publisher","unstructured":"Papernot, N., McDaniel, P., Goodfellow, I.: Transferability in machine learning: from phenomena to black-box attacks using adversarial samples (2016). https:\/\/doi.org\/10.48550\/arXiv.1605.07277","DOI":"10.48550\/arXiv.1605.07277"},{"key":"16_CR29","doi-asserted-by":"publisher","unstructured":"Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506\u2013519. ASIA CCS \u201917, Association for Computing Machinery, New York, NY, USA (2017). https:\/\/doi.org\/10.1145\/3052973.3053009","DOI":"10.1145\/3052973.3053009"},{"key":"16_CR30","doi-asserted-by":"publisher","unstructured":"Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 372\u2013387 (2016). https:\/\/doi.org\/10.1109\/EuroSP.2016.36","DOI":"10.1109\/EuroSP.2016.36"},{"key":"16_CR31","volume-title":"PyTorch: an imperative style, high-performance deep learning library","author":"A Paszke","year":"2019","unstructured":"Paszke, A., et al.: PyTorch: an imperative style, high-performance deep learning library. Curran Associates Inc., Red Hook, NY, USA (2019)"},{"key":"16_CR32","unstructured":"Raghunathan, A., Steinhardt, J., Liang, P.: Certified defenses against adversarial examples. In: International Conference on Learning Representations (2018). https:\/\/openreview.net\/forum?id=Bys4ob-Rb"},{"issue":"3","key":"16_CR33","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1016\/j.eng.2019.12.012","volume":"6","author":"K Ren","year":"2020","unstructured":"Ren, K., Zheng, T., Qin, Z., Liu, X.: Adversarial attacks and defenses in deep learning. Engineering 6(3), 346\u2013360 (2020). https:\/\/doi.org\/10.1016\/j.eng.2019.12.012","journal-title":"Engineering"},{"key":"16_CR34","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684\u201310695 (2022)","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"16_CR35","unstructured":"Smith, L., Gal, Y.: Understanding measures of uncertainty for adversarial example detection. In: Globerson, A., Silva, R. (eds.) Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, Monterey, California, USA, August 6-10, 2018. pp, 560\u2013569. AUAI Press (2018), http:\/\/auai.org\/uai2018\/proceedings\/papers\/207.pdf"},{"key":"16_CR36","doi-asserted-by":"publisher","unstructured":"Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2014). https:\/\/doi.org\/10.48550\/arXiv.1312.6199","DOI":"10.48550\/arXiv.1312.6199"},{"key":"16_CR37","unstructured":"Wang, Y., Jha, S., Chaudhuri, K.: Analyzing the robustness of nearest neighbors to adversarial examples. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.\u00a080, pp. 5133\u20135142. PMLR (2018). https:\/\/proceedings.mlr.press\/v80\/wang18c.html"},{"key":"16_CR38","unstructured":"Weiguang\u00a0Ding, G., Yik Chau\u00a0Lui, K., Jin, X., Wang, L., Huang, R.: On the sensitivity of adversarial robustness to input data distributions. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2019)"},{"key":"16_CR39","unstructured":"Wong, E., Kolter, Z.: Provable defenses against adversarial examples via the convex outer adversarial polytope. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.\u00a080, pp. 5286\u20135295 (2018). https:\/\/proceedings.mlr.press\/v80\/wong18a.html"},{"key":"16_CR40","unstructured":"Xiao, K.Y., Tjeng, V., Shafiullah, N.M.M., Madry, A.: Training for faster adversarial robustness verification via inducing reLU stability. In: International Conference on Learning Representations (2019). https:\/\/openreview.net\/forum?id=BJfIVjAcKm"},{"key":"16_CR41","doi-asserted-by":"crossref","unstructured":"Yu, F., Qin, Z., Liu, C., Zhao, L., Wang, Y., Chen, X.: Interpreting and evaluating neural network robustness. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence, pp. 4199\u20134205. IJCAI\u201919, AAAI Press (2019)","DOI":"10.24963\/ijcai.2019\/583"},{"issue":"5","key":"16_CR42","doi-asserted-by":"publisher","first-page":"1071","DOI":"10.1109\/TIFS.2016.2516916","volume":"11","author":"M Zhang","year":"2016","unstructured":"Zhang, M., Wang, L., Jajodia, S., Singhal, A., Albanese, M.: Network diversity: a security metric for evaluating the resilience of networks against zero-day attacks. IEEE Trans. Inf. Forensics Secur. 11(5), 1071\u20131086 (2016). https:\/\/doi.org\/10.1109\/TIFS.2016.2516916","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"16_CR43","doi-asserted-by":"publisher","unstructured":"Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J.Z., Fredrikson, M.: Universal and transferable adversarial attacks on aligned language models (2023). https:\/\/doi.org\/10.48550\/arXiv.2307.15043","DOI":"10.48550\/arXiv.2307.15043"}],"container-title":["Lecture Notes in Computer Science","Availability, Reliability and Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-032-00642-4_16","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,5]],"date-time":"2026-08-05T08:22:40Z","timestamp":1785918160000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-032-00642-4_16"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"ISBN":["9783032006417","9783032006424"],"references-count":43,"URL":"https:\/\/doi.org\/10.1007\/978-3-032-00642-4_16","relation":{},"ISSN":["0302-9743","1611-3349"],"issn-type":[{"value":"0302-9743","type":"print"},{"value":"1611-3349","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025]]},"assertion":[{"value":"10 August 2025","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"ARES","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"International Conference on Availability, Reliability and Security","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Ghent","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Belgium","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2025","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"11 August 2025","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"14 August 2025","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"20","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"ares-12025","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"https:\/\/2025.ares-conference.eu","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}}]}}