{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T16:22:28Z","timestamp":1771950148955,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T00:00:00Z","timestamp":1586217600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T00:00:00Z","timestamp":1586217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. on Info. Security"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Despite the impressive performances reported by deep neural networks in different application domains, they remain largely vulnerable to adversarial examples, i.e., input samples that are carefully perturbed to cause misclassification at test time. In this work, we propose a deep neural rejection mechanism to detect adversarial examples, based on the idea of rejecting samples that exhibit anomalous feature representations at different network layers. With respect to competing approaches, our method does not require generating adversarial examples at training time, and it is less computationally demanding. To properly evaluate our method, we define an adaptive white-box attack that is aware of the defense mechanism and aims to bypass it. Under this worst-case setting, we empirically show that our approach outperforms previously proposed methods that detect adversarial examples by only analyzing the feature representation provided by the output network layer.<\/jats:p>","DOI":"10.1186\/s13635-020-00105-y","type":"journal-article","created":{"date-parts":[[2020,4,7]],"date-time":"2020-04-07T11:03:33Z","timestamp":1586257413000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Deep neural rejection against adversarial examples"],"prefix":"10.1186","volume":"2020","author":[{"given":"Angelo","family":"Sotgiu","sequence":"first","affiliation":[]},{"given":"Ambra","family":"Demontis","sequence":"additional","affiliation":[]},{"given":"Marco","family":"Melis","sequence":"additional","affiliation":[]},{"given":"Battista","family":"Biggio","sequence":"additional","affiliation":[]},{"given":"Giorgio","family":"Fumera","sequence":"additional","affiliation":[]},{"given":"Xiaoyi","family":"Feng","sequence":"additional","affiliation":[]},{"given":"Fabio","family":"Roli","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,4,7]]},"reference":[{"key":"105_CR1","doi-asserted-by":"crossref","unstructured":"A. D. Joseph, B. Nelson, B. I. P. Rubinstein, J. Tygar, Adversarial machine learning (Cambridge University Press, 2018).","DOI":"10.1017\/9781107338548"},{"key":"105_CR2","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1016\/j.patcog.2018.07.023","volume":"84","author":"B. Biggio","year":"2018","unstructured":"B. Biggio, F. Roli, Wild patterns: ten years after the rise of adversarial machine learning. Pattern. Recog.84:, 317\u2013331 (2018).","journal-title":"Pattern. Recog."},{"key":"105_CR3","doi-asserted-by":"crossref","unstructured":"N Dalvi, P Domingos, Mausam, S Sanghai, D Verma, in Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). Adversarial classification (Seattle, 2004), pp. 99\u2013108.","DOI":"10.1145\/1014052.1014066"},{"key":"105_CR4","volume-title":"Second Conference on Email and Anti-Spam (CEAS)","author":"D. Lowd","year":"2005","unstructured":"D. Lowd, C. Meek, in Second Conference on Email and Anti-Spam (CEAS). Good word attacks on statistical spam filters (Mountain ViewUSA, 2005)."},{"key":"105_CR5","unstructured":"B. Biggio, B. Nelson, P. Laskov, in 29th Int\u2019l Conf. on Machine Learning, ed. by J. Langford, J. Pineau. Poisoning attacks against support vector machines (Omnipress, 2012), pp. 1807\u20131814."},{"key":"105_CR6","doi-asserted-by":"crossref","unstructured":"B Biggio, I Corona, D Maiorca, B Nelson, \u0160rndi\u0107, P Laskov, G Giacinto, F Roli, in Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Part III, 8190, ed. by H. Blockeel, K. Kersting, S. Nijssen, and F. \u017eelezn\u00fd. Evasion attacks against machine learning at test time, LNCS (Springer Berlin Heidelberg, 2013), pp. 387\u2013402.","DOI":"10.1007\/978-3-642-40994-3_25"},{"key":"105_CR7","volume-title":"International Conference on Learning Representations","author":"C. Szegedy","year":"2014","unstructured":"C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, R. Fergus, in International Conference on Learning Representations. Intriguing properties of neural networks (ICLRCalgary, 2014)."},{"key":"105_CR8","volume-title":"International Conference on Learning Representations","author":"I. J. Goodfellow","year":"2015","unstructured":"I. J. Goodfellow, J. Shlens, C. Szegedy, in International Conference on Learning Representations. Explaining and harnessing adversarial examples (ICLRSan Diego, 2015)."},{"key":"105_CR9","doi-asserted-by":"publisher","unstructured":"A. Globerson, S. T Roweis, in Proceedings of the 23rd International Conference on Machine Learning, ed. by W. W. Cohen, A. Moore. Nightmare at test time: robust learning by feature deletion, vol. 148 (ACM, 2006), pp. 353\u2013360. https:\/\/doi.org\/10.1145\/1143844.1143889.","DOI":"10.1145\/1143844.1143889"},{"key":"105_CR10","unstructured":"M. Br\u00fcckner, C. Kanzow, T. Scheffer, Static prediction games for adversarial learning problems. J. Mach. Learn. Res.13, 2617\u20132654 (2012)."},{"key":"105_CR11","doi-asserted-by":"crossref","unstructured":"S. Rota Bul\u00f2, B. Biggio, I. Pillai, M. Pelillo, F. Roli, Randomized prediction games for adversarial machine learning. IEEE Trans. Neural Netw. Learn. Syst.28(11), 2466\u20132478 (2017).","DOI":"10.1109\/TNNLS.2016.2593488"},{"key":"105_CR12","doi-asserted-by":"crossref","unstructured":"M. Melis, A. Demontis, B. Biggio, G. Brown, G. Fumera, F. Roli, in ICCVW Vision in Practice on Autonomous Robots (ViPAR). Is deep learning safe for robot vision? Adversarial examples against the iCub humanoid (IEEE, 2017), pp. 751\u2013759. https:\/\/doi.org\/10.1109\/iccvw.2017.94.","DOI":"10.1109\/ICCVW.2017.94"},{"key":"105_CR13","doi-asserted-by":"publisher","unstructured":"A. Bendale, T. E. Boult, in IEEE Conference on Computer Vision and Pattern Recognition. Towards open set deep networks, (2016), pp. 1563\u20131572. https:\/\/doi.org\/10.1109\/cvpr.2016.173.","DOI":"10.1109\/cvpr.2016.173"},{"key":"105_CR14","unstructured":"F. Crecchi, D. Bacciu, B. Biggio, in ESANN \u201919. Detecting adversarial examples through nonlinear dimensionality reduction. In press."},{"key":"105_CR15","doi-asserted-by":"publisher","unstructured":"J. Lu, T. Issaranon, D. Forsyth, in The IEEE International Conference on Computer Vision (ICCV). Safetynet: detecting and rejecting adversarial examples robustly, (2017). https:\/\/doi.org\/10.1109\/iccv.2017.56.","DOI":"10.1109\/iccv.2017.56"},{"key":"105_CR16","unstructured":"N. Papernot, P. D. McDaniel, Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning. CoRR. abs\/1803.04765 (2018)."},{"key":"105_CR17","unstructured":"A. Athalye, N. Carlini, D. A. Wagner, in ICML, vol. 80 of JMLR Workshop and Conference Proceedings. Obfuscated gradients give a false sense of security: circumventing defenses to adversarial examples (JMLR.org, 2018), pp. 274\u2013283."},{"key":"105_CR18","doi-asserted-by":"publisher","unstructured":"N. Papernot, P. McDaniel, X. Wu, S. Jha, A. Swami, in 2016 IEEE Symposium on Security and Privacy (SP). Distillation as a defense to adversarial perturbations against deep neural networks, (2016), pp. 582\u2013597. https:\/\/doi.org\/10.1109\/sp.2016.41.","DOI":"10.1109\/sp.2016.41"},{"key":"105_CR19","doi-asserted-by":"publisher","unstructured":"D. Meng, H. Chen, in 24th ACM Conf. Computer and Comm. Sec. (CCS). MagNet: a two-pronged defense against adversarial examples, (2017). https:\/\/doi.org\/10.1145\/3133956.3134057.","DOI":"10.1145\/3133956.3134057"},{"key":"105_CR20","first-page":"3","volume-title":"10th ACM Workshop on Artificial Intelligence and Security","author":"N. Carlini","year":"2017","unstructured":"N. Carlini, D. A. Wagner, in 10th ACM Workshop on Artificial Intelligence and Security, ed. by B. M. Thuraisingham, B. Biggio, D. M. Freeman, B. Miller, and A. Sinha. Adversarial examples are not easily detected: bypassing ten detection methods, AISec \u201917 (ACMNew York, 2017), pp. 3\u201314."},{"key":"105_CR21","doi-asserted-by":"publisher","unstructured":"N. Carlini, D. A. Wagner, in IEEE Symposium on Security and Privacy. Towards evaluating the robustness of neural networks (IEEE Computer Society, 2017), pp. 39\u201357. https:\/\/doi.org\/10.1109\/sp.2017.49.","DOI":"10.1109\/sp.2017.49"},{"key":"105_CR22","first-page":"59","volume-title":"9th ACM Workshop on Artificial Intelligence and Security","author":"P. Russu","year":"2016","unstructured":"P. Russu, A. Demontis, B. Biggio, G. Fumera, F. Roli, in 9th ACM Workshop on Artificial Intelligence and Security. Secure kernel machines against evasion attacks, AISec \u201916 (ACMNew York, 2016), pp. 59\u201369."},{"key":"105_CR23","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1145\/3052973.3053009","volume-title":"Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security","author":"N. Papernot","year":"2017","unstructured":"N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, A. Swami, in Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. Practical black-box attacks against machine learning, ASIA CCS \u201917 (ACMNew York, 2017), pp. 506\u2013519."},{"key":"105_CR24","unstructured":"A. Demontis, M. Melis, M. Pintor, M. Jagielski, B. Biggio, A. Oprea, C. Nita-Rotaru, F. Roli, in 28th USENIX Security Symposium (USENIX Security 19). Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks (USENIX Association, 2019)."},{"key":"105_CR25","first-page":"524","volume-title":"26th European Signal Processing Conf","author":"M. Melis","year":"2018","unstructured":"M. Melis, D. Maiorca, B. Biggio, G. Giacinto, F. Roli, in 26th European Signal Processing Conf. Explaining black-box android malware detection, EUSIPCO (IEEERome, 2018), pp. 524\u2013528."},{"key":"105_CR26","doi-asserted-by":"crossref","unstructured":"B. Biggio, G. Fumera, F. Roli, Security evaluation of pattern classifiers under attack. IEEE Trans. Knowl. Data Eng.26, 984\u2013996 (2014).","DOI":"10.1109\/TKDE.2013.57"},{"key":"105_CR27","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","volume":"5","author":"D. H. Wolpert","year":"1992","unstructured":"D. H. Wolpert, Stacked generalization. Neural Netw.5:, 241\u2013259 (1992).","journal-title":"Neural Netw."},{"key":"105_CR28","doi-asserted-by":"crossref","unstructured":"W. Scheirer, L. Jain, T. Boult, Probability models for open set recognition. IEEE Trans. Patt. An. Mach. Intell. 36(11), 2317\u20132324 (2014).","DOI":"10.1109\/TPAMI.2014.2321392"},{"key":"105_CR29","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1145\/1390156.1390191","volume-title":"Proceedings of the 25th International Conference on Machine Learning, ICML \u201908","author":"J. Duchi","year":"2008","unstructured":"J. Duchi, S. Shalev-Shwartz, Y. Singer, T. Chandra, in Proceedings of the 25th International Conference on Machine Learning, ICML \u201908. Efficient projections onto the l1-ball for learning in high dimensions (ACMNew York, 2008), pp. 272\u2013279."},{"key":"105_CR30","unstructured":"M. Melis, A. Demontis, M. Pintor, A. Sotgiu, B. Biggio, secml: A Python library for secure and explainable machine learning. arXiv (2019)."},{"key":"105_CR31","unstructured":"S. Thulasidasan, T. Bhattacharya, J. Bilmes, G. Chennupati, J. Mohd-Yusof, Knows when it doesn\u2019t know: deep abstaining classifiers, (OpenReview.net, 2019). https:\/\/openreview.net\/forum?id=rJxF73R9tX."},{"key":"105_CR32","first-page":"2151","volume-title":"Proceedings of the 36th International Conference on Machine Learning, (ICML) 2019","author":"R. E. -Y. Yonatan Geifman","year":"2019","unstructured":"R. E. -Y. Yonatan Geifman, in Proceedings of the 36th International Conference on Machine Learning, (ICML) 2019, 97. Selectivenet: a deep neural network with an integrated reject option (PMLRLong Beach, 2019), pp. 2151\u20132159."},{"key":"105_CR33","unstructured":"Y. Geifman, R. El-Yaniv, in Advances in Neural Information Processing Systems 30, ed. by I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett. Selective classification for deep neural networks (Curran Associates, Inc., 2017), pp. 4878\u20134887."},{"key":"105_CR34","doi-asserted-by":"publisher","unstructured":"F. Carrara, R. Becarelli, R. Caldelli, F. Falchi, G. Amato, in The European Conference on Computer Vision (ECCV) Workshops. Adversarial examples detection in features distance spaces, (2018). https:\/\/doi.org\/10.1007\/978-3-030-11012-3_26.","DOI":"10.1007\/978-3-030-11012-3_26"},{"key":"105_CR35","unstructured":"T. Pang, C. Du, J. Zhu, in Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan, Stockholm, Sweden, July 10-15, 2018. Max-mahalanobis linear discriminant analysis networks (PMLR, 2018), pp. 4013\u20134022."},{"key":"105_CR36","doi-asserted-by":"publisher","unstructured":"M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-Rotaru, B. Li, in IEEE Symposium on Security and Privacy, SP \u201918. Manipulating machine learning: poisoning attacks and countermeasures for regression learning (IEEE CS, 2018), pp. 931\u2013947. https:\/\/doi.org\/10.1109\/sp.2018.00057.","DOI":"10.1109\/sp.2018.00057"},{"key":"105_CR37","unstructured":"H. Xiao, B. Biggio, G. Brown, G. Fumera, C. Eckert, F. Roli, in JMLR W&CP - Proc. 32nd Int\u2019l Conf. Mach. Learning (ICML), ed. by F. Bach, D. Blei. Is feature selection secure against training data poisoning? vol. 37 (PMLRLille, 2015), pp. 1689\u20131698."},{"key":"105_CR38","doi-asserted-by":"crossref","unstructured":"S. Mei, X. Zhu, in 29th AAAI Conf. Artificial Intelligence (AAAI \u201915). Using machine teaching to identify optimal training-set attacks on machine learners (Austin, 2015).","DOI":"10.1609\/aaai.v29i1.9569"},{"key":"105_CR39","unstructured":"Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition. Proc. IEEE. 86, 2278\u20132324 (1998)."},{"key":"105_CR40","unstructured":"A. Krizhevsky, Learning multiple layers of features from tiny images (University of Toronto, 2012)."}],"container-title":["EURASIP Journal on Information Security"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13635-020-00105-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s13635-020-00105-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13635-020-00105-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,20]],"date-time":"2022-10-20T20:36:05Z","timestamp":1666298165000},"score":1,"resource":{"primary":{"URL":"https:\/\/jis-eurasipjournals.springeropen.com\/articles\/10.1186\/s13635-020-00105-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,7]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["105"],"URL":"https:\/\/doi.org\/10.1186\/s13635-020-00105-y","relation":{},"ISSN":["2510-523X"],"issn-type":[{"value":"2510-523X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,7]]},"assertion":[{"value":"30 September 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 March 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 April 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"5"}}