{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T22:20:52Z","timestamp":1773094852895,"version":"3.50.1"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T00:00:00Z","timestamp":1619568000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005713","name":"Technische Universit\u00e4t M\u00fcnchen","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005713","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Neural networks achieve outstanding accuracy in classification and regression tasks. However, understanding their behavior still remains an open challenge that requires questions to be addressed on the robustness, explainability and reliability of predictions. We answer these questions by computing <jats:italic>reachable sets<\/jats:italic> of neural networks, i.e. sets of outputs resulting from continuous sets of inputs. We provide two efficient approaches that lead to over- and under-approximations of the reachable set. This principle is highly versatile, as we show. First, we use it to analyze and enhance the robustness properties of both classifiers and regression models. This is in contrast to existing works, which are mainly focused on classification. Specifically, we verify (non-)robustness, propose a robust training procedure, and show that our approach outperforms adversarial attacks as well as state-of-the-art methods of verifying classifiers for non-norm bound perturbations. Second, we provide techniques to distinguish between reliable and non-reliable predictions for unlabeled inputs, to quantify the influence of each feature on a prediction, and compute a feature ranking.<\/jats:p>","DOI":"10.1007\/s10994-021-05973-0","type":"journal-article","created":{"date-parts":[[2021,4,28]],"date-time":"2021-04-28T15:59:09Z","timestamp":1619625549000},"page":"1175-1197","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Reachable sets of classifiers and regression models: (non-)robustness analysis and robust training"],"prefix":"10.1007","volume":"110","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2695-0816","authenticated-orcid":false,"given":"Anna-Kathrin","family":"Kopetzki","sequence":"first","affiliation":[]},{"given":"Stephan","family":"G\u00fcnnemann","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,4,28]]},"reference":[{"key":"5973_CR1","unstructured":"Bastani, O., Ioannou, Y., Lampropoulos, L., Vytiniotis, D., Nori, A.\u00a0V., & Criminisi, A. (2016). Measuring neural net robustness with constraints. In NeurIPS, Vol. 29."},{"key":"5973_CR2","unstructured":"Brooks, T.\u00a0F., Pope, D.\u00a0S., & Marcolini, A.\u00a0M. (1989). Airfoil self-noise and prediction. NASA Technical Reports."},{"key":"5973_CR3","unstructured":"Bunel, R., Turkaslan, I., Torr, P.\u00a0H., Kohli, P., & Kumar, M.\u00a0P. (2018). A unified view of piecewise linear neural network verification. In NeurIPS, Vol. 31, PP. 4795\u20134804."},{"key":"5973_CR4","unstructured":"Dua, D., & Graff, C. (2017). UCI machine learning repository. In University of California."},{"key":"5973_CR5","doi-asserted-by":"crossref","unstructured":"Ehlers, R. (2017). Formal verification of piece-wise linear feed-forward neural networks. In Automated Technology for Verification and Analysis, PP. 269\u2013286.","DOI":"10.1007\/978-3-319-68167-2_19"},{"key":"5973_CR6","doi-asserted-by":"crossref","unstructured":"Fisher, R.\u00a0A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics.","DOI":"10.1111\/j.1469-1809.1936.tb02137.x"},{"key":"5973_CR7","unstructured":"Forina, M., Leardi, R., Armanino, C., & Lanteri, S. (1990). Parvus: An extendable package of programs for data exploration. Journal of Chemometrics."},{"key":"5973_CR8","doi-asserted-by":"crossref","unstructured":"Gehr, T., Mirman, M., Drachsler-Cohen, D., Tsankov, P., Chaudhuri, S., & Vechev, M. (2018). Ai2: Safety and robustness certification of neural networks with abstract interpretation. In IEEE Symposium on Security and Privacy, PP. 3\u201318.","DOI":"10.1109\/SP.2018.00058"},{"key":"5973_CR9","unstructured":"Goodfellow, I.\u00a0J., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. ICLR."},{"key":"5973_CR10","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1016\/j.laa.2010.01.031","volume":"433","author":"E Gover","year":"2010","unstructured":"Gover, E., & Krikorian, N. (2010). Determinants and the volumes of parallelotopes and zonotopes. Linear Algebra and its Applications, 433, 28\u201340.","journal-title":"Linear Algebra and its Applications"},{"key":"5973_CR11","doi-asserted-by":"crossref","unstructured":"Gowal, S., Dvijotham, K., Stanforth, R., Bunel, R., Qin, C., Uesato, J., Arandjelovic, R., Mann, R., & Kohli, P. (2019). Scalable verified training for provably robust image classification. In ICCV, PP. 4841\u20134850.","DOI":"10.1109\/ICCV.2019.00494"},{"key":"5973_CR12","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/0095-0696(78)90006-2","volume":"5","author":"D Harrison","year":"1978","unstructured":"Harrison, D., & Rubinfeld, D. L. (1978). Hedonic prices and the demand for clean air. Journal of Environmental Economics and Management, 5, 81\u2013102.","journal-title":"Journal of Environmental Economics and Management"},{"key":"5973_CR13","unstructured":"Hein, M., & Andriushchenko, M. (2017). Formal guarantees on the robustness of a classifier against adversarial manipulation. In NeurIPS, Vol. 30."},{"key":"5973_CR14","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1007\/BF02520002","volume":"34","author":"J Jossinet","year":"1996","unstructured":"Jossinet, J. (1996). Variability of impedivity in normal and pathological breast tissue. Medical and Biological Engineering and Computing, 34, 346\u2013350.","journal-title":"Medical and Biological Engineering and Computing"},{"key":"5973_CR15","first-page":"97","volume":"10426","author":"G Katz","year":"2017","unstructured":"Katz, G., Barrett, C. W., Dill, D. L., Julian, K., & Kochenderfer, M. J. (2017). Reluplex: An efficient SMT solver for verifying deep neural networks. CAV, 10426, 97\u2013117.","journal-title":"CAV"},{"key":"5973_CR16","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1007\/BF02684450","volume":"61","author":"W K\u00fchn","year":"1998","unstructured":"K\u00fchn, W. (1998). Rigorously computed orbits of dynamical systems without the wrapping effect. Computing, 61, 47\u201367.","journal-title":"Computing"},{"key":"5973_CR17","volume-title":"Mnist handwritten digit database","author":"Y LeCun","year":"2010","unstructured":"LeCun, Y., Cortes, C., & Burges, C. J. (2010). Mnist handwritten digit database. NYU: Courant Institute."},{"key":"5973_CR18","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1561\/2400000035","volume":"4","author":"C Liu","year":"2019","unstructured":"Liu, C., Arnon, T., Lazarus, C., Barrett, C. W., & Kochenderfer, M. J. (2019). Algorithms for verifying deep neural networks. Foundations and Trends in Optimization, 4, 244\u2013404.","journal-title":"Foundations and Trends in Optimization"},{"key":"5973_CR19","unstructured":"Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. ICLR."},{"key":"5973_CR20","first-page":"3578","volume":"80","author":"M Mirman","year":"2018","unstructured":"Mirman, M., Gehr, T., & Vechev, M. (2018). Differential abstract interpretation for provably robust neural networks. ICML, 80, 3578\u20133586.","journal-title":"ICML"},{"key":"5973_CR21","unstructured":"Nash, W.\u00a0J., Sellers, T.\u00a0L., Talbot, S.\u00a0R., Cawthorn, A.\u00a0J., & Ford, W.\u00a0B. (1994). The population biology of abalone (haliotis species) in tasmania. i. Blacklip abalone (h. rubra) from the north coast and islands of bass strait. Sea Fisheries Division, Technical Report, 48."},{"key":"5973_CR22","unstructured":"Raghunathan, A., Steinhardt, J., & Liang, P. (2018). Semidefinite relaxations for certifying robustness to adversarial examples. In NeurIPS, Vol. 31."},{"key":"5973_CR23","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.\u00a0T., Singh, S., & Guestrin, C. (2016). \u201cwhy should i trust you?\u201d: Explaining the predictions of any classifier. In SIGKDD, PP. 1135\u20131144.","DOI":"10.1145\/2939672.2939778"},{"key":"5973_CR24","doi-asserted-by":"crossref","unstructured":"Ruan, W., Huang, X., & Kwiatkowska, M. (2018). Reachability analysis of deep neural networks with provable guarantees. In IJCAI, PP. 2651\u20132659.","DOI":"10.24963\/ijcai.2018\/368"},{"key":"5973_CR25","unstructured":"Singh, G., Ganvir, R., P\u00fcschel, M., & Vechev, M. (2019a). Beyond the single neuron convex barrier for neural network certification. NeurIPS, 32."},{"key":"5973_CR26","unstructured":"Singh, G., Gehr, T., Mirman, M., P\u00fcschel, M., & Vechev, M. (2018). Fast and effective robustness certification. In NeurIPS, Vol. 31."},{"key":"5973_CR27","doi-asserted-by":"crossref","unstructured":"Singh, G., Gehr, T., P\u00fcschel, M., & Vechev, M. (2019). An abstract domain for certifying neural networks. In Proceedings of the ACM Programming Languages, 3.","DOI":"10.1145\/3290354"},{"key":"5973_CR28","unstructured":"Singh, G., Gehr, T., P\u00fcschel, M., & Vechev, M. (2019). Boosting robustness certification of neural networks. ICLR."},{"key":"5973_CR29","unstructured":"Steinhardt, J., Koh, P.\u00a0W., & Liang, P. (2017). Certified defenses for data poisoning attacks. In NeurIPS, Vol. 30, PP. 3520\u20133532."},{"key":"5973_CR30","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1117\/12.148698","volume":"1905","author":"N Street","year":"1999","unstructured":"Street, N., Wolberg, W., & Mangasarian, O. L. (1999). Nuclear feature extraction for breast tumor diagnosis. Biomedical Image Processing and Biomedical Visualization, 1905, 861\u2013870.","journal-title":"Biomedical Image Processing and Biomedical Visualization"},{"key":"5973_CR31","unstructured":"Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. ICML, 79."},{"key":"5973_CR32","unstructured":"Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2014). Intriguing properties of neural networks. ICLR."},{"key":"5973_CR33","unstructured":"Tjeng, V., Xiao, K.\u00a0Y., & Tedrake, R. (2019). Evaluating robustness of neural networks with mixed integer programming. ICLR."},{"key":"5973_CR34","first-page":"5283","volume":"80","author":"E Wong","year":"2018","unstructured":"Wong, E., & Kolter, J. Z. (2018). Provable defenses against adversarial examples via the convex outer adversarial polytope. ICML, 80, 5283\u20135292.","journal-title":"ICML"},{"key":"5973_CR35","unstructured":"Xiang, W., Tran, H.-D., & Johnson, T. (2017). Reachable set computation and safety verification for neural networks with relu activations. CoRR."},{"key":"5973_CR36","unstructured":"Xiao, H., Rasul, K., & Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. Zalando SE."},{"key":"5973_CR37","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.\u00a0N., & Lopez-Paz, D. (2018). Mixup: Beyond empirical risk minimization. ICLR."}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-05973-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-05973-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-05973-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,18]],"date-time":"2021-06-18T15:12:15Z","timestamp":1624029135000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-05973-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,28]]},"references-count":37,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["5973"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-05973-0","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"value":"0885-6125","type":"print"},{"value":"1573-0565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,28]]},"assertion":[{"value":"18 September 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 January 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 March 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 April 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}