{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T02:29:00Z","timestamp":1790908140380,"version":"4.1.0"},"reference-count":59,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"4","funder":[{"DOI":"10.13039\/501100003443","name":"Ministry of Education and Science of the Russian Federation","doi-asserted-by":"publisher","award":["14.756.31.0001"],"award-info":[{"award-number":["14.756.31.0001"]}],"id":[{"id":"10.13039\/501100003443","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100007183","name":"University of California, Santa Barbara","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007183","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Journal on Mathematics of Data Science"],"published-print":{"date-parts":[[2020,1]]},"abstract":"<jats:p>Active subspace is a model reduction method widely used in the uncertainty quantification community. In this paper, we propose analyzing the internal structure and vulnerability of deep neural networks using active subspace. Firstly, we employ the active subspace to measure the number of \u201cactive neurons\u201d at each intermediate layer, which indicates that the number of neurons can be reduced from several thousands to several dozens. This motivates us to change the network structure and to develop a new and more compact network, referred to as ASNet, that has significantly fewer model parameters. Secondly, we propose analyzing the vulnerability of a neural network using active subspace by finding an additive universal adversarial attack vector that can misclassify a dataset with a high probability. Our experiments on CIFAR-10 show that ASNet can achieve 23.98x parameter and 7.30x flops reduction. The universal active subspace attack vector can achieve around 20% higher attack ratio compared with the existing approaches in our numerical experiments. The PyTorch codes for this paper are available online.<\/jats:p>","DOI":"10.1137\/19m1296070","type":"journal-article","created":{"date-parts":[[2020,10,29]],"date-time":"2020-10-29T10:44:17Z","timestamp":1603968257000},"page":"1096-1122","source":"Crossref","is-referenced-by-count":20,"title":["Active Subspace of Neural Networks: Structural Analysis and Universal Attacks"],"prefix":"10.1137","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6815-4060","authenticated-orcid":true,"given":"Chunfeng","family":"Cui","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaiqi","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Talgat","family":"Daulbaev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julia","family":"Gusak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ivan","family":"Oseledets","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zheng","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2020,10,29]]},"reference":[{"key":"atypb1","unstructured":"S. Abdoli, L. G. Hafemann, J. Rony, I. B. Ayed, P. Cardinal, and A. L. Koerich,\n                      Universal Adversarial Audio Perturbations\n                      , arXiv preprint,arXiv:1908.03173, 2019."},{"key":"atypb2","first-page":"3177","author":"Aghasi A.","year":"2017","journal-title":"Proceedings of the Conference on Neural Information Processing Systems"},{"key":"atypb3","doi-asserted-by":"crossref","unstructured":"L. Armijo,\n                      Minimization of functions having Lipschitz continuous first partial derivatives\n                      , Pacific J. mathematics, 16 (1966), pp. 1-3.","DOI":"10.2140\/pjm.1966.16.1"},{"key":"atypb4","doi-asserted-by":"crossref","unstructured":"S. Baluja and I. Fischer,\n                      Adversarial Transformation Networks: Learning to Generate Adversarial Examples\n                      , arXiv preprint,arXiv:1703.09387, 2017.","DOI":"10.1609\/aaai.v32i1.11672"},{"key":"atypb5","first-page":"7345","author":"Behjati M.","year":"2019","journal-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623497330963"},{"key":"atypb7","unstructured":"H. Cai, L. Zhu, and S. Han,\n                      ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware\n                      , arXiv preprint,arXiv:1812.00332, 2018."},{"key":"atypb8","first-page":"39","author":"Carlini N.","year":"2017","journal-title":"IEEE"},{"key":"atypb9","doi-asserted-by":"crossref","unstructured":"P. G. Constantine,\n                      Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies\n                      , SIAM, Philadelphia, PA, 2015.","DOI":"10.1137\/1.9781611973860"},{"key":"atypb10","doi-asserted-by":"publisher","DOI":"10.1137\/130916138"},{"key":"atypb11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jcp.2015.09.001","volume":"302","author":"Constantine P. G.","year":"2015","journal-title":"J. Comput. Phys."},{"key":"atypb12","unstructured":"M. Courbariaux, I. Hubara, D. Soudry, R. El-Yaniv, and Y. Bengio,\n                      Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1\n                      , arXiv preprint,arXiv:1602.02830, 2016."},{"key":"atypb13","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1109\/TCPMT.2018.2889266","volume":"9","author":"Cui C.","year":"2019","journal-title":"IEEE Trans. Components Packaging Manuf. Tech."},{"key":"atypb14","doi-asserted-by":"crossref","first-page":"1649","DOI":"10.1109\/TCAD.2019.2925340","volume":"39","author":"Cui C.","year":"2020","journal-title":"IEEE Trans. Comput-Aided Design Integr. Circuits Syst."},{"key":"atypb15","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.neunet.2018.01.010","volume":"100","author":"Deng L.","year":"2018","journal-title":"Neural Networks"},{"key":"atypb16","unstructured":"G. K. Dziugaite, Z. Ghahramani, and D. M. Roy,\n                      A Study of the Effect of JPG Compression on Adversarial Images\n                      , arXiv preprint,arXiv:1608.00853, 2016."},{"key":"atypb17","unstructured":"J. Frankle and M. Carbin,\n                      The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks\n                      , arXiv preprint,arXiv:1803.03635, 2018."},{"key":"atypb18","unstructured":"T. Garipov, D. Podoprikhin, A. Novikov, and D. Vetrov,\n                      Ultimate Tensorization: Compressing Convolutional and FC Layers Alike\n                      , arXiv preprint,arXiv:1611.03214, 2016."},{"key":"atypb19","unstructured":"R. Ge, R. Wang, and H. Zhao,\n                      Mildly Overparametrized Neural Nets Can Memorize Training Data Efficiently\n                      , arXiv preprint,arXiv:1909.11837, 2019."},{"key":"atypb20","first-page":"101","author":"Ghanem R. G.","year":"1991","journal-title":"Cham"},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1137\/15M1009718"},{"key":"atypb22","unstructured":"I. J. Goodfellow, J. Shlens, and C. Szegedy,\n                      Explaining and Harnessing Adversarial Examples\n                      , arXiv preprint,arXiv:1412.6572, 2014."},{"key":"atypb23","first-page":"369","author":"Graves A.","year":"2006","journal-title":"ACM"},{"key":"atypb24","doi-asserted-by":"publisher","DOI":"10.1137\/090771806"},{"key":"atypb25","unstructured":"S. Han, H. Mao, and W. J. Dally,\n                      Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding\n                      , arXiv preprint,arXiv:1510.00149, 2015."},{"key":"atypb26","unstructured":"C. Hawkins and Z. Zhang,\n                      Bayesian Tensorized Neural Networks with Automatic Rank Selection\n                      , arXiv preprint,arXiv:1905.10478, 2019."},{"key":"atypb27","doi-asserted-by":"crossref","unstructured":"Y. He, J. Lin, Z. Liu, H. Wang, L.J. Li, and S. Han,\n                      AMC: AutoML for model compression and acceleration on mobile devices\n                      , in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 784-800.","DOI":"10.1007\/978-3-030-01234-2_48"},{"key":"atypb28","unstructured":"G. Hinton, O. Vinyals, and J. Dean,\n                      Distilling the Knowledge in a Neural Network\n                      , arXiv preprint,arXiv:1503.02531, 2015."},{"key":"atypb29","first-page":"409","author":"Hoeffding W.","year":"1994","journal-title":"Cham"},{"key":"atypb30","doi-asserted-by":"crossref","unstructured":"D. W. Hosmer, Jr., S. Lemeshow, and R. X. Sturdivant,\n                      Applied Logistic Regression\n                      , John Wiley & Sons, New York, 2013.","DOI":"10.1002\/9781118548387"},{"key":"atypb31","first-page":"1094","author":"Jolliffe I.","year":"2011","journal-title":"Cham"},{"key":"atypb32","first-page":"4441","author":"Kanbak C.","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb33","first-page":"8562","author":"Khrulkov V.","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb34","unstructured":"D. P. Kingma and J. Ba,\n                      Adam: A Method for Stochastic Optimization\n                      , arXiv preprint,arXiv:1412.6980, 2014."},{"key":"atypb35","first-page":"1097","author":"Krizhevsky A.","year":"2012","journal-title":"Proceedings of the Conference on Neural Information Processing Systems"},{"key":"atypb36","unstructured":"V. Lebedev, Y. Ganin, M. Rakhuba, I. Oseledets, and V. Lempitsky,\n                      Speeding-up Convolutional Neural Networks Using Fine-Tuned CP-Decomposition\n                      , arXiv preprint,arXiv:1412.6553, 2014."},{"key":"atypb37","first-page":"135","author":"Lide D. R.","year":"2018","journal-title":"Boca Raton, FL"},{"key":"atypb38","unstructured":"L. Liu, L. Deng, X. Hu, M. Zhu, G. Li, Y. Ding, and Y. Xie,\n                      Dynamic Sparse Graph for Efficient Deep Learning\n                      , arXiv preprint,arXiv:1810.00859, 2018."},{"key":"atypb39","unstructured":"Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell,\n                      Rethinking the Value of Network Pruning\n                      , arXiv preprint,arXiv:1810.05270, 2018."},{"key":"atypb40","first-page":"1765","author":"Moosavi-Dezfooli S.-M.","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb41","first-page":"2574","author":"Moosavi-Dezfooli S.-M.","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"atypb42","doi-asserted-by":"crossref","unstructured":"P. Neekhara, S. Hussain, P. Pandey, S. Dubnov, J. McAuley, and F. Koushanfar,\n                      Universal Adversarial Perturbations for Speech Recognition Systems\n                      , arXiv preprint,arXiv:1905.03828, 2019.","DOI":"10.21437\/Interspeech.2019-1353"},{"key":"atypb43","first-page":"442","author":"Novikov A.","year":"2015","journal-title":"Proceedings of the Conference on Neural Information Processing Systems"},{"key":"atypb44","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/JSAIT.2020.2991332","volume":"1","author":"Oymak S.","year":"2020","journal-title":"IEEE J. Sel. Areas Inform. Therory"},{"key":"atypb45","first-page":"372","author":"Papernot N.","year":"2016","journal-title":"IEEE"},{"key":"atypb46","unstructured":"A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio,\n                      FitNets: Hints for Thin Deep Nets\n                      , arXiv preprint,arXiv:1412.6550, 2014."},{"key":"atypb47","unstructured":"H. L. Royden,\n                      Real Analysis\n                      , Macmillan, New York, 2010."},{"key":"atypb48","unstructured":"T. M. Russi,\n                      Uncertainty Quantification with Experimental Data and Complex System Models\n                      , PhD thesis, UC Berkeley, Berkeley, CA, 2010."},{"key":"atypb49","first-page":"6655","author":"Sainath T. N.","year":"2013","journal-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing"},{"key":"atypb50","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.neucom.2017.02.029","volume":"241","author":"Scardapane S.","year":"2017","journal-title":"Neurocomputing"},{"key":"atypb51","first-page":"456","author":"Schmidt M.","year":"2009","journal-title":"Proceedings of the International Conference on Artificial Intelligence and Statistics"},{"key":"atypb52","unstructured":"A. C. Serban and E. Poll,\n                      Adversarial Examples-A Complete Characterisation of the Phenomenon\n                      , arXiv preprint,arXiv:1810.01185, 2018."},{"key":"atypb53","first-page":"64","author":"Shalev-Shwartz S.","year":"2014","journal-title":"Proceedings of the International Conference on Machine Learning"},{"key":"atypb54","unstructured":"C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus,\n                      Intriguing Properties of Neural Networks\n                      , arXiv preprint,arXiv:1312.6199, (2013)."},{"key":"atypb55","doi-asserted-by":"crossref","first-page":"4927","DOI":"10.1016\/S0045-7825(02)00421-8","volume":"191","author":"Xiu D.","year":"2002","journal-title":"Comput. Methods Appl. Mech. Engrg."},{"key":"atypb56","doi-asserted-by":"publisher","DOI":"10.1137\/S1064827501387826"},{"key":"atypb57","unstructured":"S. Ye, X. Feng, T. Zhang, X. Ma, S. Lin, Z. Li, K. Xu, W. Wen, S. Liu, J. Tang, M. Fardad, X. Lin, Y. Liu, and Y. Wang,\n                      Progressive DNN Compression: A Key to Achieve Ultra-High Weight Pruning and Quantization Rates Using ADMM\n                      , arXiv preprint,arXiv:1903.09769, 2019."},{"key":"atypb58","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MCI.2018.2840738","volume":"13","author":"Young T.","year":"2018","journal-title":"IEEE Comput. Intel. Mag."},{"key":"atypb59","unstructured":"V. P. Zankin, G. V. Ryzhakov, and I. Oseledets,\n                      Gradient Descent-Based D-Optimal Design for the Least-Squares Polynomial Approximation\n                      , arXiv preprint,arXiv:1806.06631, 2018."}],"container-title":["SIAM Journal on Mathematics of Data Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/19M1296070","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T16:06:27Z","timestamp":1787328387000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/19M1296070"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1]]},"references-count":59,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,1]]}},"alternative-id":["10.1137\/19M1296070"],"URL":"https:\/\/doi.org\/10.1137\/19m1296070","relation":{},"ISSN":["2577-0187"],"issn-type":[{"value":"2577-0187","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1]]}}}