{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T06:44:25Z","timestamp":1773384265216,"version":"3.50.1"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T00:00:00Z","timestamp":1700697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>A ReLU neural network functions as a continuous piecewise linear map from an input space to an output space. The weights in the neural network determine a partitioning of the input space into convex polytopes, where each polytope is associated with a distinct affine mapping. The structure of this partitioning, together with the affine map attached to each polytope, can be analyzed to investigate the behavior of the associated neural network. We investigate simple problems to build intuition on how these regions act and both how they can potentially be reduced in number and how similar structures occur across different networks. To validate these intuitions, we apply them to networks trained on MNIST to demonstrate similarity between those networks and the potential for them to be reduced in complexity.<\/jats:p>","DOI":"10.3389\/frai.2023.1255192","type":"journal-article","created":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T11:40:46Z","timestamp":1700739646000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Locally linear attributes of ReLU neural networks"],"prefix":"10.3389","volume":"6","author":[{"given":"Ben","family":"Sattelberg","sequence":"first","affiliation":[]},{"given":"Renzo","family":"Cavalieri","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Kirby","sequence":"additional","affiliation":[]},{"given":"Chris","family":"Peterson","sequence":"additional","affiliation":[]},{"given":"Ross","family":"Beveridge","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,11,23]]},"reference":[{"key":"B1","first-page":"129","article-title":"\u201cWhat is the state of neural network pruning?\u201d","author":"Blalock","year":"2020","journal-title":"Proceedings of Machine Learning and Systems 2 (MLSys 2020)"},{"key":"B2","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/BF02551274","article-title":"Approximation by superpositions of a sigmoidal function","volume":"2","author":"Cybenko","year":"1989","journal-title":"Mathem. Control Sign. Syst"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00482","article-title":"\u201cArcFace: additive angular margin loss for deep face recognition,\u201d","author":"Deng","year":"2019","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B4","article-title":"\u201cThe lottery ticket hypothesis: finding sparse, trainable neural networks,\u201d","author":"Frankle","year":"2018","journal-title":"International Conference on Learning Representations"},{"key":"B5","article-title":"Complexity of linear regions in deep networks","author":"Hanin","year":"","journal-title":"arXiv preprint arXiv:1901.09021"},{"key":"B6","first-page":"361","article-title":"\u201cDeep ReLU networks have surprisingly few activation patterns,\u201d","author":"Hanin","year":"","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B7","article-title":"Approximating continuous functions by ReLU nets of minimal width","author":"Hanin","year":"2017","journal-title":"arXiv preprint arXiv:1710.11278"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition"},{"key":"B9","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1016\/0893-6080(91)90009-T","article-title":"Approximation capabilities of multilayer feedforward networks","volume":"4","author":"Hornik","year":"1991","journal-title":"Neural Netw"},{"key":"B10","unstructured":"LeCunY.\n            CortesC.\n            BurgesC. J.\n          The MNIST database of handwritten digits1998"},{"key":"B11","first-page":"6169","article-title":"\u201cResNet with one-neuron hidden layers is a universal approximator,\u201d","author":"Lin","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B12","article-title":"\u201cRelu neural networks, polyhedral decompositions, and persistent homology,\u201d","author":"Liu","year":"2023","journal-title":"the ICML 2023 Workshop on Topology, Algebra, and Geometry in Machine Learning"},{"key":"B13","first-page":"6231","article-title":"\u201cThe expressive power of neural networks: A view from the width,\u201d","author":"Lu","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1145\/1873951.1874254","article-title":"\u201cTorchvision the machine-vision package of torch,\u201d","author":"Marcel","year":"2010","journal-title":"Proceedings of the 18th ACM international conference on Multimedia"},{"key":"B15","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.cogsys.2019.10.004","article-title":"Inception and ResNet features are (almost) equivalent","volume":"59","author":"McNeely-White","year":"2019","journal-title":"Cogn. Syst. Res"},{"key":"B16","first-page":"2924","article-title":"\u201cOn the number of linear regions of deep neural networks,\u201d","author":"Montufar","year":"2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B17","article-title":"Sensitivity and generalization in neural networks: an empirical study","author":"Novak","year":"2018","journal-title":"arXiv preprint arXiv:1802.08760"},{"key":"B18","article-title":"On the number of response regions of deep feed forward networks with piece-wise linear activations","author":"Pascanu","year":"2013","journal-title":"arXiv preprint arXiv:1312.6098"},{"key":"B19","first-page":"8024","article-title":"\u201cPyTorch: an imperative style, high-performance deep learning library,\u201d","author":"Paszke","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B20","first-page":"2847","article-title":"\u201cOn the expressive power of deep neural networks,\u201d","author":"Raghu","year":"2017","journal-title":"international Conference on Machine Learning, pages"},{"key":"B21","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis"},{"key":"B22","article-title":"Locally linear attributes of relu neural networks","author":"Sattelberg","year":"2020","journal-title":"arXiv preprint arXiv:2012.01940"},{"key":"B23","article-title":"Deep inside convolutional networks: Visualising image classification models and saliency maps","author":"Simonyan","year":"2013","journal-title":"arXiv preprint arXiv:1312.6034"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.308","article-title":"\u201cRethinking the inception architecture for computer vision,\u201d","author":"Szegedy","year":"2016","journal-title":"Proceedings of the IEEE conference on Computer Vision and Pattern Recognition"},{"key":"B25","article-title":"Tropical geometry of deep neural networks","author":"Zhang","year":"2018","journal-title":"arXiv preprint arXiv:1805.07091"},{"key":"B26","article-title":"\u201cEmpirical studies on the properties of linear regions in deep neural networks,\u201d","author":"Zhang","year":"2019","journal-title":"International Conference on Learning Representations"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1255192\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T11:40:56Z","timestamp":1700739656000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1255192\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,23]]},"references-count":26,"alternative-id":["10.3389\/frai.2023.1255192"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1255192","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,23]]},"article-number":"1255192"}}