{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T19:02:02Z","timestamp":1763665322360,"version":"3.41.2"},"reference-count":25,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T00:00:00Z","timestamp":1699920000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>This paper investigates the integration of multiple geometries present within a ReLU-based neural network. A ReLU neural network determines a piecewise affine linear continuous map, <jats:italic>M<\/jats:italic>, from an input space \u211d<jats:sup><jats:italic>m<\/jats:italic><\/jats:sup> to an output space \u211d<jats:sup><jats:italic>n<\/jats:italic><\/jats:sup>. The piecewise behavior corresponds to a polyhedral decomposition of \u211d<jats:sup><jats:italic>m<\/jats:italic><\/jats:sup>. Each polyhedron in the decomposition can be labeled with a binary vector (whose length equals the number of ReLU nodes in the network) and with an affine linear function (which agrees with <jats:italic>M<\/jats:italic> when restricted to points in the polyhedron). We develop a toolbox that calculates the binary vector for a polyhedra containing a given data point with respect to a given ReLU FFNN. We utilize this binary vector to derive bounding facets for the corresponding polyhedron, extraction of \u201cactive\u201d bits within the binary vector, enumeration of neighboring binary vectors, and visualization of the polyhedral decomposition (Python code is available at <jats:ext-link>https:\/\/github.com\/cglrtrgy\/GoL_Toolbox<\/jats:ext-link>). Polyhedra in the polyhedral decomposition of \u211d<jats:sup><jats:italic>m<\/jats:italic><\/jats:sup> are neighbors if they share a facet. Binary vectors for neighboring polyhedra differ in exactly 1 bit. Using the toolbox, we analyze the Hamming distance between the binary vectors for polyhedra containing points from adversarial\/nonadversarial datasets revealing distinct geometric properties. A bisection method is employed to identify sample points with a Hamming distance of 1 along the shortest Euclidean distance path, facilitating the analysis of local geometric interplay between Euclidean geometry and the polyhedral decomposition along the path. Additionally, we study the distribution of Chebyshev centers and related radii across different polyhedra, shedding light on the polyhedral shape, size, clustering, and aiding in the understanding of decision boundaries.<\/jats:p>","DOI":"10.3389\/fdata.2023.1274831","type":"journal-article","created":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T11:44:42Z","timestamp":1699962282000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Integrating geometries of ReLU feedforward neural networks"],"prefix":"10.3389","volume":"6","author":[{"given":"Yajing","family":"Liu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Turgay","family":"Caglar","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher","family":"Peterson","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Kirby","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,11,14]]},"reference":[{"key":"B1","first-page":"1","article-title":"\u201cUnderstanding deep neural networks with rectified linear units,\u201d","author":"Arora","year":"2018","journal-title":"International Conference on Learning Representations"},{"key":"B2","first-page":"374","article-title":"\u201cA spline theory of deep learning,\u201d","author":"Balestriero","year":"2018","journal-title":"Proceedings of the 35th International Conference on Machine Learning"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex Optimization","author":"Boyd","year":"2004"},{"key":"B4","first-page":"1","article-title":"Pconvex geometry and duality of over-parameterized neural networks","volume":"22","author":"Ergen","year":"2021","journal-title":"J. Mach. Learn. Res"},{"key":"B5","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow","year":"2015","journal-title":"arXiv"},{"key":"B6","first-page":"361","article-title":"\u201cDeep ReLU networks have surprisingly few activation patterns,\u201d","author":"Hanin","year":"2019","journal-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems"},{"key":"B7","doi-asserted-by":"publisher","first-page":"7304","DOI":"10.1109\/TIT.2019.2927252","article-title":"A framework for the construction of upper bounds on the number of affine linear regions of relu feed-forward neural networks","volume":"65","author":"Hinz","year":"2019","journal-title":"IEEE Trans. Inf. Theory"},{"key":"B8","doi-asserted-by":"crossref","first-page":"2913","DOI":"10.1109\/BigData55660.2022.10020880","article-title":"\u201cDual graphs of polyhedral decompositions for the detection of adversarial attacks,\u201d","author":"Jamil","year":"2022","journal-title":"2022 IEEE International Conference on Big Data (Big Data)"},{"key":"B9","first-page":"590","article-title":"\u201cHamming similarity and graph Laplacians for class partitioning and adversarial image detection,\u201d","author":"Jamil","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops"},{"key":"B10","article-title":"\u201cReLU neural networks, polyhedral decompositions, and persistent homology,\u201d","author":"Liu","year":"2023","journal-title":"the ICML 2023 Workshop on Topology, Algebra, and Geometry in Machine Learning"},{"key":"B11","article-title":"Algorithmic determination of the combinatorial structure of the linear regions of relu neural networks","author":"Masden","year":"2022","journal-title":"arXiv"},{"key":"B12","first-page":"2924","article-title":"\u201cOn the number of linear regions of deep neural networks,\u201d","author":"Montufar","year":"2014","journal-title":"Proceedings of the 27th International Conference on Neural Information Processing Systems"},{"key":"B13","article-title":"Sensitivity and generalization in neural networks: an empirical study","author":"Novak","year":"2018","journal-title":"arXiv"},{"key":"B14","article-title":"On the number of response regions of deep feed forward networks with piece-wise linear activations","author":"Pascanu","year":"2013","journal-title":"arXiv"},{"key":"B15","first-page":"8026","article-title":"\u201cPytorch: An imperative style, high-performance deep learning library,\u201d","author":"Paszke","year":"2019","journal-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems"},{"key":"B16","first-page":"2847","article-title":"\u201cOn the expressive power of deep neural networks,\u201d","author":"Raghu","year":"2017","journal-title":"International Conference on Machine Learning"},{"key":"B17","article-title":"On the effective number of linear regions in shallow univariate ReLU networks: convergence guarantees and implicit bias","author":"Safran","year":"2022","journal-title":"arXiv"},{"key":"B18","article-title":"Locally linear attributes of ReLU neural networks","author":"Sattelberg","year":"2020","journal-title":"arXiv"},{"year":"2018","author":"Serra","journal-title":"Bounding and Counting Linear Regions of Deep Neural Networks","key":"B19"},{"key":"B20","doi-asserted-by":"crossref","first-page":"9029","DOI":"10.1109\/ICRA48506.2021.9561956","article-title":"\u201cReachable polyhedral marching (RPM): a safety verification algorithm for robotic systems with deep neural network components,\u201d","volume-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)","author":"Vincent","year":"2021"},{"key":"B21","first-page":"3544","article-title":"\u201cEstimation and comparison of linear regions for ReLU networks,\u201d","author":"Wang","year":"2022","journal-title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence"},{"key":"B22","doi-asserted-by":"crossref","first-page":"1574","DOI":"10.23919\/ACC.2018.8431048","article-title":"\u201cReachable set estimation and safety verification for piecewise linear systems with neural network controllers,\u201d","author":"Xiang","year":"2018","journal-title":"2018 Annual American Control Conference (ACC)"},{"key":"B23","article-title":"Traversing the local polytopes of ReLU neural networks: a unified approach for network verification","author":"Xu","year":"2021","journal-title":"arXiv"},{"key":"B24","article-title":"Reachability analysis for feed-forward neural networks using face lattices","author":"Yang","year":"2020","journal-title":"arXiv"},{"key":"B25","article-title":"Tropical geometry of deep neural networks","author":"Zhang","year":"2018","journal-title":"arXiv"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1274831\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T11:45:03Z","timestamp":1699962303000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1274831\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,14]]},"references-count":25,"alternative-id":["10.3389\/fdata.2023.1274831"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2023.1274831","relation":{},"ISSN":["2624-909X"],"issn-type":[{"type":"electronic","value":"2624-909X"}],"subject":[],"published":{"date-parts":[[2023,11,14]]},"article-number":"1274831"}}