{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T16:30:55Z","timestamp":1784737855373,"version":"3.55.0"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:p>The residual network is now one of the most effective structures in deep learning, which utilizes the skip connections to \u201cguarantee\" the performance will not get worse. However, the non-convexity of the neural network makes it unclear whether the skip connections do provably improve the learning ability since the nonlinearity may create many local minima. In some previous works  [Freeman and Bruna, 2016], it is shown that despite the non-convexity, the loss landscape of the two-layer ReLU network has good properties when the number m of hidden nodes is very large. In this paper, we follow this line to study the topology (sub-level sets) of the loss landscape of deep ReLU neural networks with a skip connection and theoretically prove that the skip connection network inherits the good properties of the two-layer network and skip connections can  help to control the connectedness of the sub-level sets, such that any local minima worse than the global minima of some two-layer ReLU network will be very \u201cshallow\". The \u201cdepth\" of these local minima are at most O(m^(\u03b7-1)\/n), where n is the input dimension, \u03b7&lt;1. This provides a theoretical explanation for the effectiveness of the skip connection in deep learning.<\/jats:p>","DOI":"10.24963\/ijcai.2020\/387","type":"proceedings-article","created":{"date-parts":[[2020,7,8]],"date-time":"2020-07-08T12:12:10Z","timestamp":1594210330000},"page":"2792-2798","source":"Crossref","is-referenced-by-count":6,"title":["Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?"],"prefix":"10.24963","author":[{"given":"Lifu","family":"Wang","sequence":"first","affiliation":[{"name":"Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bo","family":"Shen","sequence":"additional","affiliation":[{"name":"Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ning","family":"Zhao","sequence":"additional","affiliation":[{"name":"Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiyuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}","theme":"Artificial Intelligence","location":"Yokohama, Japan","acronym":"IJCAI-PRICAI-2020","number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2020,7,11]]},"end":{"date-parts":[[2020,7,17]]}},"container-title":["Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T02:14:51Z","timestamp":1594260891000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2020\/387"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2020\/387","relation":{},"subject":[],"published":{"date-parts":[[2020,7]]}}}