{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,17]],"date-time":"2025-05-17T12:44:15Z","timestamp":1747485855748,"version":"3.37.3"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","issue":"19","license":[{"start":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T00:00:00Z","timestamp":1614816000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T00:00:00Z","timestamp":1614816000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Universit\u00e0 degli Studi di Roma La Sapienza"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soft Comput"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, we study the convergence of a block-coordinate incremental gradient method. Under some specific assumptions on the objective function, we prove that the block-coordinate incremental gradient method can be seen as a gradient method with errors and convergence can be proved by showing the error at each iteration satisfies some standard conditions. Thus, we can prove convergence towards stationary points when the block incremental gradient method is coupled with a diminishing stepsize and towards an<jats:inline-formula><jats:alternatives><jats:tex-math>$$\\epsilon $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\"><mml:mi>\u03f5<\/mml:mi><\/mml:math><\/jats:alternatives><\/jats:inline-formula>-approximate solution when a bounded away from zero stepsize is employed.<\/jats:p>","DOI":"10.1007\/s00500-021-05695-4","type":"journal-article","created":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T13:07:12Z","timestamp":1614863232000},"page":"12615-12626","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["On the convergence of a Block-Coordinate Incremental Gradient method"],"prefix":"10.1007","volume":"25","author":[{"given":"Laura","family":"Palagi","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5292-1774","authenticated-orcid":false,"given":"Ruggiero","family":"Seccia","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,4]]},"reference":[{"issue":"4","key":"5695_CR1","doi-asserted-by":"publisher","first-page":"2037","DOI":"10.1137\/120887679","volume":"23","author":"A Beck","year":"2013","unstructured":"Beck A, Tetruashvili L (2013) On the convergence of block coordinate descent type methods. SIAM J Optim 23(4):2037\u20132060","journal-title":"SIAM J Optim"},{"issue":"3","key":"5695_CR2","doi-asserted-by":"publisher","first-page":"807","DOI":"10.1137\/S1052623494268522","volume":"6","author":"DP Bertsekas","year":"1996","unstructured":"Bertsekas DP (1996) Incremental least squares methods and the extended Kalman filter. SIAM J Optim 6(3):807\u2013822. https:\/\/doi.org\/10.1137\/S1052623494268522","journal-title":"SIAM J Optim"},{"issue":"1\u201338","key":"5695_CR3","first-page":"3","volume":"2010","author":"DP Bertsekas","year":"2011","unstructured":"Bertsekas DP (2011) Incremental gradient, subgradient, and proximal methods for convex optimization: a survey. Optim Mach Learn 2010(1\u201338):3","journal-title":"Optim Mach Learn"},{"key":"5695_CR4","unstructured":"Bertsekas DP (2015) Incremental gradient, subgradient, and proximal methods for convex optimization: a survey. CoRR http:\/\/arxiv.org\/abs\/1507.01030"},{"key":"5695_CR5","volume-title":"Parallel and distributed computation: numerical methods","author":"DP Bertsekas","year":"1989","unstructured":"Bertsekas DP, Tsitsiklis JN (1989) Parallel and distributed computation: numerical methods, vol 23. Prentice Hall, Englewood Cliffs"},{"issue":"3","key":"5695_CR6","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1137\/S1052623497331063","volume":"10","author":"DP Bertsekas","year":"2000","unstructured":"Bertsekas DP, Tsitsiklis JN (2000) Gradient convergence in gradient methods with errors. SIAM J Optim 10(3):627\u2013642. https:\/\/doi.org\/10.1137\/S1052623497331063","journal-title":"SIAM J Optim"},{"issue":"1","key":"5695_CR7","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1137\/040615961","volume":"18","author":"D Blatt","year":"2007","unstructured":"Blatt D, Hero AO, Gauchman H (2007) A convergent incremental gradient method with a constant step size. SIAM J Optim 18(1):29\u201351","journal-title":"SIAM J Optim"},{"key":"5695_CR8","doi-asserted-by":"crossref","unstructured":"Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: in COMPSTAT","DOI":"10.1007\/978-3-7908-2604-3_16"},{"issue":"2","key":"5695_CR9","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1137\/16M1080173","volume":"60","author":"L Bottou","year":"2018","unstructured":"Bottou L, Curtis FE, Nocedal J (2018) Optimization methods for large-scale machine learning. SIAM Rev 60(2):223\u2013311","journal-title":"SIAM Rev"},{"key":"5695_CR10","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.amc.2014.02.088","volume":"235","author":"L Bravi","year":"2014","unstructured":"Bravi L, Sciandrone M (2014) An incremental decomposition method for unconstrained optimization. Appl Math Comput 235:80\u201386","journal-title":"Appl Math Comput"},{"issue":"8","key":"5695_CR11","doi-asserted-by":"publisher","first-page":"1891","DOI":"10.1162\/08997660152469396","volume":"13","author":"C Buzzi","year":"2001","unstructured":"Buzzi C, Grippo L, Sciandrone M (2001) Convergent decomposition techniques for training RBF neural networks. Neural Comput 13(8):1891\u20131920","journal-title":"Neural Comput"},{"key":"5695_CR12","unstructured":"Chauhan VK, Dahiya K, Sharma A (2017) Mini-batch block-coordinate based stochastic average adjusted gradient methods to solve big data problems. In: Proceedings of the Ninth Asian conference on machine learning, proceedings of machine learning research, vol 77, pp. 49\u201364. PMLR. http:\/\/proceedings.mlr.press\/v77\/chauhan17a.html"},{"issue":"1\u20133","key":"5695_CR13","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1023\/A:1013912006537","volume":"48","author":"M Collins","year":"2002","unstructured":"Collins M, Schapire RE, Singer Y (2002) Logistic regression, adaboost and bregman distances. Mach Learn 48(1\u20133):253\u2013285","journal-title":"Mach Learn"},{"key":"5695_CR14","volume-title":"Deep learning","author":"I Goodfellow","year":"2016","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, Cambridge"},{"issue":"11","key":"5695_CR15","doi-asserted-by":"publisher","first-page":"2146","DOI":"10.1109\/TNNLS.2015.2475621","volume":"27","author":"L Grippo","year":"2016","unstructured":"Grippo L, Manno A, Sciandrone M (2016) Decomposition techniques for multilayer perceptron training. IEEE Trans Neural Netw Learn Syst 27(11):2146\u20132159. https:\/\/doi.org\/10.1109\/TNNLS.2015.2475621","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"issue":"4","key":"5695_CR16","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1080\/10556789908805730","volume":"10","author":"L Grippo","year":"1999","unstructured":"Grippo L, Sciandrone M (1999) Globally convergent block-coordinate techniques for unconstrained optimization. Optim Methods Softw 10(4):587\u2013637. https:\/\/doi.org\/10.1080\/10556789908805730","journal-title":"Optim Methods Softw"},{"issue":"1\u20132","key":"5695_CR17","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1007\/s10107-014-0800-2","volume":"152","author":"Z Lu","year":"2015","unstructured":"Lu Z, Xiao L (2015) On the complexity analysis of randomized block-coordinate descent methods. Math Program 152(1\u20132):615\u2013642","journal-title":"Math Program"},{"issue":"2","key":"5695_CR18","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1137\/100802001","volume":"22","author":"Y Nesterov","year":"2012","unstructured":"Nesterov Y (2012) Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM J Optim 22(2):341\u2013362","journal-title":"SIAM J Optim"},{"issue":"1","key":"5695_CR19","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1007\/s10898-019-00856-0","volume":"77","author":"L Palagi","year":"2020","unstructured":"Palagi L, Seccia R (2020) Block layer decomposition schemes for training deep neural networks. J Global Optim 77(1):97\u2013124","journal-title":"J Global Optim"},{"key":"5695_CR20","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","author":"H Robbins","year":"1951","unstructured":"Robbins H, Monro S (1951) A stochastic approximation method. Ann Math Stat 22:400\u2013407","journal-title":"Ann Math Stat"},{"key":"5695_CR21","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809682","volume-title":"Kernel methods for pattern analysis","author":"J Shawe-Taylor","year":"2004","unstructured":"Shawe-Taylor J, Cristianini N et al (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge"},{"key":"5695_CR22","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556"},{"issue":"1","key":"5695_CR23","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1023\/A:1018366000512","volume":"11","author":"MV Solodov","year":"1998","unstructured":"Solodov MV (1998) Incremental gradient algorithms with stepsizes bounded away from zero. Comput Optim Appl 11(1):23\u201335","journal-title":"Comput Optim Appl"},{"key":"5695_CR24","unstructured":"Wang H, Banerjee A (2014) Randomized block coordinate descent for online and stochastic optimization. arXiv preprint arXiv:1407.0107"},{"issue":"1","key":"5695_CR25","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10107-015-0892-3","volume":"151","author":"SJ Wright","year":"2015","unstructured":"Wright SJ (2015) Coordinate descent algorithms. Math Program 151(1):3\u201334. https:\/\/doi.org\/10.1007\/s10107-015-0892-3","journal-title":"Math Program"},{"key":"5695_CR26","unstructured":"Zhao T, Yu M, Wang Y, Arora R, Liu H (2014) Accelerated mini-batch randomized block coordinate descent method. In: Advances in neural information processing systems, pp 3329\u20133337"},{"issue":"2","key":"5695_CR27","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1080\/10556789408805580","volume":"4","author":"L Zhi-Quan","year":"1994","unstructured":"Zhi-Quan L, Paul T (1994) Analysis of an approximate gradient projection method with applications to the backpropagation algorithm. Optim Methods Softw 4(2):85\u2013101","journal-title":"Optim Methods Softw"}],"container-title":["Soft Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-021-05695-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00500-021-05695-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-021-05695-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T02:26:21Z","timestamp":1698719181000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00500-021-05695-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,4]]},"references-count":27,"journal-issue":{"issue":"19","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["5695"],"URL":"https:\/\/doi.org\/10.1007\/s00500-021-05695-4","relation":{},"ISSN":["1432-7643","1433-7479"],"issn-type":[{"type":"print","value":"1432-7643"},{"type":"electronic","value":"1433-7479"}],"subject":[],"published":{"date-parts":[[2021,3,4]]},"assertion":[{"value":"13 February 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 March 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}