{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T06:31:42Z","timestamp":1790749902053,"version":"4.1.0"},"reference-count":49,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Optim."],"published-print":{"date-parts":[[2018,1]]},"abstract":"<jats:p>We analyze the convergence of a decentralized consensus algorithm with delayed gradient information across the network. The nodes in the network privately hold parts of the objective function and collaboratively solve for the consensus optimal solution of the total objective while they can only communicate with their immediate neighbors. In real-world networks, it is often difficult and sometimes impossible to synchronize the nodes, and therefore they have to use stale gradient information during computations. We show that, as long as the random delays are bounded in expectation and a proper diminishing step size policy is employed, the iterates generated by the decentralized gradient descent method converge to a consensual optimal solution. Convergence rates of both objective and consensus are derived. Numerical results on a number of synthetic problems and real-world seismic tomography datasets in decentralized sensor networks are presented to show the performance of the method.<\/jats:p>","DOI":"10.1137\/16m1081257","type":"journal-article","created":{"date-parts":[[2018,4,26]],"date-time":"2018-04-26T11:22:05Z","timestamp":1524741725000},"page":"1232-1254","source":"Crossref","is-referenced-by-count":32,"title":["Decentralized Consensus Algorithm with Delayed and Stochastic Gradients"],"prefix":"10.1137","volume":"28","author":[{"given":"Benjamin","family":"Sirb","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaojing","family":"Ye","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2018,4,26]]},"reference":[{"key":"atypb1","unstructured":"A. Agarwal and J. C. Duchi,\n                      Distributed delayed stochastic optimization\n                      , in Advances in Neural Information Processing Systems, MIT Press, 2011, pp. 873-881."},{"key":"atypb2","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2009.2016247"},{"key":"atypb3","doi-asserted-by":"publisher","DOI":"10.1561\/2200000016"},{"key":"atypb4","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2014.2329397"},{"key":"atypb5","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2016.2537271"},{"key":"atypb6","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2014.2367458"},{"key":"atypb7","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2016.2537261"},{"key":"atypb8","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2011.2161027"},{"key":"atypb9","first-page":"1","author":"Feyzmahdavian H. R.","year":"2014","journal-title":"IEEE"},{"key":"atypb10","first-page":"1663","volume":"11","author":"Forero P. A.","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"atypb11","doi-asserted-by":"publisher","DOI":"10.1109\/TPWRS.2012.2210288"},{"key":"atypb12","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2015.2448011"},{"key":"atypb13","first-page":"3117","author":"Iutzeler F.","year":"2012","journal-title":"IEEE"},{"key":"atypb14","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2014.2363299"},{"key":"atypb15","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2014.2298712"},{"key":"atypb16","first-page":"2","author":"Kraska T.","year":"2013","journal-title":"CIDR Conference"},{"key":"atypb17","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2015.12.017"},{"key":"atypb18","unstructured":"M. Li, D. G. Andersen, and A. Smola,\n                      Distributed delayed proximal gradient methods\n                      , in NIPS Workshop on Optimization for Machine Learning, MIT Press, 2013."},{"key":"atypb19","unstructured":"M. Li, D. G. Andersen, A. J. Smola, and K. Yu,\n                      Communication efficient distributed machine learning with the parameter server\n                      , in Advances in Neural Information Processing Systems, MIT Press, 2014, pp. 19-27."},{"key":"atypb20","first-page":"285","volume":"16","author":"Liu J.","year":"2015","journal-title":"J. Mach. Learn. Res."},{"key":"atypb21","doi-asserted-by":"publisher","DOI":"10.1109\/TSG.2012.2228282"},{"key":"atypb22","first-page":"1","volume":"2","author":"Lova\u0301sz L.","year":"1993","journal-title":"Combinatorics"},{"key":"atypb23","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2017.2677879"},{"key":"atypb24","first-page":"406","author":"Mokhtari A.","year":"2015","journal-title":"Systems and Computers, IEEE"},{"key":"atypb25","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2014.2364096"},{"key":"atypb26","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2016.2529285"},{"key":"atypb27","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2008.2009515"},{"key":"atypb28","unstructured":"Y. Nesterov,\n                      A Method for Unconstrained Convex Minimization Problem with the Rate of Convergence o (1\/k2)\n                      , Technical report 3, Doklady AN SSSR, 1983."},{"key":"atypb29","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2004.834113"},{"key":"atypb30","first-page":"20","author":"Rabbat M.","year":"2004","journal-title":"ACM"},{"key":"atypb31","doi-asserted-by":"publisher","DOI":"10.1561\/2200000051"},{"key":"atypb32","first-page":"1","author":"Sayed A. H.","year":"2013","journal-title":"IEEE"},{"key":"atypb33","first-page":"850","author":"Shamir O.","year":"2014","journal-title":"IEEE"},{"key":"atypb34","doi-asserted-by":"publisher","DOI":"10.1137\/14096668X"},{"key":"atypb35","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2014.2304432"},{"key":"atypb36","first-page":"305","author":"Song W.-Z.","year":"2009","journal-title":"ACM"},{"key":"atypb37","first-page":"957","author":"Sra S.","year":"2016","journal-title":"PMLR"},{"key":"atypb38","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2008.930184"},{"key":"atypb40","unstructured":"J. N. Tsitsiklis,\n                      Problems in Decentralized Decision Making and Computation\n                      , Technical report, DTIC Document, 1984."},{"key":"atypb41","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.2014.2332306"},{"key":"atypb42","first-page":"551","author":"Wei E.","year":"2013","journal-title":"IEEE"},{"key":"atypb43","doi-asserted-by":"crossref","unstructured":"T. Wu, K. Yuan, Q. Ling, W. Yin, and A. H. Sayed,\n                      Decentralized consensus optimization with asynchrony and delays\n                      , in Proceedings of IEEE Asilomar Conference on Signals, Systems, and Computers, IEEE, 2016.","DOI":"10.1109\/ACSSC.2016.7869516"},{"key":"atypb44","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysconle.2004.02.022"},{"key":"atypb45","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2015.2464255"},{"key":"atypb46","doi-asserted-by":"publisher","DOI":"10.1137\/130943170"},{"key":"atypb47","first-page":"1701","author":"Zhang R.","year":"2014","journal-title":"PMLR"},{"key":"atypb48","unstructured":"W. Zhang, S. Gupta, X. Lian, and J. Liu,\n                      Staleness-aware ASYNC-SGD for distributed deep learning\n                      , in Proceedings of the 25th International Joint Conference on Artificial Intelligence, AAAI Press, 2016, pp. 2350-2356."},{"key":"atypb49","first-page":"1","volume":"1","author":"Zhao L.","year":"2015","journal-title":"Cyber-Phys. Syst."},{"key":"atypb50","first-page":"908","author":"Zhao L.","year":"2015","journal-title":"IEEE"}],"container-title":["SIAM Journal on Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/16M1081257","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T19:03:07Z","timestamp":1787338987000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/16M1081257"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,1]]},"references-count":49,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2018,1]]}},"alternative-id":["10.1137\/16M1081257"],"URL":"https:\/\/doi.org\/10.1137\/16m1081257","relation":{},"ISSN":["1052-6234","1095-7189"],"issn-type":[{"value":"1052-6234","type":"print"},{"value":"1095-7189","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,1]]}}}