{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T10:31:12Z","timestamp":1786098672751,"version":"3.56.0"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T00:00:00Z","timestamp":1633996800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T00:00:00Z","timestamp":1633996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100010890","name":"Chinese Government Scholarship","doi-asserted-by":"publisher","award":["201806240132"],"award-info":[{"award-number":["201806240132"]}],"id":[{"id":"10.13039\/501100010890","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1913080"],"award-info":[{"award-number":["1913080"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2012465"],"award-info":[{"award-number":["2012465"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Math. Program."],"published-print":{"date-parts":[[2022,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Convergence analysis of accelerated first-order methods for convex optimization problems are developed from the point of view of ordinary differential equation solvers. A new dynamical system, called Nesterov accelerated gradient (NAG) flow, is derived from the connection between acceleration mechanism and <jats:italic>A<\/jats:italic>-stability of ODE solvers, and the exponential decay of a tailored Lyapunov function along with the solution trajectory is proved. Numerical discretizations of NAG flow are then considered and convergence rates are established via a discrete Lyapunov function. The proposed differential equation solver approach can not only cover existing accelerated methods, such as FISTA, G\u00fcler\u2019s proximal algorithm and Nesterov\u2019s accelerated gradient method, but also produce new algorithms for composite convex optimization that possess accelerated convergence rates. Both the convex and the strongly convex cases are handled in a unified way in our approach.\n<\/jats:p>","DOI":"10.1007\/s10107-021-01713-3","type":"journal-article","created":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T01:57:39Z","timestamp":1634090259000},"page":"735-781","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":50,"title":["From differential equation solvers to accelerated first-order methods for convex optimization"],"prefix":"10.1007","volume":"195","author":[{"given":"Hao","family":"Luo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7345-5116","authenticated-orcid":false,"given":"Long","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,10,12]]},"reference":[{"issue":"4","key":"1713_CR1","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.1137\/S0363012998335802","volume":"38","author":"F Alvarez","year":"2000","unstructured":"Alvarez, F.: On the minimizing property of a second order dissipative system in Hilbert spaces. SIAM J. Control. Optim. 38(4), 1102\u20131119 (2000)","journal-title":"SIAM J. Control. Optim."},{"key":"1713_CR2","doi-asserted-by":"crossref","unstructured":"Apidopoulos, V., Aujol, J.-F., Dossal, C.: Convergence rate of inertial Forward\u2013Backward algorithm beyond Nesterov\u2019s rule. Math. Program. (2018)","DOI":"10.1007\/s10107-018-1350-9"},{"issue":"1","key":"1713_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1142\/S0219199700000025","volume":"2","author":"H Attouch","year":"2000","unstructured":"Attouch, H., Goudou, X., Redont, P.: The heavy ball with friction method, I. The continuous dynamical system: Global exploration of the local minima of a real-valued function by asymptotic analysis of a dissipative dynamical system. Commun. Contemp. Math. 2(1), 1\u201334 (2000)","journal-title":"Commun. Contemp. Math."},{"key":"1713_CR4","doi-asserted-by":"crossref","unstructured":"Attouch, H., Buttazzo, G., Michaille, G.: Variational Analysis in Sobolev and BV Spaces. Society for Industrial and Applied Mathematics, MOS-SIAM Series on Optimization (2014)","DOI":"10.1137\/1.9781611973488"},{"key":"1713_CR5","unstructured":"Attouch, H., Chbani, Z.: Fast inertial dynamics and FISTA algorithms in convex optimization. Perturbation aspects. arXiv:1507.01367 (2015)"},{"issue":"1\u20132","key":"1713_CR6","first-page":"123","volume":"168","author":"H Attouch","year":"2016","unstructured":"Attouch, H., Chbani, Z., Peypouquet, J., Redont, P.: Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity. Math. Program. 168(1\u20132), 123\u2013175 (2016)","journal-title":"Math. Program."},{"issue":"1","key":"1713_CR7","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1137\/17M1114739","volume":"28","author":"H Attouch","year":"2018","unstructured":"Attouch, H., Cabot, A.: Convergence rates of inertial forward-backward algorithms. SIAM J. Optim. 28(1), 849\u2013874 (2018)","journal-title":"SIAM J. Optim."},{"key":"1713_CR8","doi-asserted-by":"crossref","unstructured":"Attouch, H., Chbani, Z., Riahi, H.: Rate of convergence of the Nesterov accelerated gradient method in the subcritical case $$\\alpha \\leqslant 3$$. ESAIM: Control Optim. Calc. Variat. 25(2), 1 (2019)","DOI":"10.1051\/cocv\/2017083"},{"key":"1713_CR9","doi-asserted-by":"crossref","unstructured":"Ahmad, S., Ambrosetti, A.: A Textbook on Ordinary Differential Equations, 2nd. UNITEXT - La Matematica per il 3+2, vol. 88. Springer, Cham (2015)","DOI":"10.1007\/978-3-319-16408-3"},{"key":"1713_CR10","unstructured":"Aujol, J., Dossal, C.: Optimal rate of convergence of an ODE associated to the fast gradient descent schemes for $$b>0$$. hal-01547251v2:22, (2017)"},{"key":"1713_CR11","doi-asserted-by":"crossref","unstructured":"Balti, M., May, R.: Asymptotic for the perturbed heavy ball system with vanishing damping term. Evol. Equ. Control Theory 6(2),1 (2016)","DOI":"10.3934\/eect.2017010"},{"issue":"1","key":"1713_CR12","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1137\/080716542","volume":"2","author":"A Beck","year":"2009","unstructured":"Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imag. Sci. 2(1), 183\u2013202 (2009)","journal-title":"SIAM J. Imag. Sci."},{"key":"1713_CR13","unstructured":"Bellman, R.: Stability Theory of Differential Equations. MeGraw-Hill Book Company (1953)"},{"issue":"11","key":"1713_CR14","doi-asserted-by":"publisher","first-page":"5983","DOI":"10.1090\/S0002-9947-09-04785-0","volume":"361","author":"A Cabot","year":"2009","unstructured":"Cabot, A., Engler, H., Gadat, S.: On the long time behavior of second order differential equations with asymptotically small dissipation. Trans. Am. Math. Soc. 361(11), 5983\u20136017 (2009)","journal-title":"Trans. Am. Math. Soc."},{"issue":"3","key":"1713_CR15","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1016\/j.matpur.2007.01.003","volume":"87","author":"A Cabot","year":"2007","unstructured":"Cabot, A., Paoli, L.: Asymptotics for some vibro-impact problems with a linear dissipation term. J. Math. Pures Appl. 87(3), 291\u2013323 (2007)","journal-title":"J. Math. Pures Appl."},{"key":"1713_CR16","doi-asserted-by":"crossref","unstructured":"Demmel, J.: Applied Numerical Linear Algebra. Society for Industrial and Applied Mathematics (1997)","DOI":"10.1137\/1.9781611971446"},{"key":"1713_CR17","doi-asserted-by":"crossref","unstructured":"Diakonikolas, J., Orecchia, L.: The approximate duality gap technique: A unified theory of first-order methods. arXiv:1712.02485, (2018)","DOI":"10.1137\/18M1172314"},{"key":"1713_CR18","doi-asserted-by":"crossref","unstructured":"Ghadimi, E., Feyzmahdavian, H.\u00a0R., Johansson, M.: Global convergence of the Heavy-ball method for convex optimization. In European Control Conference (ECC), pp. 310\u2013315, (2015)","DOI":"10.1109\/ECC.2015.7330562"},{"issue":"1\u20132","key":"1713_CR19","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1007\/s10107-007-0109-5","volume":"116","author":"X Goudou","year":"2009","unstructured":"Goudou, X., Munier, J.: The gradient and heavy ball with friction dynamical systems: the quasiconvex case. Math. Program. 116(1\u20132), 173\u2013191 (2009)","journal-title":"Math. Program."},{"issue":"4","key":"1713_CR20","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1137\/0802032","volume":"2","author":"O G\u00fcler","year":"1992","unstructured":"G\u00fcler, O.: New proximal point algorithms for convex minimization. SIAM J. Optim. 2(4), 649\u2013664 (1992)","journal-title":"SIAM J. Optim."},{"key":"1713_CR21","unstructured":"Kreuter, M.: Sobolev Spaces of Vector-Valued Functions. Master Thesis, Ulm University (2015)"},{"issue":"1","key":"1713_CR22","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1137\/15M1009597","volume":"26","author":"L Lessard","year":"2016","unstructured":"Lessard, L., Recht, B., Packard, A.: Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints. SIAM J. Optim. 26(1), 57\u201395 (2016)","journal-title":"SIAM J. Optim."},{"key":"1713_CR23","doi-asserted-by":"crossref","unstructured":"LeVeque, R.: Finite Difference Methods for Ordinary and Partial Differential Equations: Steady-State and Time-Dependent Problems. Society for Industrial and Applied Mathematics (2007)","DOI":"10.1137\/1.9780898717839"},{"key":"1713_CR24","doi-asserted-by":"crossref","unstructured":"Lin, Z., Li, H., Fang, C.: Accelerated Optimization for Machine Learning. Springer, Singapore (2020)","DOI":"10.1007\/978-981-15-2910-8"},{"key":"1713_CR25","doi-asserted-by":"crossref","unstructured":"Luo, H.: Accelerated differential inclusion for convex optimization. arXiv:2103.06629 (2021)","DOI":"10.1080\/02331934.2021.2002327"},{"issue":"5","key":"1713_CR26","doi-asserted-by":"publisher","first-page":"A3157","DOI":"10.1137\/17M1141369","volume":"40","author":"N Nguyen","year":"2018","unstructured":"Nguyen, N., Fernandez, P., Freund, R.M., Peraire, J.: Accelerated residual methods for the iterative solution of systems of equations. SIAM J. Sci. Comput. 40(5), A3157\u2013A3179 (2018)","journal-title":"SIAM J. Sci. Comput."},{"issue":"2","key":"1713_CR27","first-page":"372","volume":"27","author":"Y Nesterov","year":"1983","unstructured":"Nesterov, Y.: A method of solving a convex programming problem with convergence rate $${O}(1\/k^2)$$. Sov. Math. Doklady 27(2), 372\u2013376 (1983)","journal-title":"Sov. Math. Doklady"},{"issue":"1","key":"1713_CR28","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/s10107-012-0629-5","volume":"140","author":"Y Nesterov","year":"2012","unstructured":"Nesterov, Y.: Gradient methods for minimizing composite functions. Math. Program. 140(1), 125\u2013161 (2012)","journal-title":"Math. Program."},{"key":"1713_CR29","unstructured":"Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course, volume\u00a087. Springer (2013)"},{"issue":"1","key":"1713_CR30","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s10107-018-1232-1","volume":"175","author":"I Necoara","year":"2019","unstructured":"Necoara, I., Nesterov, Y., Glineur, F.: Linear convergence of first order methods for non-strongly convex optimization. Math. Program. 175(1), 69\u2013107 (2019)","journal-title":"Math. Program."},{"key":"1713_CR31","doi-asserted-by":"crossref","unstructured":"O\u2019Donoghue, B., Cand\u00e8s, E.: Adaptive restart for accelerated gradient schemes. Found. Comput. Math. 15(3), 715\u2013732 (2015)","DOI":"10.1007\/s10208-013-9150-3"},{"issue":"06","key":"1713_CR32","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1142\/S0218202500000422","volume":"10","author":"L Paoli","year":"2000","unstructured":"Paoli, L.: An existence result for vibrations with unilateral constraints: case of a nonsmooth set of constraints. Math. Models Methods Appl. Sci. 10(06), 815\u2013831 (2000)","journal-title":"Math. Models Methods Appl. Sci."},{"issue":"3","key":"1713_CR33","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1561\/2400000003","volume":"1","author":"N Parikh","year":"2014","unstructured":"Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1(3), 127\u2013239 (2014)","journal-title":"Found. Trends Optim."},{"issue":"5","key":"1713_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/0041-5553(64)90137-5","volume":"4","author":"B Polyak","year":"1964","unstructured":"Polyak, B.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1\u201317 (1964)","journal-title":"USSR Comput. Math. Math. Phys."},{"key":"1713_CR35","doi-asserted-by":"crossref","unstructured":"Rockafellar, R.: Convex Analysis. Princeton University Press (1970)","DOI":"10.1515\/9781400873173"},{"issue":"3","key":"1713_CR36","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1016\/0362-546X(78)90022-6","volume":"2","author":"M Schatzman","year":"1978","unstructured":"Schatzman, M.: A class of nonlinear differential equations of second order in time. Nonlinear Anal. 2(3), 355\u2013373 (1978)","journal-title":"Nonlinear Anal."},{"key":"1713_CR37","unstructured":"Su, W., Boyd, S., Cand\u00e8s, E.: A differential equation for modeling Nesterov\u2019s accelerated gradient method: Theory and insights. J. Mach. Learn. Res. 17(153), 1\u201343 (2016)"},{"key":"1713_CR38","unstructured":"Siegel, J.: Accelerated first-order methods: Differential equations and Lyapunov functions. arXiv preprint: arXiv:1903.05671 (2019)"},{"key":"1713_CR39","volume-title":"Numerical Solution of Ordinary Differential Equations","author":"E S\u00fcli","year":"2010","unstructured":"S\u00fcli, E.: Numerical Solution of Ordinary Differential Equations. University of Oxford, Mathematical Institute (2010)"},{"key":"1713_CR40","doi-asserted-by":"crossref","unstructured":"Sun, T., Yin, P., Li, D., Huang, C., Guan, L., Jiang, H.: Non-ergodic convergence analysis of heavy-ball algorithms. arXiv:1811.01777 (2018)","DOI":"10.1609\/aaai.v33i01.33015033"},{"key":"1713_CR41","unstructured":"Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. Unpublished manuscript (2008)"},{"issue":"1","key":"1713_CR42","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1137\/17M1128642","volume":"28","author":"A Vassilis","year":"2018","unstructured":"Vassilis, A., Jean-Fran\u00e7ois, A., Charles, D.: The differential inclusion modeling FISTA algorithm and optimality of convergence rate in the case $$b\\leqslant 3$$. SIAM J. Optim. 28(1), 551\u2013574 (2018)","journal-title":"SIAM J. Optim."},{"issue":"47","key":"1713_CR43","doi-asserted-by":"publisher","first-page":"E7351","DOI":"10.1073\/pnas.1614734113","volume":"113","author":"A Wibisono","year":"2016","unstructured":"Wibisono, A., Wilson, A., Jordan, M.: A variational perspective on accelerated methods in optimization. Proc. Natl. Acad. Sci. 113(47), E7351\u2013E7358 (2016)","journal-title":"Proc. Natl. Acad. Sci."},{"key":"1713_CR44","unstructured":"Wilson, A., Recht, B., Jordan, M.: A Lyapunov analysis of momentum methods in optimization. arXiv preprint: arXiv:1611.02635 (2016)"},{"key":"1713_CR45","volume-title":"Iterative Methods for Sparse Linear Systems","author":"S Yousef","year":"2003","unstructured":"Yousef, S.: Iterative Methods for Sparse Linear Systems. Society for Industrial and Applied Mathematics, USA (2003)"}],"container-title":["Mathematical Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10107-021-01713-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10107-021-01713-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10107-021-01713-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,21]],"date-time":"2022-10-21T15:37:12Z","timestamp":1666366632000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10107-021-01713-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,12]]},"references-count":45,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2022,9]]}},"alternative-id":["1713"],"URL":"https:\/\/doi.org\/10.1007\/s10107-021-01713-3","relation":{},"ISSN":["0025-5610","1436-4646"],"issn-type":[{"value":"0025-5610","type":"print"},{"value":"1436-4646","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,12]]},"assertion":[{"value":"13 June 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}