{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T07:53:49Z","timestamp":1769068429028,"version":"3.49.0"},"reference-count":53,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T00:00:00Z","timestamp":1717459200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T00:00:00Z","timestamp":1717459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["19H04069"],"award-info":[{"award-number":["19H04069"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004721","name":"The University of Tokyo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Math. Program."],"published-print":{"date-parts":[[2025,7]]},"abstract":"Abstract<\/jats:title>\n We propose a new first-order method for minimizing nonconvex functions with Lipschitz continuous gradients and H\u00f6lder continuous Hessians. The proposed algorithm is a heavy-ball method equipped with two particular restart mechanisms. It finds a solution where the gradient norm is less than \n \n $$\\varepsilon $$<\/jats:tex-math>\n \n \u03b5<\/mml:mi>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> in \n \n $$O(H_{\\nu }^{\\frac{1}{2 + 2 \\nu }} \\varepsilon ^{- \\frac{4 + 3 \\nu }{2 + 2 \\nu }})$$<\/jats:tex-math>\n \n \n O<\/mml:mi>\n (<\/mml:mo>\n \n H<\/mml:mi>\n \n \u03bd<\/mml:mi>\n <\/mml:mrow>\n \n 1<\/mml:mn>\n \n 2<\/mml:mn>\n +<\/mml:mo>\n 2<\/mml:mn>\n \u03bd<\/mml:mi>\n <\/mml:mrow>\n <\/mml:mfrac>\n <\/mml:msubsup>\n \n \u03b5<\/mml:mi>\n \n -<\/mml:mo>\n \n \n 4<\/mml:mn>\n +<\/mml:mo>\n 3<\/mml:mn>\n \u03bd<\/mml:mi>\n <\/mml:mrow>\n \n 2<\/mml:mn>\n +<\/mml:mo>\n 2<\/mml:mn>\n \u03bd<\/mml:mi>\n <\/mml:mrow>\n <\/mml:mfrac>\n <\/mml:mrow>\n <\/mml:msup>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> function and gradient evaluations, where \n \n $$\\nu \\in [0, 1]$$<\/jats:tex-math>\n \n \n \u03bd<\/mml:mi>\n \u2208<\/mml:mo>\n [<\/mml:mo>\n 0<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n ]<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> and \n \n $$H_{\\nu }$$<\/jats:tex-math>\n \n \n H<\/mml:mi>\n \u03bd<\/mml:mi>\n <\/mml:msub>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> are the H\u00f6lder exponent and constant, respectively. This complexity result covers the classical bound of \n \n $$O(\\varepsilon ^{-2})$$<\/jats:tex-math>\n \n \n O<\/mml:mi>\n (<\/mml:mo>\n \n \u03b5<\/mml:mi>\n \n -<\/mml:mo>\n 2<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msup>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> for \n \n $$\\nu = 0$$<\/jats:tex-math>\n \n \n \u03bd<\/mml:mi>\n =<\/mml:mo>\n 0<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> and the state-of-the-art bound of \n \n $$O(\\varepsilon ^{-7\/4})$$<\/jats:tex-math>\n \n \n O<\/mml:mi>\n (<\/mml:mo>\n \n \u03b5<\/mml:mi>\n \n -<\/mml:mo>\n 7<\/mml:mn>\n \/<\/mml:mo>\n 4<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msup>\n )<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> for \n \n $$\\nu = 1$$<\/jats:tex-math>\n \n \n \u03bd<\/mml:mi>\n =<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>. Our algorithm is \n \n $$\\nu $$<\/jats:tex-math>\n \n \u03bd<\/mml:mi>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>-independent and thus universal; it automatically achieves the above complexity bound with the optimal \n \n $$\\nu \\in [0, 1]$$<\/jats:tex-math>\n \n \n \u03bd<\/mml:mi>\n \u2208<\/mml:mo>\n [<\/mml:mo>\n 0<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n ]<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula> without knowledge of \n \n $$H_{\\nu }$$<\/jats:tex-math>\n \n \n H<\/mml:mi>\n \u03bd<\/mml:mi>\n <\/mml:msub>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>. In addition, the algorithm does not require other problem-dependent parameters as input, including the gradient\u2019s Lipschitz constant or the target accuracy \n \n $$\\varepsilon $$<\/jats:tex-math>\n \n \u03b5<\/mml:mi>\n <\/mml:math>\n <\/jats:alternatives>\n <\/jats:inline-formula>. Numerical results illustrate that the proposed method is promising.<\/jats:p>","DOI":"10.1007\/s10107-024-02100-4","type":"journal-article","created":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T15:04:46Z","timestamp":1717513486000},"page":"147-175","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Universal heavy-ball method for nonconvex optimization under H\u00f6lder continuous Hessians"],"prefix":"10.1007","volume":"212","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7372-4275","authenticated-orcid":false,"given":"Naoki","family":"Marumo","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Akiko","family":"Takeda","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,6,4]]},"reference":[{"key":"2100_CR1","doi-asserted-by":"publisher","unstructured":"Agarwal, N., Allen-Zhu, Z., Bullins, B., Hazan, E., Ma, T.: Finding approximate local minima faster than gradient descent. In: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of Computing, STOC 2017, pages 1195\u20131199, New York, NY, USA, (2017). Association for Computing Machinery. ISBN 9781450345286. https:\/\/doi.org\/10.1145\/3055399.3055464","DOI":"10.1145\/3055399.3055464"},{"key":"2100_CR2","unstructured":"Allen-Zhu, Z., Li, Y.: NEON2: finding local minima via first-order oracles. In: S.\u00a0Bengio, H.\u00a0Wallach, H.\u00a0Larochelle, K.\u00a0Grauman, N.\u00a0Cesa-Bianchi, and R.\u00a0Garnett, editors, Advances in Neural Information Processing Systems, volume\u00a031. Curran Associates, Inc., (2018). https:\/\/proceedings.neurips.cc\/paper\/2018\/file\/d4b2aeb2453bdadaa45cbe9882ffefcf-Paper.pdf"},{"key":"2100_CR3","doi-asserted-by":"publisher","unstructured":"Beck, A.: First-Order Methods in Optimization. Society for Industrial and Applied Mathematics, Philadelphia, PA (2017). https:\/\/doi.org\/10.1137\/1.9781611974997. https:\/\/epubs.siam.org\/doi\/abs\/10.1137\/1.9781611974997","DOI":"10.1137\/1.9781611974997"},{"key":"2100_CR4","unstructured":"Bradbury, J., Frostig, R., Hawkins, P., Johnson, M.\u00a0J., Leary, C., Maclaurin, D., Necula, G., Paszke, A., VanderPlas, J., Wanderman-Milne, S., Zhang, Q.: JAX: composable transformations of Python+NumPy programs, (2018). https:\/\/github.com\/google\/jax"},{"issue":"5","key":"2100_CR5","doi-asserted-by":"publisher","first-page":"1190","DOI":"10.1137\/0916069","volume":"16","author":"RH Byrd","year":"1995","unstructured":"Byrd, R.H., Lu, P., Nocedal, J., Zhu, C.: A limited memory algorithm for bound constrained optimization. SIAM J. Sci. Comput. 16(5), 1190\u20131208 (1995). https:\/\/doi.org\/10.1137\/0916069","journal-title":"SIAM J. Sci. Comput."},{"key":"2100_CR6","unstructured":"Carmon, Y., Duchi, J.\u00a0C., Hinder, O., Sidford, A.: Convex until proven guilty: dimension-free acceleration of gradient descent on non-convex functions. In: D.\u00a0Precup and Y.\u00a0W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume\u00a070 of Proceedings of Machine Learning Research, pp. 654\u2013663. PMLR, (06\u201311 Aug 2017). URL https:\/\/proceedings.mlr.press\/v70\/carmon17a.html"},{"issue":"2","key":"2100_CR7","doi-asserted-by":"publisher","first-page":"1751","DOI":"10.1137\/17M1114296","volume":"28","author":"Y Carmon","year":"2018","unstructured":"Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Accelerated methods for nonconvex optimization. SIAM J. Optim. 28(2), 1751\u20131772 (2018). https:\/\/doi.org\/10.1137\/17M1114296","journal-title":"SIAM J. Optim."},{"issue":"1","key":"2100_CR8","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1007\/s10107-019-01406-y","volume":"184","author":"Y Carmon","year":"2020","unstructured":"Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Lower bounds for finding stationary points I. Math. Program. 184(1), 71\u2013120 (2020). https:\/\/doi.org\/10.1007\/s10107-019-01406-y","journal-title":"Math. Program."},{"issue":"1","key":"2100_CR9","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1007\/s10107-019-01431-x","volume":"185","author":"Y Carmon","year":"2021","unstructured":"Carmon, Y., Duchi, J.C., Hinder, O., Sidford, A.: Lower bounds for finding stationary points II: first-order methods. Math. Program. 185(1), 315\u2013355 (2021). https:\/\/doi.org\/10.1007\/s10107-019-01431-x","journal-title":"Math. Program."},{"issue":"2","key":"2100_CR10","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1007\/s10107-009-0286-5","volume":"127","author":"C Cartis","year":"2011","unstructured":"Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results. Math. Program. 127(2), 245\u2013295 (2011). https:\/\/doi.org\/10.1007\/s10107-009-0286-5","journal-title":"Math. Program."},{"issue":"2","key":"2100_CR11","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1007\/s10107-009-0337-y","volume":"130","author":"C Cartis","year":"2011","unstructured":"Cartis, C., Gould, N.I.M., Toint, P.L.: Adaptive cubic regularisation methods for unconstrained optimization. Part II: worst-case function- and derivative-evaluation complexity. Math. Programm. 130(2), 295\u2013319 (2011). https:\/\/doi.org\/10.1007\/s10107-009-0337-y","journal-title":"Math. Programm."},{"issue":"6","key":"2100_CR12","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1080\/10556788.2016.1268136","volume":"32","author":"C Cartis","year":"2017","unstructured":"Cartis, C., Gould, N.I.M., Toint, P.L.: Worst-case evaluation complexity of regularization methods for smooth unconstrained optimization using H\u00f6lder continuous gradients. Optim. Methods Softw. 32(6), 1273\u20131298 (2017). https:\/\/doi.org\/10.1080\/10556788.2016.1268136","journal-title":"Optim. Methods Softw."},{"issue":"1","key":"2100_CR13","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1137\/16M1106316","volume":"29","author":"C Cartis","year":"2019","unstructured":"Cartis, C., Gould, N.I.M., Toint, P.L.: Universal regularization methods: varying the power, the smoothness and the accuracy. SIAM J. Optim. 29(1), 595\u2013615 (2019). https:\/\/doi.org\/10.1137\/16M1106316","journal-title":"SIAM J. Optim."},{"issue":"1","key":"2100_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10107-016-1026-2","volume":"162","author":"FE Curtis","year":"2017","unstructured":"Curtis, F.E., Robinson, D.P., Samadi, M.: A trust region algorithm with a worst-case iteration complexity of $$O(\\epsilon ^{-3\/2})$$ for nonconvex optimization. Math. Program. 162(1), 1\u201332 (2017). https:\/\/doi.org\/10.1007\/s10107-016-1026-2","journal-title":"Math. Program."},{"key":"2100_CR15","unstructured":"Cutkosky, A., Mehta, H.: Momentum improves normalized SGD. In: H.\u00a0D. III and A.\u00a0Singh (eds.) Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 2260\u20132268. PMLR, (13\u201318 Jul 2020). URL https:\/\/proceedings.mlr.press\/v119\/cutkosky20b.html"},{"key":"2100_CR16","unstructured":"Danilova, M., Malinovsky, G.: Averaged heavy-ball method. arXiv preprint, (2021). arxiv:2111.05430"},{"key":"2100_CR17","doi-asserted-by":"crossref","unstructured":"Danilova, M., Kulakova, A., Polyak, B.: Non-monotone behavior of the heavy ball method. In: M.\u00a0Bohner, S.\u00a0Siegmund, R.\u00a0\u0160imon\u00a0Hilscher, and P.\u00a0Stehl\u00edk (eds.) Difference Equations and Discrete Dynamical Systems with Applications, pp. 213\u2013230, Cham, (2020). Springer International Publishing. ISBN 978-3-030-35502-9","DOI":"10.1007\/978-3-030-35502-9_9"},{"issue":"1","key":"2100_CR18","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1007\/s10107-013-0677-5","volume":"146","author":"O Devolder","year":"2014","unstructured":"Devolder, O., Glineur, F., Nesterov, Y.: First-order methods of smooth convex optimization with inexact oracle. Math. Program. 146(1), 37\u201375 (2014). https:\/\/doi.org\/10.1007\/s10107-013-0677-5","journal-title":"Math. Program."},{"issue":"2","key":"2100_CR19","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1007\/BF00940007","volume":"60","author":"LCW Dixon","year":"1989","unstructured":"Dixon, L.C.W., Price, R.C.: Truncated Newton method for sparse unconstrained optimization using automatic differentiation. J. Optim. Theory Appl. 60(2), 261\u2013275 (1989). https:\/\/doi.org\/10.1007\/BF00940007","journal-title":"J. Optim. Theory Appl."},{"key":"2100_CR20","unstructured":"Dvurechensky, P.: Gradient method with inexact oracle for composite non-convex optimization. arXiv preprint, (2017). arxiv:1703.09180"},{"key":"2100_CR21","doi-asserted-by":"publisher","unstructured":"Ghadimi, E., Feyzmahdavian, H.\u00a0R., Johansson, M.: Global convergence of the heavy-ball method for convex optimization. In: 2015 European Control Conference (ECC), pp. 310\u2013315, (2015). https:\/\/doi.org\/10.1109\/ECC.2015.7330562","DOI":"10.1109\/ECC.2015.7330562"},{"issue":"3","key":"2100_CR22","doi-asserted-by":"publisher","first-page":"1854","DOI":"10.1007\/s10915-019-00915-4","volume":"79","author":"S Ghadimi","year":"2019","unstructured":"Ghadimi, S., Lan, G., Zhang, H.: Generalized uniformly optimal methods for nonlinear programming. J. Sci. Comput. 79(3), 1854\u20131881 (2019). https:\/\/doi.org\/10.1007\/s10915-019-00915-4","journal-title":"J. Sci. Comput."},{"issue":"1","key":"2100_CR23","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1137\/16M1087801","volume":"27","author":"GN Grapiglia","year":"2017","unstructured":"Grapiglia, G.N., Nesterov, Y.: Regularized Newton methods for minimizing functions with H\u00f6lder continuous Hessians. SIAM J. Optim. 27(1), 478\u2013506 (2017). https:\/\/doi.org\/10.1137\/16M1087801","journal-title":"SIAM J. Optim."},{"issue":"1","key":"2100_CR24","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1137\/17M1142077","volume":"29","author":"GN Grapiglia","year":"2019","unstructured":"Grapiglia, G.N., Nesterov, Y.: Accelerated regularized Newton methods for minimizing composite convex functions. SIAM J. Optim. 29(1), 77\u201399 (2019). https:\/\/doi.org\/10.1137\/17M1142077","journal-title":"SIAM J. Optim."},{"issue":"4","key":"2100_CR25","doi-asserted-by":"publisher","first-page":"2750","DOI":"10.1137\/19M1259432","volume":"30","author":"GN Grapiglia","year":"2020","unstructured":"Grapiglia, G.N., Nesterov, Y.: Tensor methods for minimizing convex functions with H\u00f6lder continuous higher-order derivatives. SIAM J. Optim. 30(4), 2750\u20132779 (2020). https:\/\/doi.org\/10.1137\/19M1259432","journal-title":"SIAM J. Optim."},{"key":"2100_CR26","unstructured":"Heek, J., Levskaya, A., Oliver, A., Ritter, M., Rondepierre, B., Steiner, A., van Zee, M.: Flax: a neural network library and ecosystem for JAX, (2020). https:\/\/github.com\/google\/flax"},{"issue":"2","key":"2100_CR27","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1504\/IJMMNO.2013.055204","volume":"4","author":"M Jamil","year":"2013","unstructured":"Jamil, M., Yang, X.-S.: A literature survey of benchmark functions for global optimisation problems. Int. J. Math. Model. Numer. Optim. 4(2), 150\u2013194 (2013). https:\/\/doi.org\/10.1504\/IJMMNO.2013.055204","journal-title":"Int. J. Math. Model. Numer. Optim."},{"key":"2100_CR28","unstructured":"Jin, C., Netrapalli, P., Jordan, M.\u00a0I.: Accelerated gradient descent escapes saddle points faster than gradient descent. In: S.\u00a0Bubeck, V.\u00a0Perchet, and P.\u00a0Rigollet (eds.) Proceedings of the 31st Conference On Learning Theory, volume\u00a075 of Proceedings of Machine Learning Research, pp. 1042\u20131085. PMLR, (06\u201309 Jul 2018). URL https:\/\/proceedings.mlr.press\/v75\/jin18a.html"},{"key":"2100_CR29","unstructured":"Kingma, D.\u00a0P., Ba, J.: Adam: A method for stochastic optimization. In: Y.\u00a0Bengio and Y.\u00a0LeCun, (eds.) 3rd International Conference on Learning Representations, (2015). URL http:\/\/arxiv.org\/abs\/1412.6980"},{"issue":"1","key":"2100_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10107-013-0737-x","volume":"149","author":"G Lan","year":"2015","unstructured":"Lan, G.: Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization. Math. Program. 149(1), 1\u201345 (2015). https:\/\/doi.org\/10.1007\/s10107-013-0737-x","journal-title":"Math. Program."},{"key":"2100_CR31","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-39568-1","volume-title":"First-order and Stochastic Optimization Methods for Machine Learning","author":"G Lan","year":"2020","unstructured":"Lan, G.: First-order and Stochastic Optimization Methods for Machine Learning. Springer, Cham (2020)"},{"issue":"1","key":"2100_CR32","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1137\/15M1009597","volume":"26","author":"L Lessard","year":"2016","unstructured":"Lessard, L., Recht, B., Packard, A.: Analysis and design of optimization algorithms via integral quadratic constraints. SIAM J. Optim. 26(1), 57\u201395 (2016). https:\/\/doi.org\/10.1137\/15M1009597","journal-title":"SIAM J. Optim."},{"key":"2100_CR33","unstructured":"Li, H., Lin, Z.: Restarted nonconvex accelerated gradient descent: No more polylogarithmic factor in the $$O(\\epsilon ^{-7\/4})$$ complexity. In: K.\u00a0Chaudhuri, S.\u00a0Jegelka, L.\u00a0Song, C.\u00a0Szepesvari, G.\u00a0Niu, and S.\u00a0Sabato, (eds.) Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 12901\u201312916. PMLR, (17\u201323 Jul 2022). URL https:\/\/proceedings.mlr.press\/v162\/li22o.html"},{"issue":"157","key":"2100_CR34","first-page":"1","volume":"24","author":"H Li","year":"2023","unstructured":"Li, H., Lin, Z.: Restarted nonconvex accelerated gradient descent: No more polylogarithmic factor in the $$O(\\epsilon ^{-7\/4})$$ complexity. J. Mach. Learn. Res. 24(157), 1\u201337 (2023)","journal-title":"J. Mach. Learn. Res."},{"key":"2100_CR35","unstructured":"Marumo, N., Takeda, A.: Parameter-free accelerated gradient descent for nonconvex minimization. To appear in SIAM J. Optim. arxiv:2212.06410"},{"issue":"3","key":"2100_CR36","first-page":"372","volume":"269","author":"Y Nesterov","year":"1983","unstructured":"Nesterov, Y.: A method for solving a convex programming problem with convergence rate $$O(1\/k^2)$$. Soviet Mathematics Doklady 269(3), 372\u2013376 (1983)","journal-title":"Soviet Mathematics Doklady"},{"key":"2100_CR37","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-8853-9","volume-title":"Introductory Lectures on Convex Optimization: A Basic Course","author":"Y Nesterov","year":"2004","unstructured":"Nesterov, Y.: Introductory Lectures on Convex Optimization: A Basic Course. Springer, New York (2004)"},{"issue":"1","key":"2100_CR38","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1007\/s10107-014-0790-0","volume":"152","author":"Y Nesterov","year":"2015","unstructured":"Nesterov, Y.: Universal gradient methods for convex optimization problems. Math. Program. 152(1), 381\u2013404 (2015). https:\/\/doi.org\/10.1007\/s10107-014-0790-0","journal-title":"Math. Program."},{"key":"2100_CR39","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-91578-4","volume-title":"Lectures on Convex Optimization","author":"Y Nesterov","year":"2018","unstructured":"Nesterov, Y.: Lectures on Convex Optimization, vol. 137. Springer, Cham (2018)"},{"issue":"1","key":"2100_CR40","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1007\/s10107-006-0706-8","volume":"108","author":"Y Nesterov","year":"2006","unstructured":"Nesterov, Y., Polyak, B.T.: Cubic regularization of Newton method and its global performance. Math. Program. 108(1), 177\u2013205 (2006). https:\/\/doi.org\/10.1007\/s10107-006-0706-8","journal-title":"Math. Program."},{"issue":"2","key":"2100_CR41","doi-asserted-by":"publisher","first-page":"1388","DOI":"10.1137\/130942954","volume":"7","author":"P Ochs","year":"2014","unstructured":"Ochs, P., Chen, Y., Brox, T., Pock, T.: iPiano: Inertial proximal algorithm for nonconvex optimization. SIAM J. Imag. Sci. 7(2), 1388\u20131419 (2014). https:\/\/doi.org\/10.1137\/130942954","journal-title":"SIAM J. Imag. Sci."},{"issue":"1","key":"2100_CR42","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s10107-018-1340-y","volume":"176","author":"M O\u2019Neill","year":"2019","unstructured":"O\u2019Neill, M., Wright, S.J.: Behavior of accelerated gradient methods near critical points of nonconvex functions. Math. Program. 176(1), 403\u2013427 (2019). https:\/\/doi.org\/10.1007\/s10107-018-1340-y","journal-title":"Math. Program."},{"issue":"5","key":"2100_CR43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/0041-5553(64)90137-5","volume":"4","author":"BT Polyak","year":"1964","unstructured":"Polyak, B.T.: Some methods of speeding up the convergence of iteration methods. USSR Comput. Math. Math. Phys. 4(5), 1\u201317 (1964)","journal-title":"USSR Comput. Math. Math. Phys."},{"issue":"2","key":"2100_CR44","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1093\/comjnl\/5.2.147","volume":"5","author":"MJD Powell","year":"1962","unstructured":"Powell, M.J.D.: An iterative method for finding stationary values of a function of several variables. Comput. J. 5(2), 147\u2013151 (1962)","journal-title":"Comput. J."},{"issue":"1","key":"2100_CR45","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1109\/TGRS.2005.859347","volume":"44","author":"A Qing","year":"2006","unstructured":"Qing, A.: Dynamic differential evolution strategy and applications in electromagnetic inverse scattering problems. IEEE Trans. Geosci. Remote Sens. 44(1), 116\u2013125 (2006). https:\/\/doi.org\/10.1109\/TGRS.2005.859347","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"2100_CR46","unstructured":"Reddi, S.\u00a0J., Kale, S., Kumar, S.: On the convergence of Adam and beyond. In International Conference on Learning Representations, (2018). URL https:\/\/openreview.net\/forum?id=ryQu7f-RZ"},{"issue":"3","key":"2100_CR47","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1093\/comjnl\/3.3.175","volume":"3","author":"HH Rosenbrock","year":"1960","unstructured":"Rosenbrock, H.H.: An automatic method for finding the greatest or least value of a function. Comput. J. 3(3), 175\u2013184 (1960)","journal-title":"Comput. J."},{"issue":"2","key":"2100_CR48","doi-asserted-by":"publisher","first-page":"1448","DOI":"10.1137\/17M1134329","volume":"28","author":"CW Royer","year":"2018","unstructured":"Royer, C.W., Wright, S.J.: Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim. 28(2), 1448\u20131477 (2018). https:\/\/doi.org\/10.1137\/17M1134329","journal-title":"SIAM J. Optim."},{"issue":"1","key":"2100_CR49","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1007\/s10107-019-01362-7","volume":"180","author":"CW Royer","year":"2020","unstructured":"Royer, C.W., O\u2019Neill, M., Wright, S.J.: A Newton-CG algorithm with complexity guarantees for smooth unconstrained optimization. Math. Program. 180(1), 451\u2013488 (2020). https:\/\/doi.org\/10.1007\/s10107-019-01362-7","journal-title":"Math. Program."},{"key":"2100_CR50","unstructured":"Sutskever, I., Martens, J., Dahl, G., Hinton, G.: On the importance of initialization and momentum in deep learning. In: S.\u00a0Dasgupta and D.\u00a0McAllester (eds.) Proceedings of the 30th International Conference on Machine Learning, volume\u00a028 of Proceedings of Machine Learning Research, pp. 1139\u20131147, Atlanta, Georgia, USA, 17\u201319 Jun 2013. PMLR. URL https:\/\/proceedings.mlr.press\/v28\/sutskever13.html"},{"key":"2100_CR51","unstructured":"Tu, S., Boczar, R., Simchowitz, M., Soltanolkotabi, M., Recht, B.: Low-rank solutions of linear matrix equations via Procrustes flow. In: M.\u00a0F. Balcan and K.\u00a0Q. Weinberger (eds.) Proceedings of The 33rd International Conference on Machine Learning, volume\u00a048 of Proceedings of Machine Learning Research, pp. 964\u2013973, New York, New York, USA, (20\u201322 Jun 2016). PMLR. URL https:\/\/proceedings.mlr.press\/v48\/tu16.html"},{"key":"2100_CR52","doi-asserted-by":"crossref","unstructured":"Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, \u0130, Feng, Y., Moore, E.W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R., Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P.: Fundamental algorithms for scientific computing in python. Nat. Methods 17, 261\u2013272 (2020)","DOI":"10.1038\/s41592-020-0772-5"},{"key":"2100_CR53","unstructured":"Xu, Y., Jin, R., Yang, T.: NEON+: Accelerated gradient methods for extracting negative curvature for non-convex optimization. arXiv preprint, (2017). arxiv:1712.01033"}],"container-title":["Mathematical Programming"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10107-024-02100-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10107-024-02100-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10107-024-02100-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:02:54Z","timestamp":1750176174000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10107-024-02100-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,4]]},"references-count":53,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2025,7]]}},"alternative-id":["2100"],"URL":"https:\/\/doi.org\/10.1007\/s10107-024-02100-4","relation":{},"ISSN":["0025-5610","1436-4646"],"issn-type":[{"value":"0025-5610","type":"print"},{"value":"1436-4646","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,4]]},"assertion":[{"value":"2 March 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 June 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}