{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,10]],"date-time":"2026-01-10T01:02:54Z","timestamp":1768006974789,"version":"3.49.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2025,2,17]],"date-time":"2025-02-17T00:00:00Z","timestamp":1739750400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,17]],"date-time":"2025-02-17T00:00:00Z","timestamp":1739750400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"GNCS Progetti 2023"},{"name":"GNCS Progetti 2023"},{"name":"PNRR FAIR","award":["PE00000013"],"award-info":[{"award-number":["PE00000013"]}]},{"name":"PNRR FAIR","award":["PE00000013"],"award-info":[{"award-number":["PE00000013"]}]},{"name":"Science Fund of the Republic of Serbia, Project LASCADO","award":["7359"],"award-info":[{"award-number":["7359"]}]},{"DOI":"10.13039\/501100007109","name":"Universit\u00e0 degli Studi di Ferrara","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007109","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Optim Appl"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>In this paper, we propose a new stochastic gradient method for numerical minimization of finite sums. We also propose a modified version of this method applicable on more general problems referred to as infinite sum problems, where the objective function is in the form of mathematical expectation. The method is based on a strategy to exploit the effectiveness of the well-known Barzilai\u2013Borwein (BB) rules or variants of these (BB-like) rules for updating the step length in the standard gradient method. The proposed method adapts the aforementioned strategy into the stochastic framework by exploiting the same Sample Average Approximations (SAA) estimator of the objective function for several iterations. Furthermore, the sample size is controlled by an additional sampling which also plays a role in accepting the proposed iterate point. Moreover, the number of \u201cinner\u201d iterations with the same sample is also controlled by an adaptive rule which prevents the method from getting stuck with the same estimator for too long. Convergence results are discussed for the finite and infinite sum version, for general and strongly convex objective functions. For the strongly convex case, we provide convergence rate and worst-case complexity analysis. Numerical experiments on well-known datasets for binary classifications show very promising performance of the method, without the need to provide special values for hyperparameters on which the method depends.<\/jats:p>","DOI":"10.1007\/s10589-025-00664-1","type":"journal-article","created":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T03:42:36Z","timestamp":1739850156000},"page":"717-758","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Spectral stochastic gradient method with additional sampling for finite and infinite sums"],"prefix":"10.1007","volume":"91","author":[{"given":"Nata\u0161a","family":"Krklec Jerinki\u0107","sequence":"first","affiliation":[]},{"given":"Valeria","family":"Ruggiero","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5737-1679","authenticated-orcid":false,"given":"Ilaria","family":"Trombini","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,2,17]]},"reference":[{"key":"664_CR1","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1093\/imanum\/8.1.141","volume":"8","author":"J Barzilai","year":"1988","unstructured":"Barzilai, J., Borwein, J.M.: Two point step size gradient methods. IMA J. Numer. Anal. 8, 141\u2013148 (1988)","journal-title":"IMA J. Numer. Anal."},{"key":"664_CR2","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/s10589-006-6446-0","volume":"35","author":"B Zhou","year":"2006","unstructured":"Zhou, B., Gao, L., Dai, Y.: Gradient methods with adaptive step-sizes. Comput. Optim. Appl. 35, 69\u201386 (2006)","journal-title":"Comput. Optim. Appl."},{"key":"664_CR3","doi-asserted-by":"publisher","first-page":"299","DOI":"10.3934\/jimo.2008.4.299","volume":"4","author":"G Frassoldati","year":"2008","unstructured":"Frassoldati, G., Zanghirati, G., Zanni, L.: New adaptive stepsize selections in gradient methods. J. Ind. Manag. Optim. 4, 299\u2013312 (2008)","journal-title":"J. Ind. Manag. Optim."},{"key":"664_CR4","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1137\/S1052623494266365","volume":"7","author":"M Raydan","year":"1997","unstructured":"Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26\u201333 (1997)","journal-title":"SIAM J. Optim."},{"key":"664_CR5","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1023\/A:1013653923062","volume":"112","author":"Y Dai","year":"2002","unstructured":"Dai, Y.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315\u2013330 (2002)","journal-title":"J. Optim. Theory Appl."},{"key":"664_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/imanum\/22.1.1","volume":"22","author":"Y Dai","year":"2002","unstructured":"Dai, Y., Liao, L.: R-linear convergence of the Barzilai and Borwein gradient method. IMA J. Numer. Anal. 22, 1\u201310 (2002)","journal-title":"IMA J. Numer. Anal."},{"key":"664_CR7","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1090\/mcom\/3802","volume":"92","author":"D di Serafino","year":"2023","unstructured":"di Serafino, D., Kreji\u0107, N., Krklec Jerinki\u0107, N., Viola, M.: LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums. Math. Comput. 92, 1273\u20131299 (2023)","journal-title":"Math. Comput."},{"key":"664_CR8","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2024.2426620","author":"S Bellavia","year":"2024","unstructured":"Bellavia, S., Kreji\u0107, N., Krklec Jerinki\u0107, N., Raydan, M.: SLiSeS: Subsampled Line Search Spectral gradient method for finite sums. Optim. Methods Softw. (2024). https:\/\/doi.org\/10.1080\/10556788.2024.2426620","journal-title":"Optim. Methods Softw."},{"key":"664_CR9","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1007\/s10589-024-00580-w","volume":"89","author":"N Kreji\u0107","year":"2024","unstructured":"Kreji\u0107, N., Krklec Jerinki\u0107, N., Mart\u00ednez, A., Yousefi, M.: A non-monotone trust-region method with noisy oracles and additional sampling. Comput. Optim. Appl. 89, 247\u2013278 (2024)","journal-title":"Comput. Optim. Appl."},{"key":"664_CR10","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1137\/0723046","volume":"23","author":"L Grippo","year":"1986","unstructured":"Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line-search technique for Newton\u2019s method. SIAM J. Numer. Anal. 23, 707\u2013716 (1986)","journal-title":"SIAM J. Numer. Anal."},{"key":"664_CR11","doi-asserted-by":"publisher","first-page":"1196","DOI":"10.1137\/S1052623497330963","volume":"10","author":"E Birgin","year":"2000","unstructured":"Birgin, E., Mart\u00ecnez, J., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM J. Optim. 10, 1196\u20131211 (2000)","journal-title":"SIAM J. Optim."},{"key":"664_CR12","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1007\/s10589-021-00331-1","volume":"81","author":"O Ferreira","year":"2022","unstructured":"Ferreira, O., Lemes, M., Prudente, L.: On the inexact scaled gradient projection method. Comput. Optim. Appl. 81, 91\u2013125 (2022)","journal-title":"Comput. Optim. Appl."},{"key":"664_CR13","volume-title":"Probability Theory: A Comprehensive Course","author":"A Klenke","year":"2007","unstructured":"Klenke, A.: Probability Theory: A Comprehensive Course. Springer, Berlin (2007)"},{"key":"664_CR14","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1007\/s11075-014-9869-1","volume":"68","author":"N Kreji\u0107","year":"2015","unstructured":"Kreji\u0107, N., Krklec Jerinki\u0107, N.: Non-monotone line search methods with variable sample size. Numer. Algorithms 68, 711\u2013739 (2015)","journal-title":"Numer. Algorithms"},{"key":"664_CR15","doi-asserted-by":"crossref","unstructured":"Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming. Soc. Ind. Appl. Math. (2009)","DOI":"10.1137\/1.9780898718751"},{"key":"664_CR16","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1007\/s11075-022-01419-3","volume":"93","author":"N Kreji\u0107","year":"2022","unstructured":"Kreji\u0107, N., Krklec Jerinki\u0107, N., Ostoji\u0107, T.: Spectral projected subgradient method for nonsmooth convex optimization problems. Numer. Algorithms 93, 347\u2013365 (2022)","journal-title":"Numer. Algorithms"},{"key":"664_CR17","first-page":"516","volume":"147","author":"S Ram","year":"2011","unstructured":"Ram, S., Nedic, A., Veeravalli, V.: Distributed stochastic subgradient projection algorithms for convex optimization. J. Optim. Theory Appl. 147, 516\u2013545 (2011)","journal-title":"J. Optim. Theory Appl."},{"key":"664_CR18","first-page":"176","volume":"318","author":"D di Serafino","year":"2018","unstructured":"di Serafino, D., Ruggiero, V., Toraldo, G., Zanni, L.: On the steplength selection in gradient methods for unconstrained optimization. Appl. Math. Comput. 318, 176\u2013195 (2018)","journal-title":"Appl. Math. Comput."},{"key":"664_CR19","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1137\/16M1080173","volume":"60","author":"L Bottou","year":"2018","unstructured":"Bottou, L., Curtis, F., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Rev. 60, 223\u2013311 (2018)","journal-title":"SIAM Rev."},{"key":"664_CR20","unstructured":"Nguyen, L., Liu, J., Scheinberg, K., Tak\u00e1\u010d, M.: SARAH: A novel method for machine learning problems using stochastic recursive gradient. arXiv:1703.00102v2 (2017)"},{"key":"664_CR21","unstructured":"Liu, Y., Gao, Y., Yin, W.: An improved analysis of stochastic gradient descent with momentum. In: Proceedings of the 34th International Conference On Neural Information Processing Systems. (2020)"},{"key":"664_CR22","doi-asserted-by":"publisher","first-page":"3312","DOI":"10.1137\/17M1154679","volume":"28","author":"R Bollapragada","year":"2018","unstructured":"Bollapragada, R., Byrd, R., Nocedal, J.: Adaptive sampling strategies for stochastic optimization. SIAM J. Optim. 28, 3312\u20133343 (2018)","journal-title":"SIAM J. Optim."},{"key":"664_CR23","unstructured":"Tavakolizadeh, F., Soto, J., Gyulai, D., Beecks, C.: Industry 4.0: mining physical defects in production of surface-mount devices. In: Advances in Data Mining: 17th Industrial Conference (2017). https:\/\/api.semanticscholar.org\/CorpusID:117308853. https:\/\/api.semanticscholar.org\/CorpusID:117308853"},{"key":"664_CR24","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1080\/0951192X.2019.1571238","volume":"32","author":"JA Carvajal Soto","year":"2019","unstructured":"Carvajal Soto, J.A., Tavakolizadeh, F., Gyulai, D.: An online machine learning framework for early detection of product failures in an Industry 4.0 context. Int. J. Comput. Integr. Manuf. 32, 452\u2013465 (2019)","journal-title":"Int. J. Comput. Integr. Manuf."},{"key":"664_CR25","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.1016\/j.neucom.2007.11.023","volume":"71","author":"S Agarwal","year":"2008","unstructured":"Agarwal, S., Vijaya Saradhi, V., Karnick, H.: Kernel-based online machine learning and support vector reduction. Neurocomputing 71, 1230\u20131237 (2008)","journal-title":"Neurocomputing"}],"container-title":["Computational Optimization and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-025-00664-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10589-025-00664-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-025-00664-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,17]],"date-time":"2025-05-17T12:32:18Z","timestamp":1747485138000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10589-025-00664-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,17]]},"references-count":25,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["664"],"URL":"https:\/\/doi.org\/10.1007\/s10589-025-00664-1","relation":{},"ISSN":["0926-6003","1573-2894"],"issn-type":[{"value":"0926-6003","type":"print"},{"value":"1573-2894","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,17]]},"assertion":[{"value":"9 October 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}