{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T07:41:48Z","timestamp":1773387708812,"version":"3.50.1"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T00:00:00Z","timestamp":1667433600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004434","name":"Universit\u00e0 degli Studi di Firenze","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004434","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Optim Appl"],"published-print":{"date-parts":[[2023,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. Using a suitable reformulation of the given problem, our method combines the inexact restoration approach for constrained optimization with the trust-region procedure and random models. Differently from other recent stochastic trust-region schemes, our proposed algorithm improves feasibility and optimality in a modular way. We provide the expected number of iterations for reaching a near-stationary point by imposing some probability accuracy requirements on random functions and gradients which are, in general, less stringent than the corresponding ones in literature. We validate the proposed algorithm on some nonconvex optimization problems arising in binary classification and regression, showing that it performs well in terms of cost and accuracy, and allows to reduce the burdensome tuning of the hyper-parameters involved.<\/jats:p>","DOI":"10.1007\/s10589-022-00430-7","type":"journal-article","created":{"date-parts":[[2022,11,3]],"date-time":"2022-11-03T13:03:10Z","timestamp":1667480590000},"page":"53-84","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["A stochastic first-order trust-region method with inexact restoration for finite-sum minimization"],"prefix":"10.1007","volume":"84","author":[{"given":"Stefania","family":"Bellavia","sequence":"first","affiliation":[]},{"given":"Nata\u0161a","family":"Kreji\u0107","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9213-3622","authenticated-orcid":false,"given":"Benedetta","family":"Morini","sequence":"additional","affiliation":[]},{"given":"Simone","family":"Rebegoldi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,11,3]]},"reference":[{"key":"430_CR1","volume-title":"Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging","author":"S Bellavia","year":"2021","unstructured":"Bellavia, S., Bianconcini, T., Kreji\u0107, N., Morini, B.: Subsampled first-order optimization methods with applications in imaging. In: Chen, K., Schonlieb, C., Tai, X., Younces, L. (eds.) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Switzerland AG (2021)"},{"key":"430_CR2","doi-asserted-by":"crossref","unstructured":"Curtis, F.E., Scheinberg, K.: Optimization methods for supervised machine learning: from linear models to deep learning. Leading Developments from INFORMS Communities. INFORMS, pp. 89\u2013114 (2017)","DOI":"10.1287\/educ.2017.0168"},{"issue":"2","key":"430_CR3","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1137\/16M1080173","volume":"60","author":"L Bottou","year":"2018","unstructured":"Bottou, L., Curtis, F.C., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Rev. 60(2), 223\u2013311 (2018)","journal-title":"SIAM Rev."},{"key":"430_CR4","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1214\/aoms\/1177729586","volume":"22","author":"H Robbins","year":"1951","unstructured":"Robbins, H., Monro, S.: A stochastic approximation method. Ann. Math. Stat. 22, 400\u2013407 (1951)","journal-title":"Ann. Math. Stat."},{"key":"430_CR5","doi-asserted-by":"crossref","unstructured":"Gower, R.M., Schmidt, M., Bach, F., Richtarik, P.: Variance-reduced methods for machine learning. In Proceedings of the IEEE, vol. 108, pp. 1968\u20131983 (2020)","DOI":"10.1109\/JPROC.2020.3028013"},{"key":"430_CR6","unstructured":"Johnson, R., Zhang, T.: Accelerating stochastic gradient descent using predictive variance reduction. In Proceedings of the 26th International Conference on Neural Information Processing Systems, vol. 1, pp. 315\u2013323 (2013)"},{"key":"430_CR7","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1007\/s10107-016-1030-6","volume":"162","author":"M Schmidt","year":"2017","unstructured":"Schmidt, M., Le Roux, N., Bach, F.: Minimizing finite sums with the stochastic average gradient. Math. Prog. 162, 83\u2013112 (2017)","journal-title":"Math. Prog."},{"key":"430_CR8","unstructured":"Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2015)"},{"key":"430_CR9","unstructured":"Nguyen, L.M., Liu, J., Scheinberg, K., Taka\u010d, M.: Sarah: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient. In Proceedings of the 34th International Conference on Machine Learning, PMLR, vol. 70, pp. 2613\u20132621 (2017)"},{"issue":"5","key":"430_CR10","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1109\/MSP.2020.3003539","volume":"37","author":"FE Curtis","year":"2020","unstructured":"Curtis, F.E., Scheinberg, K.: Adaptive stochastic optimization: a framework for analyzing stochastic optimization algorithms. IEEE Signal Process. Mag. 37(5), 32\u201342 (2020)","journal-title":"IEEE Signal Process. Mag."},{"issue":"1","key":"430_CR11","doi-asserted-by":"publisher","first-page":"764","DOI":"10.1093\/imanum\/drz076","volume":"41","author":"S Bellavia","year":"2021","unstructured":"Bellavia, S., Gurioli, G., Morini, B.: Adaptive cubic regularization methods with dynamic inexact hessian information and applications to finite-sum minimization. IMA J. Numer. Anal. 41(1), 764\u2013799 (2021)","journal-title":"IMA J. Numer. Anal."},{"key":"430_CR12","doi-asserted-by":"publisher","first-page":"101591","DOI":"10.1016\/j.jco.2021.101591","volume":"68","author":"S Bellavia","year":"2022","unstructured":"Bellavia, S., Gurioli, G., Morini, B., Toint, P. L.: Adaptive regularization for nonconvex optimization using inexact function values and randomly perturbed derivatives. J. Complex 68, 101591 (2022)","journal-title":"J. Complex"},{"key":"430_CR13","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1287\/ijoo.2019.0016","volume":"1","author":"J Blanchet","year":"2019","unstructured":"Blanchet, J., Cartis, C., Menickelly, M., Scheinberg, K.: Convergence rate analysis of a stochastic trust region method via submartingales. INFORMS J. Optim. 1, 92\u2013119 (2019)","journal-title":"INFORMS J. Optim."},{"issue":"2","key":"430_CR14","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1007\/s10107-017-1141-8","volume":"169","author":"R Chen","year":"2018","unstructured":"Chen, R., Menickelly, M., Scheinberg, K.: Stochastic optimization using a trust-region method and random models. Math. Program. 169(2), 447\u2013487 (2018)","journal-title":"Math. Program."},{"key":"430_CR15","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1137\/18M1216250","volume":"30","author":"C Paquette","year":"2020","unstructured":"Paquette, C., Scheinberg, K.: A stochastic line search method with expected complexity analysis. SIAM J. Optim. 30, 349\u2013376 (2020)","journal-title":"SIAM J. Optim."},{"key":"430_CR16","doi-asserted-by":"publisher","first-page":"1775","DOI":"10.1090\/mcom\/3025","volume":"85","author":"N Kreji\u0107","year":"2016","unstructured":"Kreji\u0107, N., Mart\u00ednez, J.M.: Inexact restoration approach for minimization with inexact evaluation of the objective function. Math. Comput. 85, 1775\u20131791 (2016)","journal-title":"Math. Comput."},{"issue":"311","key":"430_CR17","doi-asserted-by":"publisher","first-page":"1307","DOI":"10.1090\/mcom\/3246","volume":"87","author":"GE Birgin","year":"2018","unstructured":"Birgin, G.E., Kreji\u0107, N., Mart\u00ednez, J.M.: On the employment of inexact restoration for the minimization of functions whose evaluation is subject to programming errors. Math. Comput. 87(311), 1307\u20131326 (2018)","journal-title":"Math. Comput."},{"key":"430_CR18","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1090\/mcom\/3445","volume":"89","author":"GE Birgin","year":"2020","unstructured":"Birgin, G.E., Kreji\u0107, N., Mart\u00ednez, J.M.: Iteration and evaluation complexity on the minimization of functions whose computation is intrinsically inexact. Math. Comput. 89, 253\u2013278 (2020)","journal-title":"Math. Comput."},{"key":"430_CR19","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1007\/s10589-020-00196-w","volume":"76","author":"S Bellavia","year":"2020","unstructured":"Bellavia, S., Kreji\u0107, N., Morini, B.: Inexact restoration with subsampled trust-region methods for finite-sum minimization. Comput. Optim. Appl. 76, 701\u2013736 (2020)","journal-title":"Comput. Optim. Appl."},{"issue":"3","key":"430_CR20","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1137\/130915984","volume":"24","author":"AS Bandeira","year":"2014","unstructured":"Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Convergence of trust-region methods based on probabilistic models. SIAM J. Optim. 24(3), 1238\u20131264 (2014)","journal-title":"SIAM J. Optim."},{"key":"430_CR21","doi-asserted-by":"crossref","unstructured":"Bellavia, S., Gurioli, G., Morini, B., Toint, P.L.: Trust-region algorithms: probabilistic complexity and intrinsic noise with applications to subsampling techniques. EURO J. Comput. Optim. 10, 100043 (2022)","DOI":"10.1016\/j.ejco.2022.100043"},{"issue":"2","key":"430_CR22","doi-asserted-by":"crossref","first-page":"294","DOI":"10.4208\/jcm.2012-m2020-0144","volume":"40","author":"W Xiaoyu","year":"2022","unstructured":"Xiaoyu, W., Yuan, Y.X.: Stochastic trust region methods with trust region radius depending on probabilistic models. J. Comput. Math. 40(2), 294\u2013334 (2022)","journal-title":"J. Comput. Math."},{"issue":"7","key":"430_CR23","doi-asserted-by":"publisher","first-page":"1541","DOI":"10.1007\/s13042-019-01055-9","volume":"11","author":"VK Chauhan","year":"2020","unstructured":"Chauhan, V.K., Sharma, A., Dahiya, K.: Stochastic trust region inexact newton method for large-scale machine learning. Int. J. Mach. Learn. Cybern. 11(7), 1541\u20131555 (2020)","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"430_CR24","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1023\/A:1004632923654","volume":"104","author":"JM Mart\u00ednez","year":"2000","unstructured":"Mart\u00ednez, J.M., Pilotta, E.A.: Inexact restoration algorithms for constrained optimization. J. Optim. Theory Appl. 104, 135\u2013163 (2000)","journal-title":"J. Optim. Theory Appl."},{"key":"430_CR25","unstructured":"Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press (2016) http:\/\/www.deeplearningbook.org"},{"issue":"2","key":"430_CR26","doi-asserted-by":"publisher","first-page":"1489","DOI":"10.1137\/19M1291832","volume":"31","author":"AS Beharas","year":"2021","unstructured":"Beharas, A.S., Cao, L., Scheinberg, K.: Global convergence rate analysis of a generic line search algorithm with noise. SIAM J. Optim. 31(2), 1489\u20131518 (2021)","journal-title":"SIAM J. Optim."},{"key":"430_CR27","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s10107-017-1137-4","volume":"169","author":"C Cartis","year":"2018","unstructured":"Cartis, C., Scheinberg, K.: Global convergence rate analysis of unconstrained optimization methods based on probabilistic models. Math. Progr. 169, 337\u2013375 (2018)","journal-title":"Math. Progr."},{"key":"430_CR28","volume-title":"Nonlinear Programming","year":"2016","unstructured":"Bertsekas, D.P. (ed.): Nonlinear Programming. Athena Scientific, Belmont, Massachusetts (2016)"},{"key":"430_CR29","unstructured":"Lichman, M.: UCI machine learning repository. (2013) https:\/\/archive.ics.uci.edu\/ml\/index.php"},{"key":"430_CR30","doi-asserted-by":"crossref","unstructured":"Chang, C.C., Lin, C.J.: LIBSVM : a library for support vector machines (2011) http:\/\/www.csie.ntu.edu.tw\/cjlin\/libsvm","DOI":"10.1145\/1961189.1961199"},{"key":"430_CR31","doi-asserted-by":"crossref","unstructured":"LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In Proceedings of the IEEE, vol. 86, pp. 2278\u20132324 (1998)","DOI":"10.1109\/5.726791"},{"key":"430_CR32","doi-asserted-by":"crossref","unstructured":"Xu, P., Roosta-Khorasani, F., Mahoney, M.W.: Second-order optimization for non-convex machine learning: an empirical study. In Proceedings of the 2020 SIAM International Conference on Data Mining, pp. 199\u2013207 (2020)","DOI":"10.1137\/1.9781611976236.23"},{"issue":"3","key":"430_CR33","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1287\/ijoo.2018.0010","volume":"1","author":"FE Curtis","year":"2019","unstructured":"Curtis, F.E., Scheinberg, K., Shi, R.: A stochastic trust region algorithm based on careful step normalization. INFORMS J. Optim. 1(3), 200\u2013220 (2019)","journal-title":"INFORMS J. Optim."},{"key":"430_CR34","doi-asserted-by":"publisher","first-page":"750","DOI":"10.1016\/j.snb.2007.09.060","volume":"129","author":"S De Vito","year":"2008","unstructured":"De Vito, S., Massera, E., Piga, M., Martinotto, L., Di Francia, G.: On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario. Sens. Actuator B 129, 750\u2013757 (2008)","journal-title":"Sens. Actuator B"}],"container-title":["Computational Optimization and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-022-00430-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10589-022-00430-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-022-00430-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,30]],"date-time":"2023-11-30T01:02:43Z","timestamp":1701306163000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10589-022-00430-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,3]]},"references-count":34,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1]]}},"alternative-id":["430"],"URL":"https:\/\/doi.org\/10.1007\/s10589-022-00430-7","relation":{},"ISSN":["0926-6003","1573-2894"],"issn-type":[{"value":"0926-6003","type":"print"},{"value":"1573-2894","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,3]]},"assertion":[{"value":"6 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 October 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 November 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}