{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T07:41:43Z","timestamp":1773387703805,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T00:00:00Z","timestamp":1672185600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"INdAM GNCS project \u201cOttimizzazione adattiva per il machine learning\u201d","award":["CUP_E55F22000270001"],"award-info":[{"award-number":["CUP_E55F22000270001"]}]},{"name":"INdAM GNCS project \u201cOttimizzazione adattiva per il machine learning\u201d","award":["RS19MO05"],"award-info":[{"award-number":["RS19MO05"]}]},{"name":"Mobility Project \u201cSecond order methods for optimization problems in Machine Learning\u201d","award":["CUP_E55F22000270001"],"award-info":[{"award-number":["CUP_E55F22000270001"]}]},{"name":"Mobility Project \u201cSecond order methods for optimization problems in Machine Learning\u201d","award":["RS19MO05"],"award-info":[{"award-number":["RS19MO05"]}]},{"name":"IEA CNRS project entitled \u201cVaMOS\u201d","award":["CUP_E55F22000270001"],"award-info":[{"award-number":["CUP_E55F22000270001"]}]},{"name":"IEA CNRS project entitled \u201cVaMOS\u201d","award":["RS19MO05"],"award-info":[{"award-number":["RS19MO05"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Axioms"],"abstract":"<jats:p>We study the convergence properties of SIRTR, a stochastic inexact restoration trust-region method suited for the minimization of a finite sum of continuously differentiable functions. This method combines the trust-region methodology with random function and gradient estimates formed by subsampling. Unlike other existing schemes, it forces the decrease of a merit function by combining the function approximation with an infeasibility term, the latter of which measures the distance of the current sample size from its maximum value. In a previous work, the expected iteration complexity to satisfy an approximate first-order optimality condition was given. Here, we elaborate on the convergence analysis of SIRTR and prove its convergence in probability under suitable accuracy requirements on random function and gradient estimates. Furthermore, we report the numerical results obtained on some nonconvex classification test problems, discussing the impact of the probabilistic requirements on the selection of the sample sizes.<\/jats:p>","DOI":"10.3390\/axioms12010038","type":"journal-article","created":{"date-parts":[[2022,12,29]],"date-time":"2022-12-29T02:54:42Z","timestamp":1672282482000},"page":"38","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["On the Convergence Properties of a Stochastic Trust-Region Method with Inexact Restoration"],"prefix":"10.3390","volume":"12","author":[{"given":"Stefania","family":"Bellavia\u00a0","sequence":"first","affiliation":[{"name":"Dipartimento di Ingegneria Industriale, Universit\u00e0 degli studi di Firenze, Viale G.B. Morgagni 40, 50134 Firenze, Italy"},{"name":"INDAM-GNCS Research Group, P.le Aldo Moro 5, 00185 Roma, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9213-3622","authenticated-orcid":false,"given":"Benedetta","family":"Morini\u00a0","sequence":"additional","affiliation":[{"name":"Dipartimento di Ingegneria Industriale, Universit\u00e0 degli studi di Firenze, Viale G.B. Morgagni 40, 50134 Firenze, Italy"},{"name":"INDAM-GNCS Research Group, P.le Aldo Moro 5, 00185 Roma, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1232-5481","authenticated-orcid":false,"given":"Simone","family":"Rebegoldi","sequence":"additional","affiliation":[{"name":"Dipartimento di Ingegneria Industriale, Universit\u00e0 degli studi di Firenze, Viale G.B. Morgagni 40, 50134 Firenze, Italy"},{"name":"INDAM-GNCS Research Group, P.le Aldo Moro 5, 00185 Roma, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,28]]},"reference":[{"key":"ref_1","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_2","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1137\/16M1080173","article-title":"Optimization Methods for Large-Scale Machine Learning","volume":"60","author":"Bottou","year":"2018","journal-title":"SIAM Rev."},{"key":"ref_4","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1007\/s10107-016-1030-6","article-title":"Minimizing Finite Sums with the Stochastic Average Gradient","volume":"162","author":"Schmidt","year":"2017","journal-title":"Math. Program."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1137\/130915984","article-title":"Convergence of trust-region methods based on probabilistic models","volume":"24","author":"Bandeira","year":"2014","journal-title":"SIAM J. Optim."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101591","DOI":"10.1016\/j.jco.2021.101591","article-title":"Adaptive regularization algorithm for nonconvex optimization using inexact function evaluations and randomly perturbed derivatives","volume":"68","author":"Bellavia","year":"2022","journal-title":"J. Complex."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"100043","DOI":"10.1016\/j.ejco.2022.100043","article-title":"Trust-region algorithms: Probabilistic complexity and intrinsic noise with applications to subsampling techniques","volume":"10","author":"Bellavia","year":"2022","journal-title":"EURO J. Comput. Optim."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2309","DOI":"10.1093\/imanum\/drz027","article-title":"Subsampled Inexact Newton methods for minimizing large sums of convex functions","volume":"40","author":"Bellavia","year":"2020","journal-title":"IMA J. Numer. Anal."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1007\/s10589-020-00196-w","article-title":"Inexact restoration with subsampled trust-region methods for finite-sum minimization","volume":"73","author":"Bellavia","year":"2020","journal-title":"Comp. Opt. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bellavia, S., Kreji\u0107, N., Morini, B., and Rebegoldi, S. (2022). A stochastic first-order trust-region method with inexact restoration for finite-sum minimization. Comput. Optim. Appl.","DOI":"10.1007\/s10589-022-00430-7"},{"key":"ref_12","first-page":"924","article-title":"Levenberg\u2013Marquardt Methods Based on Probabilistic Gradient Models and Inexact Subproblem Solution, with Application to Data Assimilation","volume":"4","author":"Bergou","year":"2016","journal-title":"SIAM-ASA J. Uncertain."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1287\/ijoo.2022.0072","article-title":"A subsampling line-search method with second-order results","volume":"4","author":"Bergou","year":"2022","journal-title":"INFORMS J. Optim."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3312","DOI":"10.1137\/17M1154679","article-title":"Adaptive sampling strategies for stochastic optimization","volume":"28","author":"Bollapragada","year":"2018","journal-title":"SIAM J. Optim."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1093\/imanum\/dry009","article-title":"Nocedal, Exact and Inexact Subsampled Newton Methods for Optimization","volume":"9","author":"Bollapragada","year":"2019","journal-title":"IMA J. Numer. Anal."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1007\/s10107-017-1141-8","article-title":"Stochastic optimization using a trust-region method and random models","volume":"169","author":"Chen","year":"2018","journal-title":"Math. Program."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"di Serafino, D., Krejic, N., Krklec Jerinkic, N., and Viola, M. (2021). LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums. arXiv.","DOI":"10.1090\/mcom\/3802"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1007\/s10589-016-9827-z","article-title":"Stochastic derivative-free optimization using a trust region framework","volume":"64","author":"Larson","year":"2016","journal-title":"Comput. Optim. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1137\/18M1216250","article-title":"A Stochastic Line Search Method with Expected Complexity Analysis","volume":"30","author":"Paquette","year":"2020","journal-title":"SIAM J. Optim."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1007\/s10107-019-01405-z","article-title":"Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information","volume":"184","author":"Xu","year":"2020","journal-title":"Math. Program."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1287\/ijoo.2019.0016","article-title":"Convergence Rate Analysis of a Stochastic Trust Region Method via Submartingales","volume":"1","author":"Blanchet","year":"2019","journal-title":"INFORMS J. Optim."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1007\/s10107-017-1137-4","article-title":"Global convergence rate analysis of unconstrained optimization methods based on probabilistic models","volume":"169","author":"Cartis","year":"2018","journal-title":"Math. Program."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"114193","DOI":"10.1016\/j.cam.2022.114193","article-title":"Inexact restoration for derivative-free expensive function minimization and applications","volume":"410","author":"Birgin","year":"2022","journal-title":"J. Comput. Appl. Math."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1137\/110856253","article-title":"Inexact Restoration Method for Derivative-Free Optimization with Smooth Constraints","volume":"23","author":"Bueno","year":"2013","journal-title":"SIAM J. Optim."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1023\/A:1004632923654","article-title":"Pilotta, Inexact restoration algorithms for constrained optimization","volume":"104","year":"2000","journal-title":"J. Optim. Theory Appl."},{"key":"ref_26","unstructured":"Lichman, M. (2022, September 01). UCI Machine Learning Repository. Available online: https:\/\/archive.ics.uci.edu\/ml\/index.php."},{"key":"ref_27","unstructured":"LeCun, Y., Bottou, L., and Bengio, Y. (2022, September 01). Haffner, The MNIST Database. Available online: http:\/\/yann.lecun.com\/exdb\/mnist."},{"key":"ref_28","unstructured":"Chang, C.C., and Lin, C.J. (2022, September 01). LIBSVM: A Library for Support Vector Machines. Available online: http:\/\/www.csie.ntu.edu.tw\/~cjlin\/libsvm."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2881","DOI":"10.1137\/18M1226282","article-title":"Adaptive regularization algorithms with inexact evaluations for nonconvex optimization","volume":"29","author":"Bellavia","year":"2019","journal-title":"SIAM J. Optim."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1080\/02331934.2021.1892104","article-title":"Stochastic analysis of an adaptive cubic regularization method under inexact gradient evaluations and dynamic Hessian accuracy","volume":"71","author":"Bellavia","year":"2022","journal-title":"Optimization"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xu, P., Roosta-Khorasani, F., and Mahoney, M.W. (2020, January 7\u20139). Second-Order Optimization for Non-Convex Machine Learning: An Empirical Study. Proceedings of the 2020 SIAM International Conference on Data Mining, Cincinnati, OH, USA.","DOI":"10.1137\/1.9781611976236.23"}],"container-title":["Axioms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-1680\/12\/1\/38\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:54:16Z","timestamp":1760147656000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-1680\/12\/1\/38"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,28]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["axioms12010038"],"URL":"https:\/\/doi.org\/10.3390\/axioms12010038","relation":{},"ISSN":["2075-1680"],"issn-type":[{"value":"2075-1680","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,28]]}}}