{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T16:10:34Z","timestamp":1791821434021,"version":"4.3.4"},"reference-count":53,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM J. Sci. Comput."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>In this paper we analyze a zeroth-order proximal stochastic gradient method suitable for the minimization of weakly convex stochastic optimization problems. We consider nonsmooth and nonlinear stochastic composite problems, for which (sub)gradient information might be unavailable. The proposed algorithm utilizes the well-known Gaussian smoothing technique, which yields unbiased zeroth-order gradient estimators of a related partially smooth surrogate problem (in which one of the two nonsmooth terms in the original problem\u2019s objective is replaced by a smooth approximation). This allows us to employ a standard proximal stochastic gradient scheme for the approximate solution of the surrogate problem, which is determined by a single smoothing parameter, and without the utilization of first-order information. We provide state-of-the-art convergence rates for the proposed zeroth-order method using minimal assumptions. The proposed scheme is numerically compared against alternative zeroth-order methods as well as a stochastic subgradient scheme on a standard phase retrieval problem. Further, we showcase the usefulness and effectiveness of our method in the unique setting of automated hyperparameter tuning. In particular, we focus on automatically tuning the parameters of optimization algorithms by minimizing a novel heuristic model. The proposed approach is tested on a proximal alternating direction method of multipliers for the solution of [Formula: see text]-regularized PDE-constrained optimal control problems, with evident empirical success.<\/jats:p>\n                  <jats:p>Reproducibility of computational results. This paper has been awarded the \u201cSIAM Reproducibility Badge: code and data available\u201d, as a recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https:\/\/github.com\/spougkakiotis\/Z-ProxSG . [Formula: see text]<\/jats:p>","DOI":"10.1137\/22m1494270","type":"journal-article","created":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T10:54:05Z","timestamp":1697453645000},"page":"A2679-A2702","source":"Crossref","is-referenced-by-count":9,"title":["A Zeroth-Order Proximal Stochastic Gradient Method for Weakly Convex Stochastic Optimization"],"prefix":"10.1137","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7903-9335","authenticated-orcid":true,"given":"Spyridon","family":"Pougkakiotis","sequence":"first","affiliation":[{"name":"School of Science and Engineering, University of Dundee, Dundee, Scotland DD1 4HR, UK."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3459-5044","authenticated-orcid":true,"given":"Dionysis","family":"Kalogerias","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Yale University, New Haven, CT 06511 USA ()."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2023,10,16]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623400377955"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1137\/040620886"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1007\/BF00935752"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-021-09499-8"},{"key":"ref5","volume-title":"Advances in Neural Information Processing Systems 31","author":"Balasubramanian K.","year":"2018"},{"key":"ref6","volume-title":"Advances in Neural Information Processing Systems 24","author":"Bergstra J.","year":"2011"},{"key":"ref7","first-page":"281","volume":"13","author":"Bergstra J.","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1780-0_3"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1561\/2200000016"},{"key":"ref10","first-page":"20","volume":"14","author":"Calvetti D.","year":"2002","journal-title":"Electron. Trans. Numer. Anal."},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-1975-0367131-6"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718768"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1137\/18M1178244"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-015-0048-x"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-018-1311-3"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2015.2409256"},{"key":"ref17","unstructured":"D. Dvinskikh , \nV. Tominin , \nY. Tominin , and \nA. Gasnikov  , Gradient-Free Optimization for Non-Smooth Minimax Problems with Maximum Value of Adversarial Noise, https:\/\/arxiv.org\/abs\/arXiv:2202.06114v1, 2022."},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1016\/j.acha.2013.08.003"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1145\/1236463.1236469"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1137\/120891393"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1007\/s10543-017-0662-0"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5_1"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1137\/120880811"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1137\/0614023"},{"key":"ref25","unstructured":"J. Rapin  and \nO. Teytaud , Nevergrad - A Gradient-Free Optimization Platform, 2018, https:\/\/GitHub.com\/FacebookResearch\/Nevergrad."},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.3934\/jimo.2018181"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1137\/20M1315403"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-022-01866-9"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"H. Kumar , \nD. S. Kalogerias , \nG. J. Pappas , and \nA. Ribeiro  , Actor-only deterministic policy gradient via zeroth-order gradient oracles in action space, in Proceedings of the 2021 IEEE International Symposium on Information Theory (ISIT), 2021, pp. 1676\u20131681, https:\/\/doi.org\/10.1109\/ISIT45174.2021.9518023.","DOI":"10.1109\/ISIT45174.2021.9518023"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-021-00313-3"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1088\/0266-5611\/31\/11\/115002"},{"key":"ref32","volume-title":"Advances in Neural Information Processing Systems","volume":"21","author":"Mairal J.","year":"2008"},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"C. Malivert  , M\u00e9thode de descente sur un ferm\u00e9 non convexe, in Analyse non convexe (Pau, 1977), Bull. Soc. Math. France M\u00e9m. (1979), no. 60, pp. 113\u2013124, https:\/\/doi.org\/10.24033\/msmf.264.","DOI":"10.24033\/msmf.264"},{"key":"ref34","first-page":"246","volume":"26","author":"Matyas J.","year":"1965","journal-title":"Automat. Remote Control"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1002\/jcc.540150606"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.24033\/bsmf.1625"},{"key":"ref37","unstructured":"P. Nazari , \nD. A. Tarzanagh , and \nG. Michailidis  , Adaptive First- and Zeroth-Order Methods for Weakly Convex Stochastic Optimization Problems, https:\/\/arxiv.org\/abs\/2005.09261v2, 2020."},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1007\/s10208-015-9296-2"},{"key":"ref39","doi-asserted-by":"crossref","unstructured":"P. Patrinos  and \nA. Bemporad  , Proximal Newton methods for convex composite optimization, in Proceedings of the 52nd IEEE Conference on Decision and Control, 2013, pp. 2358\u20132363, https:\/\/doi.org\/10.1109\/CDC.2013.6760233.","DOI":"10.1109\/CDC.2013.6760233"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1002\/nla.2276"},{"key":"ref41","unstructured":"S. Pougkakiotis , \nJ. Gondzio , and \nD. S. Kalogerias  , An Active-Set Method for Sparse Approximations, Part I: Separable \\(\\ell_1\\) Terms, 2023, https:\/\/arxiv.org\/abs\/2201.10211v2."},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-75549-2_38"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.21314\/JOR.2000.038"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02431-3"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611973433"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1287\/moor.6.1.19"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-017-9951-4"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1109\/7.705889"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1109\/9.119632"},{"key":"ref50","doi-asserted-by":"crossref","unstructured":"A. Teixeira , \nE. Ghadimi , \nI. Shames , \nH. Sandberg , and \nM. Johansson  , Optimal scaling of the ADMM algorithm for distributed quadratic programming, in Proceedings of the 52nd IEEE Conference on Decision and Control, 2013, pp. 6868\u20136873, https:\/\/doi.org\/10.1109\/CDC.2013.6760977.","DOI":"10.1109\/CDC.2013.6760977"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1090\/gsm\/112"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1287\/moor.8.2.231"},{"key":"ref53","doi-asserted-by":"publisher","DOI":"10.1051\/cocv\/2010027"}],"container-title":["SIAM Journal on Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/22M1494270","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T15:46:48Z","timestamp":1787327208000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/22M1494270"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,16]]},"references-count":53,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1137\/22M1494270"],"URL":"https:\/\/doi.org\/10.1137\/22m1494270","relation":{},"ISSN":["1064-8275","1095-7197"],"issn-type":[{"value":"1064-8275","type":"print"},{"value":"1095-7197","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,16]]}}}