{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,6]],"date-time":"2026-04-06T19:20:36Z","timestamp":1775503236347,"version":"3.50.1"},"reference-count":27,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:00:00Z","timestamp":1750291200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Non-smooth, non-convex optimization problems frequently arise in modern machine learning applications, yet solving them efficiently remains a challenge. This paper addresses the minimization of functions of the form f(x)=\u2211i=1mfi(x) where each component is Lipschitz continuous but potentially non-smooth and non-convex. We extend the incremental subgradient method by incorporating weak subgradients, resulting in a framework better suited for non-convex objectives. We provide a comprehensive convergence analysis for three step size strategies: constant, diminishing, and a novel dynamic approach. Our theoretical results show that all variants converge to a neighborhood of the optimal solution, with the size of this neighborhood governed by the weak subgradient parameters. Numerical experiments on classification tasks with non-convex regularization, evaluated on the Breast Cancer Wisconsin dataset, demonstrate the effectiveness of the proposed approach. In particular, the dynamic step size method achieves superior practical performance, outperforming both classical and diminishing step size variants in terms of accuracy and convergence speed. These results position the incremental weak subgradient framework as a promising tool for scalable and efficient optimization in machine learning settings involving non-convex objectives.<\/jats:p>","DOI":"10.3390\/info16060509","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T06:11:07Z","timestamp":1750313467000},"page":"509","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Incremental Weak Subgradient Methods for Non-Smooth Non-Convex Optimization Problems"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3334-4032","authenticated-orcid":false,"given":"Narges","family":"Araboljadidi","sequence":"first","affiliation":[{"name":"Department of Mathematics and Physics, University of Campania \u201cLuigi Vanvitelli\u201d, Viale Abramo Lincoln, 5, 81100 Caserta, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3357-5252","authenticated-orcid":false,"given":"Valentina","family":"De Simone","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Physics, University of Campania \u201cLuigi Vanvitelli\u201d, Viale Abramo Lincoln, 5, 81100 Caserta, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1137\/S1052623499362111","article-title":"Incremental subgradient methods for nondifferentiable optimization","volume":"12","author":"Nedic","year":"2001","journal-title":"SIAM J. Optim."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hiriart-Urruty, J.-B., and Lemar\u00e9chal, C. (1993). Convex Analysis and Minimization Algorithms II, Springer.","DOI":"10.1007\/978-3-662-02796-7"},{"key":"ref_3","unstructured":"Bertsekas, D.P. (2015). Convex Optimization Algorithms, Athena Scientific."},{"key":"ref_4","unstructured":"Polyak, B.T. (1987). Introduction to Optimization, Optimization Software, Inc."},{"key":"ref_5","unstructured":"Liu, J., Wright, S.J., R\u00e9, C., Bittorf, V., and Sridhar, S. (2014, January 21\u201336). An asynchronous parallel stochastic coordinate descent algorithm. Proceedings of the ICML 2014: 31st International Conference on Machine Learning, Beijing, China."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"578","DOI":"10.3934\/mfc.2023032","article-title":"Smoothing Piecewise Linear Activation Functions Based on Mollified Square Root Functions","volume":"7","author":"Pan","year":"2023","journal-title":"Math. Found. Comput."},{"key":"ref_7","first-page":"171","article-title":"On weak conjugency, weak subdifferentials and dualty with zero gap in nonconvex optimiztion","volume":"1","author":"Azimov","year":"1999","journal-title":"Int. J. Appl. Math."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1513","DOI":"10.1080\/02331934.2020.1745205","article-title":"Weak subgradient method for solving nonsmooth nonconvex optimization problems","volume":"70","author":"Kasimbeyli","year":"2021","journal-title":"Optimization"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1137\/080738106","article-title":"On weak subdifferentials, directional derivatives, and radial epiderivatives for nonconvex functions","volume":"20","author":"Kasimbeyli","year":"2009","journal-title":"SIAM J. Optim."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1080\/10556789408805582","article-title":"Convergence analysis of parallel backpropagation algorithm for neural network","volume":"4","author":"Gaivoronski","year":"1994","journal-title":"Optim. Methods Softw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1080\/10556789408805583","article-title":"A class of unconstrained minimization methods for neural network training","volume":"4","author":"Grippo","year":"1994","journal-title":"Optim. Methods Softw."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1137\/S1052623495287022","article-title":"A new class of incremental gradient methods for least squares problems","volume":"7","author":"Bertsekas","year":"1997","journal-title":"SIAM J. Optim."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1023\/A:1018366000512","article-title":"Incremental gradient algorithms with stepsizes bounded away from zero","volume":"11","author":"Solodov","year":"1998","journal-title":"Comput. Optim. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1137\/S1052623495294797","article-title":"An Incremental gradient(-projection) method with momentum term and adaptive stepsize rule","volume":"8","author":"Tseng","year":"1998","journal-title":"SIAM J. Optim."},{"key":"ref_15","unstructured":"Shor, N.Z. (1979). Minimization Methods for Nondifferentiable Functions, Naukova Dumka."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1057\/palgrave.jors.2600425","article-title":"Nonlinear Programming","volume":"48","author":"Bertsekas","year":"1997","journal-title":"J. Oper. Res. Soc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1007\/s101070050053","article-title":"Convergence of a simple subgradient level method","volume":"85","author":"Goffin","year":"1999","journal-title":"Math. Program."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"7108","DOI":"10.1002\/mma.8958","article-title":"Incremental subgradient algorithms with dynamic step sizes for separable convex optimizations","volume":"46","author":"Yang","year":"2023","journal-title":"Math. Methods Appl. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/02331934.2017.1387548","article-title":"Subgradient algorithms on Riemannian manifolds of lower bounded curvatures","volume":"67","author":"Wang","year":"2018","journal-title":"Optimization"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s10107-007-0149-x","article-title":"Primal-dual subgradient methods for convex problems","volume":"120","author":"Nesterov","year":"2009","journal-title":"Math. Program."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yao, Y.H., Naseer, S., and Yao, J.C. (2020). Projected subgradient algorithms for pseudomonotone equilibrium problems and fixed points of pseudocontractive operators. Mathematics, 8.","DOI":"10.3390\/math8040461"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1023\/A:1020316811823","article-title":"Stability and duality of nonconvex problems via augmented Lagrangian","volume":"38","author":"Azimov","year":"2002","journal-title":"Cybern. Syst. Anal."},{"key":"ref_23","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations (ICLR), San Diego, CA, USA."},{"key":"ref_24","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_25","unstructured":"Street, W.N., Wolberg, W.H., and Mangasarian, O.L. (February, January 31). Nuclear feature extraction for breast tumor diagnosis. Proceedings of the IS&T\/SPIE\u2019s Symposium on Electronic Imaging: Science and Technology, San Jose, CA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1007\/s10479-023-05524-x","article-title":"Nonconvex multi-period mean- variance portfolio optimization","volume":"332","author":"Wu","year":"2024","journal-title":"Ann. Oper. Res."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/509\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:54:40Z","timestamp":1760032480000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/6\/509"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":27,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["info16060509"],"URL":"https:\/\/doi.org\/10.3390\/info16060509","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}