{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T07:41:01Z","timestamp":1765438861056},"reference-count":15,"publisher":"MIT Press","issue":"9","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In this letter, we use composite optimization algorithms to solve sigmoid networks. We equivalently transfer the sigmoid networks to a convex composite optimization and propose the composite optimization algorithms based on the linearized proximal algorithms and the alternating direction method of multipliers. Under the assumptions of the weak sharp minima and the regularity condition, the algorithm is guaranteed to converge to a globally optimal solution of the objective function even in the case of nonconvex and nonsmooth problems. Furthermore, the convergence results can be directly related to the amount of training data and provide a general guide for setting the size of sigmoid networks. Numerical experiments on Franke\u2019s function fitting and handwritten digit recognition show that the proposed algorithms perform satisfactorily and robustly.<\/jats:p>","DOI":"10.1162\/neco_a_01603","type":"journal-article","created":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T19:41:30Z","timestamp":1689190890000},"page":"1543-1565","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":1,"title":["Composite Optimization Algorithms for Sigmoid Networks"],"prefix":"10.1162","volume":"35","author":[{"given":"Huixiong","family":"Chen","sequence":"first","affiliation":[{"name":"School of Mathematical Sciences, South China Normal University, Guangzhou 510631, China hxchen@m.scnu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qi","family":"Ye","sequence":"additional","affiliation":[{"name":"School of Mathematical Sciences, South China Normal University, Guangzhou 510631, China yeqi@m.scnu.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,8,7]]},"reference":[{"issue":"11","key":"2023080715480559800_bib1","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2018.e00938","article-title":"State-of-the-art in artificial neural network applications: A survey","volume":"4","author":"Abiodun","year":"2018","journal-title":"Heliyon"},{"key":"2023080715480559800_bib2","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511624216","volume-title":"Neural network learning: Theoretical foundations","author":"Anthony","year":"1999"},{"key":"2023080715480559800_bib3","volume-title":"Convex optimization algorithms","author":"Bertsekas","year":"2015"},{"issue":"1","key":"2023080715480559800_bib4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000016","article-title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","volume":"3","author":"Boyd","year":"2011","journal-title":"Foundations and Trends in Machine Learning"},{"issue":"5","key":"2023080715480559800_bib5","doi-asserted-by":"publisher","first-page":"1340","DOI":"10.1137\/0331063","article-title":"Weak sharp minima in mathematical programming","volume":"31","author":"Burke","year":"1993","journal-title":"SIAM Journal on Control and Optimization"},{"issue":"2","key":"2023080715480559800_bib6","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1007\/BF01585997","article-title":"A Gauss-Newton method for convex composite optimization","volume":"71","author":"Burke","year":"1995","journal-title":"Mathematical Programming"},{"issue":"3","key":"2023080715480559800_bib7","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1137\/030601296","article-title":"A robust gradient sampling algorithm for nonsmooth, nonconvex optimization","volume":"15","author":"Burke","year":"2005","journal-title":"SIAM Journal on Control and Optimization"},{"key":"2023080715480559800_bib8","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1007\/978-3-319-66709-6_22","article-title":"Adaptive regularization in convex composite optimization for variational imaging problems","author":"Hong","year":"2017","journal-title":"Proceedings of Pattern Recognition: 39th German Conference on Pattern Recognition"},{"key":"2023080715480559800_bib9","doi-asserted-by":"publisher","first-page":"1925","DOI":"10.1016\/j.csbj.2020.06.033","article-title":"Integration of single-cell multi-omics for gene regulatory network inference","volume":"18","author":"Hu","year":"2020","journal-title":"Computational and Structural Biotechnology Journal"},{"issue":"2","key":"2023080715480559800_bib10","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1137\/140993090","article-title":"On convergence rates of linearized proximal algorithms for convex composite optimization with applications","volume":"26","author":"Hu","year":"2016","journal-title":"SIAM Journal on Control and Optimization"},{"issue":"7553","key":"2023080715480559800_bib11","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"1","key":"2023080715480559800_bib12","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1007\/s10107-015-0943-9","article-title":"A proximal method for composite minimization","volume":"158","author":"Lewis","year":"2016","journal-title":"Mathematical Programming"},{"key":"2023080715480559800_bib13","article-title":"On the convergence of Adam and beyond","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Reddi","year":"2018"},{"issue":"5","key":"2023080715480559800_bib14","doi-asserted-by":"crossref","DOI":"10.1088\/1361-6420\/aa6699","article-title":"Linear convergence of CQ algorithms and applications in gene regulatory network inference","volume":"33","author":"Wang","year":"2017","journal-title":"Inverse Problems"},{"key":"2023080715480559800_bib15","article-title":"The marginal value of adaptive gradient methods in machine learning","volume-title":"Advances in neural information processing systems","author":"Wilson","year":"2017"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/9\/1543\/2152762\/neco_a_01603.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/9\/1543\/2152762\/neco_a_01603.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T15:48:41Z","timestamp":1691423321000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/9\/1543\/116702\/Composite-Optimization-Algorithms-for-Sigmoid"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,7]]},"references-count":15,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,8,7]]},"published-print":{"date-parts":[[2023,8,7]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01603","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,9]]},"published":{"date-parts":[[2023,8,7]]}}}