{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T13:28:07Z","timestamp":1787232487672,"version":"build-2736575974"},"reference-count":248,"publisher":"Society for Industrial & Applied Mathematics (SIAM)","issue":"3","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12401419"],"award-info":[{"award-number":["12401419"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12401414"],"award-info":[{"award-number":["12401414"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Singapore Ministry of Education Academic Research Fund","award":["MOE-T2EP20125-0007"],"award-info":[{"award-number":["MOE-T2EP20125-0007"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["123310103"],"award-info":[{"award-number":["123310103"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["12288101"],"award-info":[{"award-number":["12288101"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Key Research and Development Program of China","award":["2024YFA1012903"],"award-info":[{"award-number":["2024YFA1012903"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["SIAM Rev."],"published-print":{"date-parts":[[2026,9,30]]},"abstract":"<jats:p>Abstract.<\/jats:p>\n                  <jats:p>Large-scale constrained optimization is pivotal in modern scientific, engineering, and industrial computation, often involving complex systems with numerous variables and constraints. This paper provides a unified and comprehensive perspective on constructing augmented Lagrangian functions (based on the Hestenes\u2013Powell\u2013Rockafellar augmented Lagrangian) for various optimization problems, including nonlinear programming and convex and nonconvex composite programming. We present the augmented Lagrangian method (ALM), covering its theoretical foundations in both convex and nonconvex cases, and discuss several successful examples and applications. Recent advancements have extended the ALM\u2019s capabilities to handle nonconvex constraints and ensure global convergence to first- and second-order stationary points. For nonsmooth convex problems, the ALM utilizes proximal operations, preserving desirable properties such as locally linear convergence rates. Furthermore, recent progress has refined the complexity analysis of the ALM and tackled challenging integer programming instances. This review aims to offer a thorough understanding of the ALM\u2019s benefits and limitations, exploring different ALM variants designed to enhance convergence and computational performance. We also illustrate effective algorithms for ALM subproblems across different types of optimization problems and highlight practical implementations in several fields.<\/jats:p>","DOI":"10.1137\/24m1706153","type":"journal-article","created":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T07:01:19Z","timestamp":1785999679000},"page":"471-576","source":"Crossref","is-referenced-by-count":0,"title":["The Augmented Lagrangian Methods: Overview and Recent Advances"],"prefix":"10.1137","volume":"68","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4609-747X","authenticated-orcid":true,"given":"Kangkang","family":"Deng","sequence":"first","affiliation":[{"name":"School of Science, National University of Defense Technology, Changsha, 410073 China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mathematics, Southwestern University of Finance and Economics, Chengdu, 611130 China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhenyuan","family":"Zhu","sequence":"additional","affiliation":[{"name":"Beijing International Center for Mathematical Research, Peking University, Beijing, 100871 China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2194-9664","authenticated-orcid":true,"given":"Junyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Industrial Systems Engineering and Management, National University of Singapore, Singapore."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1762-0671","authenticated-orcid":true,"given":"Zaiwen","family":"Wen","sequence":"additional","affiliation":[{"name":"Corresponding author. Beijing International Center for Mathematical Research, Center for Machine Learning Research and Changsha Institute for Computing and Digital Economy, Peking University, Beijing, 100871 China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"351","published-online":{"date-parts":[[2026,8,6]]},"reference":[{"key":"ref1","volume-title":"Optimization Algorithms on Matrix Manifolds","author":"Absil P.-A.","year":"2009"},{"key":"ref2","unstructured":"A. Alacaoglu and S. J. Wright, Complexity of single loop algorithms for nonlinear programming with stochastic objective and constraints, in International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 4627\u20134635."},{"key":"ref3","first-page":"176","volume-title":"Studies in Linear Nonlinear Programming","author":"Arrow K.","year":"1958"},{"key":"ref4","unstructured":"M. Asteris, D. Papailiopoulos, and A. Dimakis, Nonnegative sparse PCA with provable guarantees, in International Conference on Machine Learning, PMLR, 2014, pp. 1728\u20131736."},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1137\/100786721"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2018.1496431"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"S. Basir and I. Senocak, An Adaptive Augmented Lagrangian Method for Training Physics and Equality Constrained Artificial Neural Networks, preprint, arXiv:2306.04904, 2023.","DOI":"10.2139\/ssrn.4484779"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9467-7"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"A. Beck, First-Order Methods in Optimization, SIAM, 2017, https:\/\/doi.org\/10.1137\/1.9781611974997.","DOI":"10.1137\/1.9781611974997"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2009.2028250"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623493259215"},{"key":"ref12","volume-title":"Convex Optimization Algorithms","author":"Bertsekas D.","year":"2015"},{"key":"ref13","doi-asserted-by":"crossref","unstructured":"D. P. Bertsekas, Control of Uncertain Systems with a Set-Membership Description of the Uncertainty, Ph.D. thesis, Massachusetts Institute of Technology, 1971.","DOI":"10.1109\/TAC.1971.1099674"},{"key":"ref14","volume-title":"Dynamic Programming and Optimal Control","volume":"1","author":"Bertsekas D. P.","year":"1995"},{"key":"ref15","volume-title":"Athena Scientific","author":"Bertsekas D. P.","year":"1999"},{"key":"ref16","volume-title":"Constrained Optimization and Lagrange Multiplier Methods","author":"Bertsekas D. P.","year":"2014"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1137\/22M1526204"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-005-1066-7"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-009-0264-y"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1007\/s10957-025-02734-0"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-017-9937-2"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-007-9050-z"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611973365"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1080\/10556788.2020.1746962"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-013-0039-0"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-014-9685-5"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1016\/0012-365X(77)90028-0"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-014-0763-3"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-013-0701-9"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2017.0900"},{"key":"ref31","volume-title":"Perturbation Analysis of Optimization Problems","author":"Bonnans J. F.","year":"2013"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-022-01879-4"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1017\/9781009166164"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1002\/cpa.21830"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"L. E. Bourkhissi and I. Necoara, Complexity of Linearized Augmented Lagrangian for Optimization with Nonlinear Equality Constraints, preprint, arXiv:2301.08345, 2023.","DOI":"10.1109\/CDC49753.2023.10384166"},{"key":"ref36","unstructured":"L. E. Bourkhissi, I. Necoara, P. Patrinos, and Q. Tran-Dinh, Complexity of Linearized Perturbed Augmented Lagrangian Methods for Nonsmooth Nonconvex Optimization with Nonlinear Equality Constraints, preprint, arXiv:2503.01056, 2025."},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511804441"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1561\/2200000016"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(67)90040-7"},{"key":"ref40","unstructured":"G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, OpenAi Gym, preprint, arXiv:1606.01540, 2016."},{"key":"ref41","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-010-0010-8"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-002-0352-8"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-004-0564-1"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1137\/040609574"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-016-9860-y"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.1287\/opre.50.5.851.362"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1287\/opre.9.2.169"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-010-0251-1"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-015-0957-3"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.1137\/110860185"},{"key":"ref51","doi-asserted-by":"crossref","unstructured":"R. Chartrand and B. Wohlberg, A nonconvex ADMM algorithm for group sparsity with sparse groups, in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, 2013, pp. 6009\u20136013.","DOI":"10.1109\/ICASSP.2013.6638818"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-014-0826-5"},{"key":"ref53","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-017-1205-9"},{"key":"ref54","first-page":"183961","volume-title":"Abstract and Applied Analysis","volume":"2013","author":"Chen C.","year":"2013"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1186\/s13663-016-0543-2"},{"key":"ref56","doi-asserted-by":"publisher","DOI":"10.1137\/18M122457X"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1137\/15M1052834"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-1975-0367131-6"},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9569-8_10"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1007\/s10957-012-0245-9"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.1137\/0728030"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1007\/BF00940582"},{"key":"ref63","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-021-01703-5"},{"key":"ref64","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022627411411"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9904-1943-07818-4"},{"key":"ref66","doi-asserted-by":"publisher","DOI":"10.1007\/s40305-021-00346-9"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-018-1300-6"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-014-0784-y"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.1007\/s40305-022-00439-z"},{"key":"ref70","doi-asserted-by":"publisher","DOI":"10.1137\/0114053"},{"key":"ref71","doi-asserted-by":"crossref","unstructured":"A. De Marchi, Constrained and sparse switching times optimization via augmented Lagrangian proximal methods, in 2020 American Control Conference (ACC), IEEE, 2020, pp. 3633\u20133638.","DOI":"10.23919\/ACC45564.2020.9147892"},{"key":"ref72","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-022-01922-4"},{"key":"ref73","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(68)90089-X"},{"key":"ref74","doi-asserted-by":"publisher","DOI":"10.1007\/BF01074499"},{"key":"ref75","unstructured":"A. Dener, M. A. Miller, R. M. Churchill, T. Munson, and C.S. Chang, Training Neural Networks under Physical Constraints Using a Stochastic Augmented Lagrangian Approach, preprint, arXiv:2009.07330, 2020."},{"key":"ref76","doi-asserted-by":"crossref","unstructured":"K. Deng, J. Hu, J. Wu, and Z. Wen, Oracle Complexities of Augmented Lagrangian Methods for Nonsmooth Manifold Optimization, preprint, arXiv:2404.05121, 2024.","DOI":"10.1287\/moor.2024.0498"},{"key":"ref77","doi-asserted-by":"publisher","DOI":"10.1093\/imanum\/drac018"},{"key":"ref78","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-016-0318-2"},{"key":"ref79","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-025-02794-4"},{"key":"ref80","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-013-0677-5"},{"key":"ref81","unstructured":"P. Dhariwal, C. Hesse, O. Klimov, A. Nichol, M. Plappert, A. Radford, J. Schulman, S. Sidor, Y. Wu, and P. Zhokhov, OpenAI Baselines, 2017, https:\/\/github.com\/openai\/baselines."},{"key":"ref82","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2018.2867589"},{"key":"ref83","doi-asserted-by":"publisher","DOI":"10.1137\/0317044"},{"key":"ref84","doi-asserted-by":"publisher","DOI":"10.1007\/BF00940544"},{"key":"ref85","doi-asserted-by":"publisher","DOI":"10.1137\/S1052623497321894"},{"key":"ref86","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-023-00493-0"},{"key":"ref87","doi-asserted-by":"publisher","DOI":"10.1007\/s101070100263"},{"key":"ref88","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-017-0603-0"},{"key":"ref89","doi-asserted-by":"publisher","DOI":"10.1007\/s10957-018-1239-z"},{"key":"ref90","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2006.871582"},{"key":"ref91","doi-asserted-by":"publisher","DOI":"10.1007\/BF01581204"},{"key":"ref92","first-page":"619","volume":"11","author":"Eckstein J.","year":"2015","journal-title":"Pacific J. Optim."},{"key":"ref93","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-016-1012-8"},{"key":"ref94","doi-asserted-by":"publisher","DOI":"10.1137\/10081085X"},{"key":"ref95","doi-asserted-by":"publisher","DOI":"10.1093\/imamat\/15.3.319"},{"key":"ref96","series-title":"Stud. Math. Appl. 15","volume-title":"Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems","author":"Fortin M.","year":"2000"},{"key":"ref97","doi-asserted-by":"publisher","DOI":"10.52202\/079017-0286"},{"key":"ref98","unstructured":"K. Frisch, The logarithmic potential method of convex programming, Memorandum, University Institute of Economics, Oslo, 5, 1955."},{"key":"ref99","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-018-0017-z"},{"key":"ref100","doi-asserted-by":"publisher","DOI":"10.1016\/0898-1221(76)90003-1"},{"key":"ref101","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2018.1473777"},{"key":"ref102","doi-asserted-by":"publisher","DOI":"10.1007\/s40305-018-0232-4"},{"key":"ref103","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020261001771"},{"key":"ref104","doi-asserted-by":"publisher","DOI":"10.1007\/s10957-024-02488-1"},{"key":"ref105","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-017-9054-3_4"},{"key":"ref106","first-page":"41","volume":"9","author":"Glowinski R.","year":"1975","journal-title":"Rev. Fran\u00e7aise Automat. Informat. Recherche Op\u00e9rationnelle S\u00e9r. Rouge Anal. Num\u00e9r."},{"key":"ref107","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008705028512"},{"key":"ref108","doi-asserted-by":"publisher","DOI":"10.1137\/19M1271695"},{"key":"ref109","doi-asserted-by":"publisher","DOI":"10.1137\/0802032"},{"key":"ref110","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-019-01365-4"},{"key":"ref111","doi-asserted-by":"publisher","DOI":"10.1137\/20M1375188"},{"key":"ref112","volume":"3","author":"He B.","year":"2010","journal-title":"Optim. Online"},{"key":"ref113","doi-asserted-by":"publisher","DOI":"10.1137\/22M1489824"},{"key":"ref114","doi-asserted-by":"crossref","unstructured":"P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, Deep reinforcement learning that matters, in Thirty-Second AAAI Conference on Artificial Intelligence, 2018, pp. 3207\u20133214.","DOI":"10.1609\/aaai.v32i1.11694"},{"key":"ref115","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-022-00218-0"},{"key":"ref116","doi-asserted-by":"publisher","DOI":"10.1007\/BF00927673"},{"key":"ref117","unstructured":"M. Hong, D. Hajinezhad, and M.M. Zhao, Prox-PDA: The proximal primal-dual algorithm for fast distributed nonconvex optimization and learning over networks, in International Conference on Machine Learning, PMLR, 2017, pp. 1529\u20131538."},{"key":"ref118","doi-asserted-by":"publisher","DOI":"10.1137\/140990309"},{"key":"ref119","doi-asserted-by":"publisher","DOI":"10.1137\/140975991"},{"key":"ref120","doi-asserted-by":"publisher","DOI":"10.1137\/17M1142478"},{"key":"ref121","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-025-02867-4"},{"key":"ref122","doi-asserted-by":"publisher","DOI":"10.1287\/moor.28.3.533.16395"},{"key":"ref123","doi-asserted-by":"publisher","DOI":"10.1007\/BF01585750"},{"key":"ref124","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718614"},{"key":"ref125","doi-asserted-by":"publisher","DOI":"10.1109\/TSIPN.2018.2846183"},{"key":"ref126","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.3007395"},{"key":"ref127","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-022-01870-z"},{"key":"ref128","doi-asserted-by":"publisher","DOI":"10.4208\/cicp.OA-2021-0176"},{"key":"ref129","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-022-00384-w"},{"key":"ref130","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-013-9686-z"},{"key":"ref131","doi-asserted-by":"crossref","unstructured":"B. Kannan and S. N. Kramer, An augmented Lagrange multiplier based method for mixed integer discrete continuous optimization and its applications to mechanical design, in International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Vol. 97690, American Society of Mechanical Engineers, 1993, pp. 103\u2013112.","DOI":"10.1115\/DETC1993-0382"},{"key":"ref132","doi-asserted-by":"crossref","unstructured":"H. Karimi, J. Nutini, and M. Schmidt, Linear convergence of gradient and proximal-gradient methods under the Polyak-\u0141ojasiewicz condition, in Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Springer, Italy, 2016, pp. 795\u2013811.","DOI":"10.1007\/978-3-319-46128-1_50"},{"key":"ref133","doi-asserted-by":"publisher","DOI":"10.1007\/s11766-017-3381-z"},{"key":"ref134","unstructured":"J. G. Kim, A. Chandra, A. Hashemi, and C. Brinton, A Fast Single-Loop Primal-Dual Algorithm for Non-Convex Functional Constrained Optimization, preprint, arXiv:2406.17107, 2024."},{"key":"ref135","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2022.1301"},{"key":"ref136","doi-asserted-by":"publisher","DOI":"10.1137\/20M136147X"},{"key":"ref137","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-023-00464-5"},{"key":"ref138","doi-asserted-by":"publisher","DOI":"10.1137\/0314020"},{"key":"ref139","unstructured":"J. Kotary and F. Fioretto, Learning Constrained Optimization with Deep Augmented Lagrangian Methods, preprint, arXiv:2403.03454, 2024."},{"key":"ref140","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023673105317"},{"key":"ref141","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-015-0861-x"},{"key":"ref142","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-021-00205-x"},{"key":"ref143","doi-asserted-by":"publisher","DOI":"10.1137\/140998135"},{"key":"ref144","doi-asserted-by":"publisher","DOI":"10.1142\/S0217595915500244"},{"key":"ref145","doi-asserted-by":"publisher","DOI":"10.1137\/20M1321000"},{"key":"ref146","doi-asserted-by":"publisher","DOI":"10.1137\/16M1097572"},{"key":"ref147","doi-asserted-by":"publisher","DOI":"10.1137\/17M1136390"},{"key":"ref148","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-018-0137-6"},{"key":"ref149","doi-asserted-by":"publisher","DOI":"10.1137\/18M1188069"},{"key":"ref150","doi-asserted-by":"publisher","DOI":"10.1137\/21M1421726"},{"key":"ref151","unstructured":"Z. Li, P.Y. Chen, S. Liu, S. Lu, and Y. Xu, Rate-improved inexact augmented Lagrangian method for constrained nonconvex optimization, in International Conference on Artificial Intelligence and Statistics, PMLR, 2021, pp. 2170\u20132178."},{"key":"ref152","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-023-00521-z"},{"key":"ref153","doi-asserted-by":"publisher","DOI":"10.1287\/ijoo.2021.0052"},{"key":"ref154","unstructured":"F.Y. Liao and Y. Zheng, A Bundle-Based Augmented Lagrangian Framework: Algorithm, Convergence, and Primal-Dual Principles, preprint, arXiv:2502.08835, 2025."},{"key":"ref155","doi-asserted-by":"publisher","DOI":"10.1137\/18M1207752"},{"key":"ref156","doi-asserted-by":"publisher","DOI":"10.1137\/140971178"},{"key":"ref157","doi-asserted-by":"publisher","DOI":"10.1007\/s40305-015-0092-0"},{"key":"ref158","doi-asserted-by":"publisher","DOI":"10.1007\/s00245-019-09564-3"},{"key":"ref159","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2018.0939"},{"key":"ref160","unstructured":"S. Lu, A single-loop gradient descent and perturbed ascent algorithm for nonconvex functional constrained optimization, in International Conference on Machine Learning, PMLR, 2022, pp. 14315\u201314357."},{"key":"ref161","doi-asserted-by":"publisher","DOI":"10.52202\/068431-2221"},{"key":"ref162","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-024-02163-3"},{"key":"ref163","doi-asserted-by":"publisher","DOI":"10.1007\/BF00939685"},{"key":"ref164","first-page":"1","volume":"20","author":"Luo Z.","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref165","doi-asserted-by":"publisher","DOI":"10.1137\/0322019"},{"key":"ref166","doi-asserted-by":"publisher","DOI":"10.1016\/0022-247X(67)90163-1"},{"key":"ref167","doi-asserted-by":"publisher","DOI":"10.1007\/s10957-023-02218-z"},{"key":"ref168","doi-asserted-by":"publisher","DOI":"10.1137\/120892167"},{"key":"ref169","doi-asserted-by":"publisher","DOI":"10.1137\/18M1181249"},{"key":"ref170","doi-asserted-by":"publisher","DOI":"10.1137\/120897547"},{"key":"ref171","doi-asserted-by":"publisher","DOI":"10.1137\/16M1084316"},{"key":"ref172","doi-asserted-by":"crossref","unstructured":"A. Nedi\u0107, A. Olshevsky, W. Shi, and C. A. Uribe, Geometrically convergent distributed optimization with uncoordinated step-sizes, in 2017 American Control Conference (ACC), IEEE, 2017, pp. 3950\u20133955.","DOI":"10.23919\/ACC.2017.7963560"},{"key":"ref173","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2008.2009515"},{"key":"ref174","series-title":"Appl. Optim. 87","volume-title":"Introductory Lectures on Convex Optimization: A Basic Course","author":"Nesterov Y.","year":"2013"},{"key":"ref175","series-title":"Springer Ser. Oper. Res. Financ. Engrg.","volume-title":"Numerical Optimization","author":"Nocedal J.","year":"2006","edition":"2"},{"key":"ref176","doi-asserted-by":"publisher","DOI":"10.1287\/moor.28.1.39.14258"},{"key":"ref177","volume-title":"Combinatorial Optimization: Algorithms and Complexity","author":"Papadimitriou C. H.","year":"2013"},{"key":"ref178","doi-asserted-by":"publisher","DOI":"10.1007\/s11590-016-1024-6"},{"key":"ref179","doi-asserted-by":"publisher","DOI":"10.1007\/BF01586050"},{"key":"ref180","first-page":"283","volume-title":"Optimization","author":"Powell M. J.","year":"1969"},{"key":"ref181","volume-title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","author":"Puterman M. L.","year":"2014"},{"key":"ref182","doi-asserted-by":"publisher","DOI":"10.1007\/BF01581275"},{"key":"ref183","doi-asserted-by":"publisher","DOI":"10.1109\/TCNS.2017.2698261"},{"key":"ref184","doi-asserted-by":"publisher","DOI":"10.1007\/BF01498415"},{"key":"ref185","unstructured":"R. T. Rockafellar, New applications of duality in nonlinear programming, in Proceedings of the Fourth Conference on Probability Theory (Bra\u015fov, Romania), Editura Academiei R.S.R., 1971, pp. 73\u201381."},{"key":"ref186","doi-asserted-by":"publisher","DOI":"10.1007\/BF01580138"},{"key":"ref187","doi-asserted-by":"publisher","DOI":"10.1007\/BF00934777"},{"key":"ref188","doi-asserted-by":"publisher","DOI":"10.1137\/0312021"},{"key":"ref189","doi-asserted-by":"publisher","DOI":"10.1287\/moor.1.2.97"},{"key":"ref190","doi-asserted-by":"publisher","DOI":"10.1137\/0314056"},{"key":"ref191","doi-asserted-by":"publisher","DOI":"10.1137\/1035044"},{"key":"ref192","volume-title":"Convex Analysis","author":"Rockafellar R. T.","year":"1997"},{"key":"ref193","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-022-01832-5"},{"key":"ref194","volume-title":"Variational Analysis","volume":"317","author":"Rockafellar R. T.","year":"2009"},{"key":"ref195","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-007-0115-7"},{"key":"ref196","doi-asserted-by":"publisher","DOI":"10.1137\/20M1375358"},{"key":"ref197","first-page":"13966","volume":"32","author":"Sahin M. F.","year":"2019","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref198","doi-asserted-by":"publisher","DOI":"10.1287\/moor.1040.0103"},{"key":"ref199","unstructured":"Q. Shi, X. Wang, and H. Wang, Bregman Linearized Augmented Lagrangian Method for Nonconvex Constrained Stochastic Zeroth-Order Optimization, preprint, arXiv:2504.09409, 2025."},{"key":"ref200","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2022.0193"},{"key":"ref201","doi-asserted-by":"publisher","DOI":"10.1137\/14096668X"},{"key":"ref202","unstructured":"N. K. Singh and I. Necoara, A Stochastic Perturbed Augmented Lagrangian Method for Smooth Convex Constrained Minimization, preprint, arXiv:2503.23572, 2025."},{"key":"ref203","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126424"},{"key":"ref204","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-023-02350-y"},{"key":"ref205","volume-title":"Optimization Theory and Methods: Nonlinear Programming","volume":"1","author":"Sun W.","year":"2006"},{"key":"ref206","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton R. S.","year":"2018"},{"key":"ref207","doi-asserted-by":"crossref","unstructured":"E. Todorov, T. Erez, and Y. Tassa, MuJoCo: A physics engine for model-based control, in 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2012, pp. 5026\u20135033.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref208","doi-asserted-by":"publisher","DOI":"10.1002\/nme.2158"},{"key":"ref209","doi-asserted-by":"crossref","unstructured":"I. Tsaknakis, P. Khanduri, and M. Hong, An implicit gradient-type method for linearly constrained bilevel problems, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 5438\u20135442.","DOI":"10.1109\/ICASSP43922.2022.9747013"},{"key":"ref210","doi-asserted-by":"publisher","DOI":"10.1007\/BF01580598"},{"key":"ref211","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.1986.1104412"},{"key":"ref212","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2023.0324"},{"key":"ref213","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2024.3416514"},{"key":"ref214","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-023-00520-0"},{"key":"ref215","doi-asserted-by":"publisher","DOI":"10.1137\/22M1530161"},{"key":"ref216","doi-asserted-by":"publisher","DOI":"10.1137\/22M1474539"},{"key":"ref217","doi-asserted-by":"publisher","DOI":"10.1137\/23M1627614"},{"key":"ref218","first-page":"1","author":"Wen Z.","year":"2009","journal-title":"IEOR report, Columbia University"},{"key":"ref219","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-017-0624-3"},{"key":"ref220","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-018-0134-9"},{"key":"ref221","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-021-01409-y"},{"key":"ref222","doi-asserted-by":"crossref","unstructured":"J. Xu, S. Zhu, Y. C. Soh, and L. Xie, Augmented distributed gradient methods for multi-agent optimization under uncoordinated constant stepsizes, in  54th IEEE Conference on Decision and Control (CDC), IEEE, 2015, pp. 2055\u20132060.","DOI":"10.1109\/CDC.2015.7402509"},{"key":"ref223","doi-asserted-by":"publisher","DOI":"10.1007\/s10589-013-9627-7"},{"key":"ref224","doi-asserted-by":"publisher","DOI":"10.1137\/16M1082305"},{"key":"ref225","doi-asserted-by":"publisher","DOI":"10.1137\/18M1229869"},{"key":"ref226","doi-asserted-by":"publisher","DOI":"10.1287\/ijoo.2019.0033"},{"key":"ref227","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-019-01425-9"},{"key":"ref228","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-018-0680-3"},{"key":"ref229","unstructured":"S. Yan and N. He, Bregman Augmented Lagrangian and Its Acceleration, preprint, arXiv:2002.06315, 2020."},{"key":"ref230","doi-asserted-by":"publisher","DOI":"10.1090\/S0025-5718-2012-02598-1"},{"key":"ref231","doi-asserted-by":"publisher","DOI":"10.1137\/090777761"},{"key":"ref232","doi-asserted-by":"publisher","DOI":"10.1007\/s12532-015-0082-6"},{"key":"ref233","doi-asserted-by":"publisher","DOI":"10.1137\/20M135474X"},{"key":"ref234","doi-asserted-by":"publisher","DOI":"10.1287\/ijoc.2022.1228"},{"key":"ref235","doi-asserted-by":"publisher","DOI":"10.1137\/20M1344160"},{"key":"ref236","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2020.102823"},{"key":"ref237","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-018-1329-6"},{"key":"ref238","doi-asserted-by":"publisher","DOI":"10.1137\/19M1267830"},{"key":"ref239","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-92900-1"},{"key":"ref240","doi-asserted-by":"publisher","DOI":"10.1137\/080718206"},{"key":"ref241","first-page":"201","author":"Zhou Y.","year":"2023","journal-title":"Math. Program."},{"key":"ref242","first-page":"1","author":"Zhou Y.","year":"2026","journal-title":"Sci. China Math."},{"key":"ref243","doi-asserted-by":"publisher","DOI":"10.1007\/s10898-013-0046-1"},{"key":"ref244","doi-asserted-by":"publisher","DOI":"10.1287\/moor.2022.1350"},{"key":"ref245","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-017-0359-1"},{"key":"ref246","first-page":"8","volume":"34","author":"Zhu M.","year":"2008","journal-title":"UCLA CAM Report"},{"key":"ref247","unstructured":"Z. Zhu, F. Chen, J. Zhang, and Z. Wen, On the Optimal Lower and Upper Complexity Bounds for a Class of Composite Optimization Problems, preprint, arXiv:2308.06470, 2023."},{"key":"ref248","doi-asserted-by":"publisher","DOI":"10.1007\/s10915-023-02346-8"}],"container-title":["SIAM Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/epubs.siam.org\/doi\/pdf\/10.1137\/24M1706153","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T12:58:30Z","timestamp":1787230710000},"score":1,"resource":{"primary":{"URL":"https:\/\/epubs.siam.org\/doi\/10.1137\/24M1706153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,6]]},"references-count":248,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,9,30]]}},"alternative-id":["10.1137\/24M1706153"],"URL":"https:\/\/doi.org\/10.1137\/24m1706153","relation":{},"ISSN":["0036-1445","1095-7200"],"issn-type":[{"value":"0036-1445","type":"print"},{"value":"1095-7200","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,6]]}}}