{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T12:36:06Z","timestamp":1754483766611,"version":"3.37.3"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,10,5]],"date-time":"2024-10-05T00:00:00Z","timestamp":1728086400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,5]],"date-time":"2024-10-05T00:00:00Z","timestamp":1728086400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004377","name":"Hong Kong Polytechnic University","doi-asserted-by":"publisher","award":["15300021"],"award-info":[{"award-number":["15300021"]}],"id":[{"id":"10.13039\/501100004377","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Sci Comput"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The first-order optimality condition of convexly constrained nonconvex nonconcave min-max optimization problems with box constraints formulates a nonmonotone variational inequality (VI), which is equivalent to a system of nonsmooth equations. In this paper, we propose a quasi-Newton subspace trust region (QNSTR) algorithm for the least squares problems defined by the smoothing approximation of nonsmooth equations. Based on the structure of the nonmonotone VI, we use an adaptive quasi-Newton formula to approximate the Hessian matrix and solve a low-dimensional strongly convex quadratic program with ellipse constraints in a subspace at each step of the QNSTR algorithm efficiently. We prove the global convergence of the QNSTR algorithm to an <jats:inline-formula><jats:alternatives><jats:tex-math>$$\\epsilon $$<\/jats:tex-math><mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                  <mml:mi>\u03f5<\/mml:mi>\n                <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-first-order stationary point of the min-max optimization problem. Moreover, we present numerical results based on the QNSTR algorithm with different subspaces for a mixed generative adversarial networks in eye image segmentation using real data to show the efficiency and effectiveness of the QNSTR algorithm for solving large-scale min-max optimization problems.\n<\/jats:p>","DOI":"10.1007\/s10915-024-02679-y","type":"journal-article","created":{"date-parts":[[2024,10,5]],"date-time":"2024-10-05T06:02:11Z","timestamp":1728108131000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["A Quasi-Newton Subspace Trust Region Algorithm for Nonmonotone Variational Inequalities in Adversarial Learning over Box Constraints"],"prefix":"10.1007","volume":"101","author":[{"given":"Zicheng","family":"Qiu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6627-2637","authenticated-orcid":false,"given":"Xiaojun","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,10,5]]},"reference":[{"key":"2679_CR1","doi-asserted-by":"crossref","unstructured":"Alom, M.Z., Hasan, M., Yakopcic, C., Taha, T.M., Asari, V.K.: Recurrent residual convolutional neural network based on U-net (R2U-Net) for medical image segmentation. arXiv preprint arXiv:1802.06955 (2018)","DOI":"10.1109\/NAECON.2018.8556686"},{"key":"2679_CR2","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"AP Bradley","year":"1997","unstructured":"Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. 30, 1145\u20131159 (1997)","journal-title":"Pattern Recognit."},{"key":"2679_CR3","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/S0377-0427(97)80133-1","volume":"80","author":"X Chen","year":"1997","unstructured":"Chen, X.: Superlinear convergence of smoothing quasi-Newton methods for nonsmooth equations. J. Comput. Appl. Math. 80, 105\u2013126 (1997)","journal-title":"J. Comput. Appl. Math."},{"key":"2679_CR4","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1090\/S0025-5718-98-00932-6","volume":"67","author":"X Chen","year":"1998","unstructured":"Chen, X., Qi, L., Sun, D.: Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities. Math. Comput. 67, 519\u2013540 (1998)","journal-title":"Math. Comput."},{"key":"2679_CR5","volume-title":"Optimization and Nonsmooth Analysis","author":"FH Clarke","year":"1983","unstructured":"Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983)"},{"key":"2679_CR6","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898719857","volume-title":"Trust Region Methods","author":"AR Conn","year":"2000","unstructured":"Conn, A.R., Gould, N.I., Toint, P.L.: Trust Region Methods. SIAM, Philadelphia (2000)"},{"key":"2679_CR7","unstructured":"Dai, B., Shaw, A., Li, L., Xiao, L., He, N., Liu, Z., Chen, J., Song, L.: SBEED: convergent reinforcement learning with nonlinear function approximation. In: Proceedings of the International Conference on Machine Learning, pp. 1125\u20131134. PMLR (2018)"},{"key":"2679_CR8","doi-asserted-by":"publisher","first-page":"296","DOI":"10.4208\/csiam-am.2020-0014","volume":"1","author":"YH Dai","year":"2020","unstructured":"Dai, Y.H., Zhang, L.: Optimality conditions for constrained minimax optimization. CSIAM Trans. Appl. Math. 1, 296\u2013315 (2020)","journal-title":"CSIAM Trans. Appl. Math."},{"key":"2679_CR9","unstructured":"Daskalakis, C., Ilyas, A., Syrgkanis, V., Zeng, H.: Training gans with optimism. arXiv preprint arXiv:1711.00141 (2017)"},{"key":"2679_CR10","volume-title":"Numerical Methods for Unconstrained Optimization and Nonlinear Equations","author":"JE Dennis","year":"1983","unstructured":"Dennis, J.E., Schnabel, R.B.: Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Prentice-Hall, Englewood Cliffs, NJ (1983)"},{"key":"2679_CR11","unstructured":"Diakonikolas, J., Daskalakis, C., Jordan, M.I.: Efficient methods for structured nonconvex-nonconcave min-max optimization. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 2746\u20132754. PMLR (2021)"},{"key":"2679_CR12","volume-title":"Finite-dimensional Variational Inequalities and Complementarity Problems","author":"F Facchinei","year":"2003","unstructured":"Facchinei, F., Pang, J.S.: Finite-dimensional Variational Inequalities and Complementarity Problems. Springer, New York (2003)"},{"key":"2679_CR13","volume-title":"Computer Vision: A Modern Approach","author":"DA Forsyth","year":"2002","unstructured":"Forsyth, D.A., Ponce, J.: Computer Vision: A Modern Approach. Prentice Hall Professional Technical Reference, Upper Saddle River (2002)"},{"key":"2679_CR14","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural Inform. Process. Syst. 27 (2014)"},{"key":"2679_CR15","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1137\/050624935","volume":"18","author":"S Gratton","year":"2007","unstructured":"Gratton, S., Lawless, A.S., Nichols, N.K.: Approximate Gauss-Newton methods for nonlinear least squares problems. SIAM J. Optim. 18, 106\u2013132 (2007)","journal-title":"SIAM J. Optim."},{"key":"2679_CR16","unstructured":"Hendrycks, D., Gimpel, K.: Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016)"},{"key":"2679_CR17","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125\u20131134 (2017)","DOI":"10.1109\/CVPR.2017.632"},{"key":"2679_CR18","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1137\/22M1482238","volume":"5","author":"J Jiang","year":"2023","unstructured":"Jiang, J., Chen, X.: Optimality conditions for nonsmooth nonconvex-nonconcave min-max problems and generative adversarial networks. SIAM J. Math. Data Sci. 5, 693\u2013722 (2023)","journal-title":"SIAM J. Math. Data Sci."},{"key":"2679_CR19","unstructured":"Jin, C., Netrapalli, P., Jordan, M.: What is local optimality in nonconvex-nonconcave minimax optimization? In: Proceedings of International Conference on Machine Learning, pp. 4880\u20134889. PMLR (2020)"},{"key":"2679_CR20","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1016\/j.knosys.2019.04.025","volume":"178","author":"Q Jin","year":"2019","unstructured":"Jin, Q., Meng, Z., Pham, T.D., Chen, Q., Wei, L., Su, R.: DUNet: a deformable network for retinal vessel segmentation. Knowl.-Based Syst. 178, 149\u2013162 (2019)","journal-title":"Knowl.-Based Syst."},{"key":"2679_CR21","doi-asserted-by":"crossref","unstructured":"Li, L., Verma, M., Nakashima, Y., Nagahara, H., Kawasaki, R.: Iternet: Retinal image segmentation utilizing structural redundancy in vessel networks. In: Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, pp. 3656\u20133665 (2020)","DOI":"10.1109\/WACV45572.2020.9093621"},{"key":"2679_CR22","unstructured":"Lin, T., Jin, C., Jordan, M.: On gradient descent ascent for nonconvex-concave minimax problems. In: Proceedings of the International Conference on Machine Learning, pp. 6083\u20136093. PMLR (2020)"},{"key":"2679_CR23","unstructured":"Lin, T., Jin, C., Jordan, M.I.: Near-optimal algorithms for minimax optimization. In: Proceedings of the Conference on Learning Theory, pp. 2738\u20132779. PMLR (2020)"},{"key":"2679_CR24","first-page":"7651","volume":"22","author":"M Liu","year":"2021","unstructured":"Liu, M., Rafique, H., Lin, Q., Yang, T.: First-order convergence theory for weakly-convex-weakly-concave min-max problems. J. Mach. Learn. Res. 22, 7651\u20137684 (2021)","journal-title":"J. Mach. Learn. Res."},{"key":"2679_CR25","unstructured":"Luc, P., Couprie, C., Chintala, S., Verbeek, J.: Semantic segmentation using adversarial networks. arXiv preprint arXiv:1611.08408 (2016)"},{"key":"2679_CR26","unstructured":"Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)"},{"key":"2679_CR27","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s10107-017-1172-1","volume":"171","author":"P Mohajerin Esfahani","year":"2018","unstructured":"Mohajerin Esfahani, P., Kuhn, D.: Data-driven distributionally robust optimization using the Wasserstein metric: performance guarantees and tractable reformulations. Math. Program. 171, 115\u2013166 (2018)","journal-title":"Math. Program."},{"key":"2679_CR28","doi-asserted-by":"publisher","first-page":"1688","DOI":"10.1137\/100801652","volume":"21","author":"RD Monteiro","year":"2011","unstructured":"Monteiro, R.D., Svaiter, B.F.: Complexity of variants of Tseng\u2019s modified FB splitting and Korpelevich\u2019s methods for hemivariational inequalities with applications to saddle-point and convex optimization problems. SIAM J. Optim. 21, 1688\u20131720 (2011)","journal-title":"SIAM J. Optim."},{"key":"2679_CR29","volume-title":"Game Theory: Analysis of Conflict","author":"RB Myerson","year":"1991","unstructured":"Myerson, R.B.: Game Theory: Analysis of Conflict. Harvard University Press, Cambridge (1991)"},{"key":"2679_CR30","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1137\/S1052623403425629","volume":"15","author":"A Nemirovski","year":"2004","unstructured":"Nemirovski, A.: Prox-method with rate of convergence O (1\/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems. SIAM J. Optim. 15, 229\u2013251 (2004)","journal-title":"SIAM J. Optim."},{"key":"2679_CR31","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1007\/s10107-006-0034-z","volume":"109","author":"Y Nesterov","year":"2007","unstructured":"Nesterov, Y.: Dual extrapolation and its applications to solving variational inequalities and related problems. Math. Program. 109, 319\u2013344 (2007)","journal-title":"Math. Program."},{"key":"2679_CR32","doi-asserted-by":"crossref","unstructured":"Rabbat, M., Nowak, R.: Distributed optimization in sensor networks. In: Proceedings of the 3rd International Symposium on Information Processing in Sensor Networks, pp. 20\u201327 (2004)","DOI":"10.1145\/984622.984626"},{"key":"2679_CR33","doi-asserted-by":"publisher","first-page":"1087","DOI":"10.1080\/10556788.2021.1895152","volume":"37","author":"H Rafique","year":"2022","unstructured":"Rafique, H., Liu, M., Lin, Q., Yang, T.: Weakly-convex-concave min-max optimization: provable algorithms and applications in machine learning. Optim. Method Softw. 37, 1087\u20131121 (2022)","journal-title":"Optim. Method Softw."},{"key":"2679_CR34","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of the Medical Image Computing and Computer-Assisted Intervention\u2013MICCAI 2015: 18th International Conference, pp. 234\u2013241. Springer (2015)","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"2679_CR35","doi-asserted-by":"crossref","unstructured":"Shapiro, A.: Monte Carlo sampling methods. In: Ruszczy\u0144ski, A., Shapiro, A. (eds.) Handbooks in Operations Research and Management Science, pp. 353\u2013425. Elsevier, Amsterdam (2003)","DOI":"10.1016\/S0927-0507(03)10006-0"},{"key":"2679_CR36","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611976595","volume-title":"Lectures on Stochastic Programming: Modeling and Theory","author":"A Shapiro","year":"2021","unstructured":"Shapiro, A., Dentcheva, D., Ruszczynski, A.: Lectures on Stochastic Programming: Modeling and Theory. SIAM, Philadelphia (2021)"},{"key":"2679_CR37","doi-asserted-by":"publisher","first-page":"499","DOI":"10.1007\/s10278-018-0126-3","volume":"32","author":"J Son","year":"2019","unstructured":"Son, J., Park, S.J., Jung, K.H.: Towards accurate segmentation of retinal vessels and the optic disc in fundoscopic images with generative adversarial networks. J. Digit. Imaging 32, 499\u2013512 (2019)","journal-title":"J. Digit. Imaging"},{"key":"2679_CR38","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34372-9","volume-title":"Computer Vision: Algorithms and Applications","author":"R Szeliski","year":"2022","unstructured":"Szeliski, R.: Computer Vision: Algorithms and Applications. Springer Nature, Switzerland (2022)"},{"key":"2679_CR39","unstructured":"Tseng, P.: On accelerated proximal gradient methods for convex-concave optimization. submitted to SIAM J. Optim. 2 (2008)"},{"key":"2679_CR40","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TIP.2003.819861","volume":"13","author":"Z Wang","year":"2004","unstructured":"Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600\u2013612 (2004)","journal-title":"IEEE Trans. Image Process."},{"key":"2679_CR41","doi-asserted-by":"publisher","first-page":"692","DOI":"10.1016\/j.jmaa.2010.03.021","volume":"368","author":"H Xu","year":"2010","unstructured":"Xu, H.: Uniform exponential convergence of sample average random functions under general sampling with applications in stochastic programming. J. Math. Anal. Appl. 368, 692\u2013710 (2010)","journal-title":"J. Math. Anal. Appl."},{"key":"2679_CR42","first-page":"1153","volume":"33","author":"J Yang","year":"2020","unstructured":"Yang, J., Kiyavash, N., He, N.: Global convergence and variance reduction for a class of nonconvex-nonconcave minimax problems. Adv. Neural. Inf. Process. Syst. 33, 1153\u20131165 (2020)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"2679_CR43","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1007\/s10107-015-0893-2","volume":"151","author":"Y Yuan","year":"2015","unstructured":"Yuan, Y.: Recent advances in trust region algorithms. Math. Program. 151, 249\u2013281 (2015)","journal-title":"Math. Program."},{"key":"2679_CR44","unstructured":"Zhang, C., Ge, D., Jiang, B., Ye, Y.: DRSOM: A dimension reduced second-order method and preliminary analyses. arXiv preprint arXiv:2208.00208 (2022)"},{"key":"2679_CR45","unstructured":"Zhang, G., Wang, Y., Lessard, L., Grosse, R.B.: Near-optimal local convergence of alternating gradient descent-ascent for minimax optimization. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 7659\u20137679. PMLR (2022)"},{"key":"2679_CR46","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Yang, L., Chen, J., Fredericksen, M., Hughes, D.P., Chen, D.Z.: Deep adversarial networks for biomedical image segmentation utilizing unannotated images. In: Proceedings of the Medical Image Computing and Computer Assisted Intervention- MICCAI 2017: 20th International Conference, pp. 408\u2013416. Springer (2017)","DOI":"10.1007\/978-3-319-66179-7_47"},{"key":"2679_CR47","doi-asserted-by":"publisher","first-page":"2422","DOI":"10.1137\/090748470","volume":"20","author":"W Zhou","year":"2010","unstructured":"Zhou, W., Chen, X.: Global convergence of a new hybrid Gauss-Newton structured BFGS method for nonlinear least squares problems. SIAM J. Optim. 20, 2422\u20132441 (2010)","journal-title":"SIAM J. Optim."}],"container-title":["Journal of Scientific Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10915-024-02679-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10915-024-02679-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10915-024-02679-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,18]],"date-time":"2024-10-18T09:17:37Z","timestamp":1729243057000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10915-024-02679-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,5]]},"references-count":47,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["2679"],"URL":"https:\/\/doi.org\/10.1007\/s10915-024-02679-y","relation":{},"ISSN":["0885-7474","1573-7691"],"issn-type":[{"type":"print","value":"0885-7474"},{"type":"electronic","value":"1573-7691"}],"subject":[],"published":{"date-parts":[[2024,10,5]]},"assertion":[{"value":"11 September 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 April 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 October 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"45"}}