{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T08:40:16Z","timestamp":1741941616917,"version":"3.38.0"},"reference-count":45,"publisher":"China Science Publishing & Media Ltd.","issue":"1","license":[{"start":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T00:00:00Z","timestamp":1710288000000},"content-version":"vor","delay-in-days":72,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,2,1]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n               <jats:p>The wide applications of Generative adversarial networks benefit from the successful training methods, guaranteeing that an object function converges to the local minimum. Nevertheless, designing an efficient and competitive training method is still a challenging task due to the cyclic behaviors of some gradient-based ways and the expensive computational cost of acquiring the Hessian matrix. To address this problem, we proposed the Adaptive Composite Gradients(ACG) method, linearly convergent in bilinear games under suitable settings. Theory analysis and toy-function experiments both suggest that our approach alleviates the cyclic behaviors and converges faster than recently proposed SOTA algorithms. The convergence speed of the ACG is improved by 33% than other methods. Our ACG method is a novel Semi-Gradient-Free algorithm that can reduce the computational cost of gradient and Hessian by utilizing the predictive information in future iterations. The mixture of Gaussians experiments and real-world digital image generative experiments show that our ACG method outperforms several existing technologies, illustrating the superiority and efficacy of our method.<\/jats:p>","DOI":"10.1162\/dint_a_00246","type":"journal-article","created":{"date-parts":[[2024,3,13]],"date-time":"2024-03-13T12:20:40Z","timestamp":1710332440000},"page":"120-157","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Training Generative Adversarial Networks with Adaptive Composite Gradient"],"prefix":"10.3724","volume":"6","author":[{"given":"Huiqing","family":"Qi","sequence":"first","affiliation":[]},{"given":"Fang","family":"Li","sequence":"additional","affiliation":[]},{"given":"Shengli","family":"Tan","sequence":"additional","affiliation":[]},{"given":"Xiangyun","family":"Zhang","sequence":"additional","affiliation":[]}],"member":"2026","published-online":{"date-parts":[[2024,2,1]]},"reference":[{"issue":"4","key":"2024041719553874700_ref1","doi-asserted-by":"crossref","first-page":"1415","DOI":"10.1080\/10556788.2021.2023522","article-title":"A quasi-Newton proximal bundle method using gradient sampling technique for minimizing nonsmooth convex functions","volume":"37","author":"Maleknia","year":"2022","journal-title":"Optimization Methods and Software"},{"issue":"6","key":"2024041719553874700_ref2","doi-asserted-by":"crossref","first-page":"3103","DOI":"10.1109\/TCYB.2020.2977661","article-title":"Deep reinforcement learning for multiobjective optimization","volume":"51","author":"Li","year":"2020","journal-title":"IEEE transactions on cybernetics"},{"key":"2024041719553874700_ref3","first-page":"3540","article-title":"Feudal networks for hierarchical reinforcement learning","volume-title":"International conference on machine learning","author":"Vezhnevets","year":"2017"},{"issue":"11","key":"2024041719553874700_ref4","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3422622","article-title":"Generative adversarial networks","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Communications of the ACM"},{"issue":"1","key":"2024041719553874700_ref5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3301282","article-title":"How generative adversarial networks and their variants work: An overview","volume":"52","author":"Hong","year":"2019","journal-title":"ACM Computing Surveys (CSUR)"},{"volume-title":"Objective-reinforced generative adversarial networks (organ) for sequence generation models.","year":"2017","author":"Guimaraes","key":"2024041719553874700_ref6"},{"volume-title":"Polyphonic music generation with sequence generative adversarial networks.","year":"2017","author":"Lee","key":"2024041719553874700_ref7"},{"volume-title":"SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.","year":"2016","author":"Yu","key":"2024041719553874700_ref8"},{"key":"2024041719553874700_ref9","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2017-63","volume-title":"Voice conversion from unaligned corpora using variational autoencoding wasserstein generative adversarial networks.","author":"Hsu","year":"2017"},{"key":"2024041719553874700_ref10","article-title":"Adversarial ranking for language generation","volume":"30","author":"Lin","year":"2017","journal-title":"Advances in neural information processing systems"},{"volume-title":"TextKD-GAN: Text Generation using KnowledgeDistillation and Generative Adversarial Networks.","year":"2019","author":"Akmal Haidar","key":"2024041719553874700_ref11"},{"key":"2024041719553874700_ref12","doi-asserted-by":"crossref","first-page":"2114","DOI":"10.18653\/v1\/2020.acl-main.191","article-title":"GAN-BERT: Generative adversarial learning for robust text classification with a bunch of labeled examples","volume-title":"Proceedings of the 58th annual meeting of the association for computational linguistics","author":"Croce","year":"2020"},{"volume-title":"Stabilizing GAN training with multiple random projections.","year":"2017","author":"Neyshabur","key":"2024041719553874700_ref13"},{"key":"2024041719553874700_ref14","first-page":"5599","article-title":"Training generative adversarial networks by solving ordinary di erential equations","volume":"33","author":"Qin","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"5","key":"2024041719553874700_ref15","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1080\/10556788.2020.1754414","article-title":"GANs with centripetal acceleration","volume":"35","author":"Peng","year":"2020","journal-title":"Optimization Methods and Software"},{"volume-title":"Training GANS with predictive projection centripetal acceleration","year":"2020","author":"Keke","key":"2024041719553874700_ref16"},{"key":"2024041719553874700_ref17","first-page":"4681","article-title":"Photo-realistic single image super-resolution using a generative adversarial network","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Ledig","year":"2017"},{"key":"2024041719553874700_ref18","article-title":"Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling","volume":"29","author":"Wu","year":"2016","journal-title":"Advances in neural information processing systems"},{"key":"2024041719553874700_ref19","first-page":"2223","article-title":"Unpaired image-to-image translation using cycleconsistent adversarial networks","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"Zhu","year":"2017"},{"issue":"8","key":"2024041719553874700_ref20","doi-asserted-by":"crossref","first-page":"4066","DOI":"10.1109\/TIP.2018.2836316","article-title":"Perceptual adversarial networks for image-to-image transformation","volume":"27","author":"Wang","year":"2018","journal-title":"IEEE Transactions on Image Processing"},{"key":"2024041719553874700_ref21","first-page":"3332","article-title":"The pose knows: Video forecasting by generating pose futures","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"Walker","year":"2017"},{"key":"2024041719553874700_ref22","first-page":"1526","article-title":"Mocogan: Decomposing motion and content for video generation","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Tulyakov","year":"2018"},{"key":"2024041719553874700_ref23","first-page":"41","article-title":"Dual Adversarial Network: Toward Real-World Noise Removal and Noise Generation","volume-title":"European Conference on Computer Vision","author":"Yue","year":"2020"},{"key":"2024041719553874700_ref24","first-page":"8183","article-title":"Deblurgan: Blind motion deblurring using conditional adversarial networks","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"Kupyn","year":"2018"},{"key":"2024041719553874700_ref25","first-page":"1857","article-title":"Learning to discover cross-domain relations with generative adversarial networks","volume-title":"International conference on machine learning","author":"Kim","year":"2017"},{"key":"2024041719553874700_ref26","first-page":"2849","article-title":"Dualgan: Unsupervised dual learning for image-to-image translation","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"Yi","year":"2017"},{"key":"2024041719553874700_ref27","first-page":"633","article-title":"Automatic vertebra labeling in large-scale 3D CT using deep image-to-image network with message passing and sparsity regularization","volume-title":"25th International Conference on Information Processing in Medical Imaging","author":"Yang","year":"2017"},{"volume-title":"A mathematical introduction to generative adversarial nets (GAN).","year":"2020","author":"Wang","key":"2024041719553874700_ref28"},{"issue":"4","key":"2024041719553874700_ref29","doi-asserted-by":"crossref","first-page":"18","DOI":"10.23915\/distill.00018","article-title":"Open questions about generative adversarial networks","volume":"4","author":"Odena","year":"2019","journal-title":"Distill"},{"volume-title":"Nips 2016 tutorial: Generative adversarial networks.","year":"2016","author":"Goodfellow","key":"2024041719553874700_ref30"},{"key":"2024041719553874700_ref31","article-title":"Trajectory of alternating direction method of multipliers and adaptive acceleration","volume":"32","author":"Poon","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024041719553874700_ref32","first-page":"354","article-title":"The mechanics of n-player di erentiable games","volume-title":"International Conference on Machine Learning","author":"Balduzzi","year":"2018"},{"key":"2024041719553874700_ref33","article-title":"f-gan: Training generative neural samplers using variational divergence minimization","volume":"29","author":"Nowozin","year":"2016","journal-title":"Advances in neural information processing systems"},{"key":"2024041719553874700_ref34","first-page":"907","article-title":"Interaction matters: A note on non-asymptotic local convergence of generative adversarial networks","volume-title":"The 22nd International Conference on Artificial Intelligence and Statistics","author":"Liang","year":"2019"},{"key":"2024041719553874700_ref35","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-48311-5_31","volume-title":"Convex Analysis and Monotone Operator Theory in Hilbert Spaces","author":"Bauschke","year":"2019"},{"issue":"1","key":"2024041719553874700_ref36","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1016\/0022-247X(77)90152-4","article-title":"On the weak convergence of an ergodic iteration for the solution of variational inequalities for monotone operators in Hilbert space","volume":"61","author":"Bruck","year":"1977","journal-title":"Journal of Mathematical Analysis and Applications"},{"key":"2024041719553874700_ref37","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1007\/s10957-009-9522-7","article-title":"Subgradient methods for saddle-point problems","volume":"142","author":"Nedic","year":"2009","journal-title":"Journal of optimization theory and applications"},{"key":"2024041719553874700_ref38","first-page":"543","article-title":"A method for solving the convex programming problem with convergence rate O(1\/k2)","volume":"269","author":"Nesterov","year":"1983","journal-title":"Proceedings of the USSR Academy of Sciences"},{"volume-title":"Unified convergence analysis of stochastic momentum methods for convex and nonconvex optimization.","year":"2016","author":"Yang","key":"2024041719553874700_ref39"},{"key":"2024041719553874700_ref40","article-title":"The numerics of gans","volume":"30","author":"Mescheder","year":"2017","journal-title":"Advances in neural information processing systems"},{"issue":"11","key":"2024041719553874700_ref41","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE"},{"volume-title":"Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.","year":"2017","author":"Xiao","key":"2024041719553874700_ref42"},{"issue":"7","key":"2024041719553874700_ref43","first-page":"1","article-title":"Convolutional deep belief networks on cifar-10","volume":"40","author":"Krizhevsky","year":"2010","journal-title":"Unpublished manuscript"},{"key":"2024041719553874700_ref44","first-page":"3730","article-title":"Deep learning face attributes in the wild","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"Liu","year":"2015"},{"volume-title":"Unsupervised representation learning with deep convolutional generative adversarial networks.","year":"2015","author":"Radford","key":"2024041719553874700_ref45"}],"container-title":["Data Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/dint\/article-pdf\/6\/1\/120\/2364222\/dint_a_00246.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/dint\/article-pdf\/6\/1\/120\/2364222\/dint_a_00246.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,14]],"date-time":"2025-03-14T07:42:49Z","timestamp":1741938169000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.sciengine.com\/doi\/10.1162\/dint_a_00246"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,1]]}},"URL":"https:\/\/doi.org\/10.1162\/dint_a_00246","relation":{},"ISSN":["2641-435X"],"issn-type":[{"type":"electronic","value":"2641-435X"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}