运筹学学报 >
2024 , Vol. 28 >Issue 1: 18 - 28
DOI: https://doi.org/10.15960/j.cnki.issn.1007-6093.2024.01.002
单边相对光滑非凸-凹极小极大问题的镜像梯度算法
收稿日期: 2021-09-24
网络出版日期: 2024-03-16
基金资助
国家自然科学基金(12071279);上海市自然科学基金(20ZR1420600)
版权
A mirror descent gradient ascent algorithm for one side relatively smooth nonconvex-concave minimax optimization problems
Received date: 2021-09-24
Online published: 2024-03-16
Copyright
本文提出了一种镜像梯度下降梯度上升算法来求解单边相对光滑的非凸-凹极小极大问题。在算法的每次迭代中, 我们采用镜像梯度下降步来更新相对光滑的变量, 采用梯度上升投影步来更新目标函数中光滑的变量。本文在理论上证明了算法收敛到
关键词: 非凸-凹极小极大问题; 相对光滑; 镜像梯度法
徐洋, 王军霖, 徐姿 . 单边相对光滑非凸-凹极小极大问题的镜像梯度算法[J]. 运筹学学报, 2024 , 28(1) : 18 -28 . DOI: 10.15960/j.cnki.issn.1007-6093.2024.01.002
In this paper, we propose a mirror descent gradient ascent algorithm to solve one side relatively smooth nonconvex-concave minimax optimization problems. At each iteration of the proposed algorithm, a mirror descent step is performed to update the relatively smooth variable, while a gradient ascent projection step is used to update the smooth variable alternately. We also prove that the iteration complexity of the proposed algorithm is
| 1 | Sinha A, Namkoong H, Du C. Certifiable distributional robustness with principled adversarial training[C]//International Conference on Learning Representations, 2018. |
| 2 | Goodfellow I , Pouget-Abadie J , Mirza M .Generative adversarial nets[J].Advances in Neural Information Processing Systems,2014,2672-2680. |
| 3 | Zhang Y, Xiao L. Stochastic primal-dual coordinate method for regularized empirical risk minimization[C]//International Conference on Machine Learning, 2015: 353-361. |
| 4 | Nemirovski A .Prox-method with rate of convergence O(1/$t$) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems[J].SIAM Journal on Optimization,2004,15(1):229-251. |
| 5 | Nesterov Y .Dual extrapolation and its applications to solving variational inequalities and related problems[J].Mathematical Programming,2007,109(2/3):319-344. |
| 6 | Lin T, Jin C, Micheal J. Near-optimal algorithms for minimax optimization[C]//Conference on Learning Theory, 2020: 2738-2779. |
| 7 | Lu S , Tsaknakis I , Hong M , et al.Hybrid block successive approximation for one-sided non-convex min-max problems: Algorithms and applications[J].IEEE Transactions on Signal Processing,2020,68,3676-3691. |
| 8 | Xu Z , Zhang H , Xu Y , et al.A unified single-loop alternating gradient projection algorithm for nonconvex-concave and convex-nonconcave minimax problems[J].Mathematical Programming,2023,201(1-2):635-706. |
| 9 | Zhang J , Xiao P , Sun R , et al.A single-loop smoothed gradient descent-ascent algorithm for nonconvex-concave min-max problems[J].NeurIPS,2020,33,7377-7389. |
| 10 | 徐姿, 张慧灵.非凸极小极大问题的优化算法与复杂度分析[J].运筹学学报,2021,25(3):1-13. |
| 11 | Tseng P. On accelerated proximal gradient methods for convex-concave optimization[EB/OL]. (2008-05-21)[2021-08-09]. https://www.csie.ntu.edu.tw/b97058/tseng/papers/apgm.pdf. |
| 12 | Hou C, Thekumparampil K, Fanti G. Efficient algorithms for federated saddle point optimization[EB/OL]. (2021-02-12)[2021-08-09]. arXiv: 2102.06333. |
| 13 | Huang F, Gao S, Heng H. Bregman gradient policy optimization[EB/OL]. (2021-06-23)[2021-08-09]. arXiv: 2106.12112. |
| 14 | Lu H , Freund M , Nesterov Y .Relatively-smooth convex optimization by first-order methods and applications[J].SIAM Journal on Optimization,2018,28(1):333-354. |
| 15 | Ghadimi S , Lan G , Zhang H .Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization[J].Mathematical Programming,2016,155,267-305. |
| 16 | Nesterov Y .Introductory Lectures on Convex Optimization[M].New York: Springer Science & Business Media,2013. |
| 17 | Balseiro S R , Lu H , Mirrokni V .The best of many worlds: Dual mirror descent for online allocation problems[J].Operations Research,2023,71(1):101-119. |
| 18 | Ibrahim M, Konar A, Hong M. Mirror-prox SCA algorithm for multicast beamforming and antenna selection[C]//IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications, 2018. |
/
| 〈 |
|
〉 |