在弱Wolfe-Powell线搜索技术下,通过PRP共轭梯度法如何获得非凸函数的全局收敛仍是一个公开问题。本文针对大规模无约束优化问题,提出了一种混合的共轭梯度方法(MPRP)。该方法是将修正的BFGS方法与修正的PRP共轭梯度法混合,采用了弱Wolfe-Powell线搜索技术来寻找步长,其搜索方向具有充分下降的性质。在理论上,通过对条件合理的假设, 确保了非凸函数的全局收敛性。在数值实验上,通过对Muskingum模型的参数估计,减少了计算量和存储量,说明了MPRP 的有效性; 在不同噪声情况下,通过对比多种图像的恢复情况,证明了MPRP 有较强的竞争力; 并且在低脉冲噪声图像下,图像的恢复情况较为显著。
It is well-known that, under the WWP (weak Wolfe-Powell) line search technique, the global convergence of the PRP conjugate gradient method for non-convex functions is still open. In this paper, a hybrid conjugate gradient method is proposed for large-scale unconstrained optimization problems. In this method, the modified BFGS method is mixed with the modified PRP conjugate gradient method, and the weak Wolfe-Powell line search technique is used to find the step size, and the search direction has the property of sufficient descent. Theoretically, the global convergence of nonconvex functions is ensured by assuming reasonable conditions. In numerical experiments, the parameter estimation of the Muskingum model reduces the amount of computation and storage, which illustrates the effectiveness of MPRP. In different noise situations, the MPRP is proved to be highly competitive by comparing the recovery of multiple images. In addition, the restoration of the image is more significant under the image of low impulse noise.
[1] 王宜举，修乃华.非线性最优化理论与方法，第2版[M].北京：科学出版社，2016:49-66.
[2] Fletcher R, Reeves C M. Function minimization by conjugate gradients [J]. The Computer Journal, 1964, 7(2): 149-154.
[3] Polak E, Ribiere G. Note sur la convergence de méthodes de directions conjuguées [J]. Revue francaise d’informatique et de recherche opérationnelle. Série rouge, 1969, 3(16): 35-43.
[4] Polyak B T. The conjugate gradient method in extremal problems [J]. USSR Computational Mathematics and Mathematical Physics, 1969, 9(4): 94-112.
[5] Hestenes M R, Stiefel E. Methods of conjugate gradients for solving linear systems [J]. Journal of Research of the National Bureau of Standards, 1952, 49(6): 409-436.
[6] Fletcher R. Practical Method of Optimization, Vol I: Unconstrained Optimization [M]. New York: John Wiley and Sons, 1987.
[7] Liu Y, Storey C. Efficient generalized conjugate gradient algorithms, part 1: Theory [J]. Journal of Optimization Theory and Applications, 1991, 69: 129-137.
[8] Dai Y H, Yuan Y. A nonlinear conjugate gradient method with a strong global convergence property [J]. SIAM Journal on Optimization, 1999, 10(1): 177-182.
[9] Dai Y, Yuan Y. An efficient hybrid conjugate gradient method for unconstrained optimization [J]. Annals of Operations Research, 2001, 103: 33-47.
[10] Touati-Ahmed D, Storey C. Efficient hybrid conjugate gradient techniques [J]. Journal of Optimization Theory and Applications, 1990, 64: 379-397.
[11] Gilbert J C, Nocedal J. Global convergence properties of conjugate gradient methods for optimization [J]. SIAM Journal on Optimization, 1992, 2(1): 21-42.
[12] Dai Y H, Liao L Z. New conjugacy conditions and related nonlinear conjugate gradient methods [J]. Applied Mathematics and Optimization, 2001, 43: 87-101.
[13] Andrei N. Hybrid conjugate gradient algorithm for unconstrained optimization [J]. Journal of Optimization Theory and Applications, 2009, 141: 249-264.
[14] Andrei N. Another hybrid conjugate gradient algorithm for unconstrained optimization [J]. Numerical Algorithms, 2008, 47(2): 143-156.
[15] Yuan G G, Li T T, Hu W J. A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems [J]. Applied Numerical Mathematics, 2020, 147: 129-141.
[16] Wei Z X, Yu G H, Yuan G L, et al. The superlinear convergence of a modified BFGS-type method for unconstrained optimization [J]. Computational Optimization and Applications, 2004, 29: 315-332.
[17] 段侠彬.几种修正的共轭梯度算法[D].南宁：广西大学，2016.
[18] Geem Z W. Parameter estimation for the nonlinear Muskingum model using the BFGS technique [J]. Journal of Irrigation and Drainage Engineering, 2006, 132(5): 474-478.
[19] Ouyang A, Liu L B, Sheng Z, et al. A class of parameter estimation methods for nonlinear Muskingum model using hybrid invasive weed optimization algorithm [J]. Mathematical Problems in Engineering, 2015, 2015: 1-15.
[20] Yuan G G, Jian A L, Zhang M X, et al. A modified HZ conjugate gradient algorithm without gradient Lipschitz continuous condition for non convex functions [J]. Journal of Applied Mathematics and Computing, 2022, 68(6): 4691-4712.
北大中文核心期刊