最优控制问题的直接法综述

doi:10.15960/j.cnki.issn.1007-6093.2023.04.005

摘要/Abstract

摘要：

最优控制是控制理论的一个重要分支, 其目标是确定一种控制策略, 在满足动态系统和约束条件的前提下, 最优化系统性能指标。最优控制在工程、经济学、金融学、机器人技术、航空航天等各个领域都有着广泛的应用。直接法是解决最优控制问题的一类常用方法, 该方法通过直接离散化控制和状态函数, 从而将连续的最优控制问题转化为有限维优化问题。当前, 直接法主要包括直接配点法和控制参数化方法。直接配点法利用特定函数形式同时近似状态和控制函数, 控制参数化方法则使用基函数的线性组合来近似控制函数, 从而使控制空间离散化。两种方法的目的均为将连续的最优控制问题转化为有限维的非线性规划问题, 进而选择合适的优化算法求解。得益于其灵活性和处理约束的能力, 近年来直接法成为实际应用中需要实时控制的重要方法。本文主要介绍直接法的相关成果与最新进展供读者参考, 并讨论直接法的研究趋势和潜在研究方向。

关键词: 最优控制, 直接法, 直接配点法, 控制参数化方法

Abstract:

Optimal control is an important branch of control theory, and its goal is to determine a control strategy that optimizes system performance indicators under the premise of satisfying dynamic systems and constraints. Optimal control has a wide range of applications in engineering, economics, finance, robotics, aerospace and other fields. The direct method is a common method to solve the optimal control problem. This method transforms the continuous optimal control problem into a finite-dimensional optimization problem by directly discretizing the control and state function. At present, the direct method mainly includes the direct collocation method and the control parameterization method. The direct collocation method uses a specific function form to approximate the state and control function at the same time; the control parameterization method uses a linear combination of basis functions to approximate the control function, thereby discretizing the control space. The purpose of the two methods is to transform the continuous optimal control problem into a finite-dimensional nonlinear programming problem, and then choose an appropriate optimization algorithm to solve it. Benefiting from its flexibility and ability to deal with constraints, the direct method has become the important method in recent years for practical applications requiring real-time control. Researchers and engineers continue to develop and improve direct methods to increase their efficiency and accuracy in solving complex optimal control problems. This article mainly introduces the relevant achievements and latest developments of the direct method for readers' reference, and discusses the research trends and potential research directions of the direct method.

Key words: optimal control, direct method, direct collocation method, control parameterization method

中图分类号:

O221.2

邵梦真, 余长君. 最优控制问题的直接法综述[J]. 运筹学学报, 2023, 27(4): 81-105.

Mengzhen SHAO, Changjun YU. A survey of direct methods for optimal control problems[J]. Operations Research Transactions, 2023, 27(4): 81-105.

图/表 6

图1

图2

图3

图4

图5

图6

参考文献 65

1	OluyisolaO E,BhallaS,SgarbossaF,et al.Designing and developing smart production planning and control systems in the industry 4.0 era: A methodology and case study[J].Journal of Intelligent Manufacturing,2022,33(1):311-332. doi: 10.1007/s10845-021-01808-w
2	LiT J,XiaoY N.Optimal strategies for coordinating infection control and socio-economic activities[J].Mathematics and Computers in Simulation,2023,207,533-555. doi: 10.1016/j.matcom.2023.01.017
3	ZhangT,LiC C,MaD Y,et al.An optimal task management and control scheme for military operations with dynamic game strategy[J].Journal of Process Control,2021,115,106815.
4	GaoY,WeiZ,ShaoZ,et al.Enhanced moving finite element method based on error geometric estimation for simultaneous trajectory optimization[J].Automatica,2023,147(2):110711.
5	MohammadiS,HejaziS H.Using particle swarm optimization and genetic algorithms for optimal control of non-linear fractional-order chaotic system of cancer cells[J].Mathematics and Computers in Simulation,2023,206,538-560. doi: 10.1016/j.matcom.2022.11.023
6	BellmanR.The theory of dynamic programming[J].Bulletin of the American Mathematical Society,1954,60(6):503-515. doi: 10.1090/S0002-9904-1954-09848-8
7	RitterA,WidmerF,DuhrP,et al.Long-term stochastic model predictive control for the energy management of hybrid electric vehicles using Pontryagin's minimum principle and scenario-based optimization[J].Applied Energy,2022,322,119192. doi: 10.1016/j.apenergy.2022.119192
8	LuoB,WuH N,HuangT W.Reinforcement learning solution for HJB equation arising in constrained optimal control problem[J].Neural Networks,2015,71,150-158. doi: 10.1016/j.neunet.2015.08.007
9	ChenL G,XiaS J.Maximizing power of irreversible multistage chemical engine with linear mass transfer law using HJB theory[J].Energy,2022,261,125277. doi: 10.1016/j.energy.2022.125277
10	TrélatE.Optimal control and applications to aerospace: some results and challenges[J].Journal of Optimization Theory and Applications,2012,154,713-758. doi: 10.1007/s10957-012-0050-5
11	GaoY,WeiZ Y,ShaoZ J.Enhanced moving finite element method based on error geometric estimation for simultaneous trajectory optimization[J].Automatica,2023,147,110711. doi: 10.1016/j.automatica.2022.110711
12	NasresfahaniF,EslahchiM R.Numerical solution of optimal control of atherosclerosis using direct and indirect methods with shooting/collocation approach[J].Computers and Mathematics with Applications,2022,126,60-76. doi: 10.1016/j.camwa.2022.08.047
13	KirkD E.Optimal Control Theory: An Introduction[M].New York:Dover Pubns,2004.
14	KellyM.An introduction to trajectory optimization: How to do your own direct collocation[J].SIAM Review,2017,59(4):849-904. doi: 10.1137/16M1062569
15	ZhuG,JieH,HongW.Nonlinear model predictive path tracking control for autonomous vehicles based on orthogonal collocation method[J].International Journal of Control, Automation and Systems,2023,21(1):257-270. doi: 10.1007/s12555-021-0812-7
16	TeoK L,LiB,YuC J,et al.Applied and Computational Optimal Control: A Control Parameterization Approach[M].Berlin:Springer International Publishing,2021.
17	ReddienG W.Collocation at Gauss points as a discretization in optimal control[J].SIAM Journal on Control and Optimization,1979,17(2):298-306. doi: 10.1137/0317023
18	HermanA L,ConwayB A.Direct optimization using collocation based on high-order Gauss-Lobatto quadrature rules[J].Journal of Guidance, Control, and Dynamics,1996,19(3):592-599. doi: 10.2514/3.21662
19	CuthrellJ E,BieglerL T.On the optimization of differential-algebraic process systems[J].AIChE Journal,1987,33(8):1257-1270. doi: 10.1002/aic.690330804
20	BettsJ T,HuffmanW P.Mesh refinement in direct transcription methods for optimal control[J].Optimal Control Applications and Methods,1998,19(1):1-21. doi: 10.1002/(SICI)1099-1514(199801/02)19:1<1::AID-OCA616>3.0.CO;2-Q
21	Zhao Y, Tsiotras P. Mesh refinement using density function for solving optimal control problems[C]//AIAA Infotech@ Aerospace Conference, 2009.
22	VlassenbroeckJ,Van DoorenR.A Chebyshev technique for solving nonlinear optimal control problems[J].IEEE Transactions on Automatic Control,1988,33(4):333-340. doi: 10.1109/9.192187
23	Gong Q, Ross I M, Fahroo F. A Chebyshev pseudospectral method for nonlinear constrained optimal control problems[C]//Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, 2009: 5057-5062.
24	GargD,PattersonM,HagerW W,et al.A unified framework for the numerical solution of optimal control problems using pseudospectral methods[J].Automatica,2010,46(11):1843-1851. doi: 10.1016/j.automatica.2010.06.048
25	GargD,HagerW W,RaoA V.Pseudospectral methods for solving infinite-horizon optimal control problems[J].Automatica,2011,47(4):829-837. doi: 10.1016/j.automatica.2011.01.085
26	DarbyC L,HagerW W,RaoA V.An hp-adaptive pseudospectral method for solving optimal control problems[J].Optimal Control Applications and Methods,2010,32(4):476-502.
27	DolejšíV,MayG.An anisotropic hp-mesh adaptation method for time-dependent problems based on interpolation error control[J].Journal of Scientific Computing,2023,95(2):36. doi: 10.1007/s10915-023-02153-1
28	LiuG,LiB,JiY.A modified hp-adaptive pseudospectral method for multi-UAV formation reconfiguration[J].ISA Transactions,2022,129,217-229. doi: 10.1016/j.isatra.2022.01.015
29	ElnagarG N,KazemiM A.Pseudospectral Legendre-based optimal computation of nonlinear constrained variational problems[J].Journal of Computational and Applied Mathematics,1998,88(2):363-375. doi: 10.1016/S0377-0427(97)00225-2
30	RossI M,FahrooF.Pseudospectral knotting methods for solving nonsmooth optimal control problems[J].Journal of Guidance, Control, and Dynamics,2004,27(3):397-405. doi: 10.2514/1.3426
31	TabrizidoozH R,MarzbanH R,PourbabaeeM,et al.A composite pseudospectral method for optimal control problems with piecewise smooth solutions[J].Journal of the Franklin Institute,2017,354(5):2393-2414. doi: 10.1016/j.jfranklin.2017.01.002
32	GohC J,TeoK L.Control parametrization: A unified approach to optimal control problems with general constraints[J].Automatica,1988,24(1):3-18. doi: 10.1016/0005-1098(88)90003-9
33	LeeH W J,TeoK L,RehbockV,et al.Control parametrization enhancing technique for time optimal control problems[J].Dynamic Systems and Applications,1997,6,243-262.
34	LeeH W J,TeoK L,RehbockV,et al.Control parametrization enhancing technique for optimal discrete-valued control problems[J].Automatica,1999,35(8):1401-1407. doi: 10.1016/S0005-1098(99)00050-3
35	ZhuX,YuC,TeoK L.Sequential adaptive switching time optimization technique for optimal control problems[J].Automatica,2022,146,110565. doi: 10.1016/j.automatica.2022.110565
36	LiB,YuC J,TeoK L,et al.An exact penalty function method for continuous inequality constrained optimal control problem[J].Journal of Optimization Theory and Applications,2011,151,260-291. doi: 10.1007/s10957-011-9904-5
37	YuC J,SuS X,BaiY Q.On the optimal control problems with characteristic time control constraints[J].Journal of Industrial and Management Optimization,2022,18(2):1305-1320. doi: 10.3934/jimo.2021021
38	AbuasbehK,MahmudovN I,AwadallaM.Relative controllability and Ulam-Hyers stability of the second-order linear time-delay systems[J].Mathematics,2023,11(4):806. doi: 10.3390/math11040806
39	LiY,HanW,ShaoW,et al.Virtual sensing for dynamic industrial process based on localized linear dynamical system models with time-delay optimization[J].ISA Transactions,2023,133,505-517. doi: 10.1016/j.isatra.2022.06.034
40	TeoK L,WongK H,ClementsD J.Optimal control computation for linear time-lag systems with linear terminal constraints[J].Journal of Optimization Theory and Applications,1984,44,509-526. doi: 10.1007/BF00935465
41	YuC J,LinQ,LoxtonR,et al.A hybrid time-scaling transformation for time-delay optimal control problems[J].Journal of Optimization Theory and Applications,2016,169,876-901. doi: 10.1007/s10957-015-0783-z
42	WuD,BaiY,YuC.A new computational approach for optimal control problems with multiple time-delay[J].Automatica,2019,101,388-395. doi: 10.1016/j.automatica.2018.12.036
43	HairerE,NørsettS P,WannerG.Solving Ordinary Differential Equations I: Nonstiff problems[M].Berlin:Springer-Vlg,1993.
44	李庆扬.数值分析[M].北京:清华大学出版社,2001.
45	DontchevA L,HagerW W,VeliovV M.Second-order Runge-Kutta approximations in control constrained optimal control[J].SIAM Journal on Numerical Analysis,2000,38(1):202-226. doi: 10.1137/S0036142999351765
46	HargravesC R,ParisS W.Direct trajectory optimization using nonlinear programming and collocation[J].Journal of Guidance, Control, and Dynamics,1987,10(4):338-342. doi: 10.2514/3.20223
47	BüskensC,MaurerH.SQP-methods for solving optimal control problems with control and state constraints: adjoint variables, sensitivity analysis and real-time control[J].Journal of Computational and Applied Mathematics,2000,120(1-2):85-108. doi: 10.1016/S0377-0427(00)00305-8
48	DontchevA L,HagerW W,PooreA B,et al.Optimality, stability, and convergence in nonlinear control[J].Applied Mathematics and Optimization,1995,31(3):297-326. doi: 10.1007/BF01215994
49	HagerW W.Runge-Kutta methods in optimal control and the transformed adjoint system[J].Numerische Mathematik,2000,87,247-282. doi: 10.1007/s002110000178
50	BonnansJ F,Laurent-VarinJ.Computation of order conditions for symplectic partitioned Runge-Kutta schemes with application to optimal control[J].Numerische Mathematik,2006,103,1-10. doi: 10.1007/s00211-005-0661-y
51	Garg D, Hager W, Rao A. Gauss pseudospectral method for solving infinite-horizon optimal control problems[C]//AIAA Guidance, Navigation, and Control Conference, 2002: 7890.
52	KameswaranS,BieglerL T.Convergence rates for direct transcription of optimal control problems using collocation at Radau points[J].Computational Optimization and Applications,1988,41,81-126.
53	Fahroo F, Ross I M. On discrete-time optimality conditions for pseudospectral methods[C]//AIAA/AAS Astrodynamics Specialist Conference and Exhibit, 2006: 6304.
54	ShenJ,TangT,WangL L.Spectral Methods: Algorithms, Analysis and Applications[M].Berlin:Springer,2011.
55	BensonD A,HuntingtonG T,ThorvaldsenT P,et al.Direct trajectory optimization and costate estimation via an orthogonal collocation method[J].Journal of Guidance, Control, and Dynamics,2006,29(6):1435-1440. doi: 10.2514/1.20478
56	GargD,PattersonM A,FrancolinC,et al.Direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using a Radau pseudospectral method[J].Computational Optimization and Applications,2011,49,335-358. doi: 10.1007/s10589-009-9291-0
57	YuC J,YuanL,SuS X.A new gradient computational formula for optimal control problems with time-delay[J].Journal of Industrial and Management Optimization,2022,18(4):2469-2482. doi: 10.3934/jimo.2021076
58	NocedalJ,WrightS.Numerical Optimization[M].New York:Springer,1999.
59	袁亚湘.非线性优化计算方法[M].北京:科学出版社,2008.
60	LoxtonR C,TeoK L,RehbockV.Optimal control problems with multiple characteristic time points in the objective and constraints[J].Automatica,2008,44(11):2923-2929. doi: 10.1016/j.automatica.2008.04.011
61	LiuC Y,LoxtonR,TeoK L,et al.Optimal state-delay control in nonlinear dynamic systems[J].Automatica,2022,135,109981. doi: 10.1016/j.automatica.2021.109981
62	TeoK L,JenningsL S.Nonlinear optimal control problems with continuous state inequality constraints[J].Journal of Optimization Theory and Applications,1989,63,1-22. doi: 10.1007/BF00940727
63	LoxtonR C,TeoK L,RehbockV,et al.Optimal control problems with a continuous inequality constraint on the state and the control[J].Automatica,2009,45,2250-2257. doi: 10.1016/j.automatica.2009.05.029
64	YuC J,TeoK L,ZhangL,et al.A new exact penalty function method for continuous inequality constrained optimization problems[J].Journal of Industrial and Management Optimization,2010,6,895-910. doi: 10.3934/jimo.2010.6.895
65	BettsJ T.Practical Methods for Optimal Control and Estimation Using Nonlinear Programming[M].Philadelphia:Society for Industrial and Applied Mathematics,2010.

[1]	徐瑾涛, 邢文训. SIR类型新型冠状病毒肺炎多阶段最优控制模型[J]. 运筹学学报, 2023, 27(1): 43-52.
[2]	唐蓓蕾, 唐应辉. 两类具有N-策略和单重休假的M/G/1排队系统的最优控制策略[J]. 运筹学学报, 2021, 25(4): 15-30.
[3]	钟瑶, 唐应辉. 具有两类失效模式的D-策略M/G/1可修排队系统分析[J]. 运筹学学报, 2020, 24(1): 40-56.
[4]	潘取玉, 唐应辉, 兰绍军. 带负顾客和N-策略的Geo^{lambda_1, lambda_2/Geo/1(MWV)排队系统分析及最优控制策略N*[J]. 运筹学学报, 2017, 21(3): 65-76.