Operations Research Transactions >
2023 , Vol. 27 >Issue 4: 81 - 105
DOI: https://doi.org/10.15960/j.cnki.issn.1007-6093.2023.04.005
A survey of direct methods for optimal control problems
Received date: 2023-05-16
Online published: 2023-12-07
Optimal control is an important branch of control theory, and its goal is to determine a control strategy that optimizes system performance indicators under the premise of satisfying dynamic systems and constraints. Optimal control has a wide range of applications in engineering, economics, finance, robotics, aerospace and other fields. The direct method is a common method to solve the optimal control problem. This method transforms the continuous optimal control problem into a finite-dimensional optimization problem by directly discretizing the control and state function. At present, the direct method mainly includes the direct collocation method and the control parameterization method. The direct collocation method uses a specific function form to approximate the state and control function at the same time; the control parameterization method uses a linear combination of basis functions to approximate the control function, thereby discretizing the control space. The purpose of the two methods is to transform the continuous optimal control problem into a finite-dimensional nonlinear programming problem, and then choose an appropriate optimization algorithm to solve it. Benefiting from its flexibility and ability to deal with constraints, the direct method has become the important method in recent years for practical applications requiring real-time control. Researchers and engineers continue to develop and improve direct methods to increase their efficiency and accuracy in solving complex optimal control problems. This article mainly introduces the relevant achievements and latest developments of the direct method for readers' reference, and discusses the research trends and potential research directions of the direct method.
Mengzhen SHAO, Changjun YU . A survey of direct methods for optimal control problems[J]. Operations Research Transactions, 2023 , 27(4) : 81 -105 . DOI: 10.15960/j.cnki.issn.1007-6093.2023.04.005
| 1 | OluyisolaO E,BhallaS,SgarbossaF,et al.Designing and developing smart production planning and control systems in the industry 4.0 era: A methodology and case study[J].Journal of Intelligent Manufacturing,2022,33(1):311-332. |
| 2 | LiT J,XiaoY N.Optimal strategies for coordinating infection control and socio-economic activities[J].Mathematics and Computers in Simulation,2023,207,533-555. |
| 3 | ZhangT,LiC C,MaD Y,et al.An optimal task management and control scheme for military operations with dynamic game strategy[J].Journal of Process Control,2021,115,106815. |
| 4 | GaoY,WeiZ,ShaoZ,et al.Enhanced moving finite element method based on error geometric estimation for simultaneous trajectory optimization[J].Automatica,2023,147(2):110711. |
| 5 | MohammadiS,HejaziS H.Using particle swarm optimization and genetic algorithms for optimal control of non-linear fractional-order chaotic system of cancer cells[J].Mathematics and Computers in Simulation,2023,206,538-560. |
| 6 | BellmanR.The theory of dynamic programming[J].Bulletin of the American Mathematical Society,1954,60(6):503-515. |
| 7 | RitterA,WidmerF,DuhrP,et al.Long-term stochastic model predictive control for the energy management of hybrid electric vehicles using Pontryagin's minimum principle and scenario-based optimization[J].Applied Energy,2022,322,119192. |
| 8 | LuoB,WuH N,HuangT W.Reinforcement learning solution for HJB equation arising in constrained optimal control problem[J].Neural Networks,2015,71,150-158. |
| 9 | ChenL G,XiaS J.Maximizing power of irreversible multistage chemical engine with linear mass transfer law using HJB theory[J].Energy,2022,261,125277. |
| 10 | TrélatE.Optimal control and applications to aerospace: some results and challenges[J].Journal of Optimization Theory and Applications,2012,154,713-758. |
| 11 | GaoY,WeiZ Y,ShaoZ J.Enhanced moving finite element method based on error geometric estimation for simultaneous trajectory optimization[J].Automatica,2023,147,110711. |
| 12 | NasresfahaniF,EslahchiM R.Numerical solution of optimal control of atherosclerosis using direct and indirect methods with shooting/collocation approach[J].Computers and Mathematics with Applications,2022,126,60-76. |
| 13 | KirkD E.Optimal Control Theory: An Introduction[M].New York:Dover Pubns,2004. |
| 14 | KellyM.An introduction to trajectory optimization: How to do your own direct collocation[J].SIAM Review,2017,59(4):849-904. |
| 15 | ZhuG,JieH,HongW.Nonlinear model predictive path tracking control for autonomous vehicles based on orthogonal collocation method[J].International Journal of Control, Automation and Systems,2023,21(1):257-270. |
| 16 | TeoK L,LiB,YuC J,et al.Applied and Computational Optimal Control: A Control Parameterization Approach[M].Berlin:Springer International Publishing,2021. |
| 17 | ReddienG W.Collocation at Gauss points as a discretization in optimal control[J].SIAM Journal on Control and Optimization,1979,17(2):298-306. |
| 18 | HermanA L,ConwayB A.Direct optimization using collocation based on high-order Gauss-Lobatto quadrature rules[J].Journal of Guidance, Control, and Dynamics,1996,19(3):592-599. |
| 19 | CuthrellJ E,BieglerL T.On the optimization of differential-algebraic process systems[J].AIChE Journal,1987,33(8):1257-1270. |
| 20 | BettsJ T,HuffmanW P.Mesh refinement in direct transcription methods for optimal control[J].Optimal Control Applications and Methods,1998,19(1):1-21. |
| 21 | Zhao Y, Tsiotras P. Mesh refinement using density function for solving optimal control problems[C]//AIAA Infotech@ Aerospace Conference, 2009. |
| 22 | VlassenbroeckJ,Van DoorenR.A Chebyshev technique for solving nonlinear optimal control problems[J].IEEE Transactions on Automatic Control,1988,33(4):333-340. |
| 23 | Gong Q, Ross I M, Fahroo F. A Chebyshev pseudospectral method for nonlinear constrained optimal control problems[C]//Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference, 2009: 5057-5062. |
| 24 | GargD,PattersonM,HagerW W,et al.A unified framework for the numerical solution of optimal control problems using pseudospectral methods[J].Automatica,2010,46(11):1843-1851. |
| 25 | GargD,HagerW W,RaoA V.Pseudospectral methods for solving infinite-horizon optimal control problems[J].Automatica,2011,47(4):829-837. |
| 26 | DarbyC L,HagerW W,RaoA V.An hp-adaptive pseudospectral method for solving optimal control problems[J].Optimal Control Applications and Methods,2010,32(4):476-502. |
| 27 | DolejšíV,MayG.An anisotropic hp-mesh adaptation method for time-dependent problems based on interpolation error control[J].Journal of Scientific Computing,2023,95(2):36. |
| 28 | LiuG,LiB,JiY.A modified hp-adaptive pseudospectral method for multi-UAV formation reconfiguration[J].ISA Transactions,2022,129,217-229. |
| 29 | ElnagarG N,KazemiM A.Pseudospectral Legendre-based optimal computation of nonlinear constrained variational problems[J].Journal of Computational and Applied Mathematics,1998,88(2):363-375. |
| 30 | RossI M,FahrooF.Pseudospectral knotting methods for solving nonsmooth optimal control problems[J].Journal of Guidance, Control, and Dynamics,2004,27(3):397-405. |
| 31 | TabrizidoozH R,MarzbanH R,PourbabaeeM,et al.A composite pseudospectral method for optimal control problems with piecewise smooth solutions[J].Journal of the Franklin Institute,2017,354(5):2393-2414. |
| 32 | GohC J,TeoK L.Control parametrization: A unified approach to optimal control problems with general constraints[J].Automatica,1988,24(1):3-18. |
| 33 | LeeH W J,TeoK L,RehbockV,et al.Control parametrization enhancing technique for time optimal control problems[J].Dynamic Systems and Applications,1997,6,243-262. |
| 34 | LeeH W J,TeoK L,RehbockV,et al.Control parametrization enhancing technique for optimal discrete-valued control problems[J].Automatica,1999,35(8):1401-1407. |
| 35 | ZhuX,YuC,TeoK L.Sequential adaptive switching time optimization technique for optimal control problems[J].Automatica,2022,146,110565. |
| 36 | LiB,YuC J,TeoK L,et al.An exact penalty function method for continuous inequality constrained optimal control problem[J].Journal of Optimization Theory and Applications,2011,151,260-291. |
| 37 | YuC J,SuS X,BaiY Q.On the optimal control problems with characteristic time control constraints[J].Journal of Industrial and Management Optimization,2022,18(2):1305-1320. |
| 38 | AbuasbehK,MahmudovN I,AwadallaM.Relative controllability and Ulam-Hyers stability of the second-order linear time-delay systems[J].Mathematics,2023,11(4):806. |
| 39 | LiY,HanW,ShaoW,et al.Virtual sensing for dynamic industrial process based on localized linear dynamical system models with time-delay optimization[J].ISA Transactions,2023,133,505-517. |
| 40 | TeoK L,WongK H,ClementsD J.Optimal control computation for linear time-lag systems with linear terminal constraints[J].Journal of Optimization Theory and Applications,1984,44,509-526. |
| 41 | YuC J,LinQ,LoxtonR,et al.A hybrid time-scaling transformation for time-delay optimal control problems[J].Journal of Optimization Theory and Applications,2016,169,876-901. |
| 42 | WuD,BaiY,YuC.A new computational approach for optimal control problems with multiple time-delay[J].Automatica,2019,101,388-395. |
| 43 | HairerE,N?rsettS P,WannerG.Solving Ordinary Differential Equations I: Nonstiff problems[M].Berlin:Springer-Vlg,1993. |
| 44 | 李庆扬.数值分析[M].北京:清华大学出版社,2001. |
| 45 | DontchevA L,HagerW W,VeliovV M.Second-order Runge-Kutta approximations in control constrained optimal control[J].SIAM Journal on Numerical Analysis,2000,38(1):202-226. |
| 46 | HargravesC R,ParisS W.Direct trajectory optimization using nonlinear programming and collocation[J].Journal of Guidance, Control, and Dynamics,1987,10(4):338-342. |
| 47 | BüskensC,MaurerH.SQP-methods for solving optimal control problems with control and state constraints: adjoint variables, sensitivity analysis and real-time control[J].Journal of Computational and Applied Mathematics,2000,120(1-2):85-108. |
| 48 | DontchevA L,HagerW W,PooreA B,et al.Optimality, stability, and convergence in nonlinear control[J].Applied Mathematics and Optimization,1995,31(3):297-326. |
| 49 | HagerW W.Runge-Kutta methods in optimal control and the transformed adjoint system[J].Numerische Mathematik,2000,87,247-282. |
| 50 | BonnansJ F,Laurent-VarinJ.Computation of order conditions for symplectic partitioned Runge-Kutta schemes with application to optimal control[J].Numerische Mathematik,2006,103,1-10. |
| 51 | Garg D, Hager W, Rao A. Gauss pseudospectral method for solving infinite-horizon optimal control problems[C]//AIAA Guidance, Navigation, and Control Conference, 2002: 7890. |
| 52 | KameswaranS,BieglerL T.Convergence rates for direct transcription of optimal control problems using collocation at Radau points[J].Computational Optimization and Applications,1988,41,81-126. |
| 53 | Fahroo F, Ross I M. On discrete-time optimality conditions for pseudospectral methods[C]//AIAA/AAS Astrodynamics Specialist Conference and Exhibit, 2006: 6304. |
| 54 | ShenJ,TangT,WangL L.Spectral Methods: Algorithms, Analysis and Applications[M].Berlin:Springer,2011. |
| 55 | BensonD A,HuntingtonG T,ThorvaldsenT P,et al.Direct trajectory optimization and costate estimation via an orthogonal collocation method[J].Journal of Guidance, Control, and Dynamics,2006,29(6):1435-1440. |
| 56 | GargD,PattersonM A,FrancolinC,et al.Direct trajectory optimization and costate estimation of finite-horizon and infinite-horizon optimal control problems using a Radau pseudospectral method[J].Computational Optimization and Applications,2011,49,335-358. |
| 57 | YuC J,YuanL,SuS X.A new gradient computational formula for optimal control problems with time-delay[J].Journal of Industrial and Management Optimization,2022,18(4):2469-2482. |
| 58 | NocedalJ,WrightS.Numerical Optimization[M].New York:Springer,1999. |
| 59 | 袁亚湘.非线性优化计算方法[M].北京:科学出版社,2008. |
| 60 | LoxtonR C,TeoK L,RehbockV.Optimal control problems with multiple characteristic time points in the objective and constraints[J].Automatica,2008,44(11):2923-2929. |
| 61 | LiuC Y,LoxtonR,TeoK L,et al.Optimal state-delay control in nonlinear dynamic systems[J].Automatica,2022,135,109981. |
| 62 | TeoK L,JenningsL S.Nonlinear optimal control problems with continuous state inequality constraints[J].Journal of Optimization Theory and Applications,1989,63,1-22. |
| 63 | LoxtonR C,TeoK L,RehbockV,et al.Optimal control problems with a continuous inequality constraint on the state and the control[J].Automatica,2009,45,2250-2257. |
| 64 | YuC J,TeoK L,ZhangL,et al.A new exact penalty function method for continuous inequality constrained optimization problems[J].Journal of Industrial and Management Optimization,2010,6,895-910. |
| 65 | BettsJ T.Practical Methods for Optimal Control and Estimation Using Nonlinear Programming[M].Philadelphia:Society for Industrial and Applied Mathematics,2010. |
/
| 〈 |
|
〉 |
