摘要
The state and input constraints of nonlinear systems could greatly impede the realization of their optimal control when using reinforcement learning (RL)-based approaches since the commonly used quadratic utility functions cannot meet the requirements of solving constrained optimization problems. This article develops a novel optimal control approach for constrained discrete-time (DT) nonlinear systems based on safe RL. Specifically, a barrier BF) is introduced and incorporated with the value function to help transform a constrained optimization problem into an unconstrained one. Meanwhile, the minimum of such an optimization problem can be guaranteed to occur at the origin. Then a constrained policy iteration (PI) algorithm is developed to realize the optimal control of the nonlinear system and to enable the state and input constraints to be satisfied. The constrained optimal control policy and its corresponding value function are derived through the implementation of two neural networks (NNs). Performance analysis shows that the proposed control approach still retains the convergence and optimality properties of the traditional PI algorithm. Simulation results of three examples reveal its effectiveness.
学科与关键词
- Reinforcement learning based adaptive optimal control for constrained nonlinear system via a novel state-dependent transformationYan, Lei; Liu, Zhi*; Zhang, Yun; Wu, Zongze; Chen, C. L. PhilipISA TRANSACTIONS, 2023, 133: 29-41.被引 4
- Reinforcement Learning-Based Optimal Stabilization for Unknown Nonlinear Systems Subject to Inputs With Uncertain ConstraintsZhao, Bo*; Liu, Derong; Luo, ChaominIEEE Transactions on Neural Networks and Learning Systems, 2020, 31(10): 4330-4340.被引 115
- Safe Reinforcement Learning-Based Robust Approximate Optimal Control for Hypersonic Flight VehiclesShi, Lei; Wang, Xuesong; Cheng, Yuhu*IEEE Transactions on Vehicular Technology, 2023, 72(9): 11401-11414.被引 7
