Hierarchical dynamic movement primitive for the smooth movement of robots based on deep reinforcement learning

作者：Yuan, Yinlong; Yu, Zhu Liang^*; Hua, Liang; Cheng, Yun; Li, Junhong; Sang, Xiaohu

来源：Applied Intelligence, 2023, 53(2): 1417-1434.

DOI：10.1007/s10489-022-03219-7

• Deep reinforcement learning
• Robotics
• Dynamic movement primitives
• Meta-parameters
• Sawyer robot

Although deep reinforcement learning (DRL) algorithms with experience replay have been used to solve many sequential learning problems, applications of DRL in real-world robotics still face some serious challenges, such as the problem of smooth movement. A robot's motion trajectory needs to be smoothly coded, with no sudden acceleration or jerk. In this paper, a novel hierarchical reinforcement learning control framework named the hierarchical dynamic movement primitive (HDMP) framework is proposed to achieve the smooth movement of robots. In contrast to traditional algorithms, the HDMP framework consists of two learning hierarchies: a lower-level controller learning hierarchy and an upper-level policy learning hierarchy. In the lower-level controller learning hierarchy, modified dynamic movement primitives (DMPs) are utilized to generate a smooth motion trajectory. In the upper-level policy learning hierarchy, an improved local proximal policy optimization (L-PPO) method is proposed to endow the robot with autonomous learning capabilities. The performance achieved with the HDMP algorithm has been evaluated in a classical reaching movement task based on a Sawyer robot. The experimental results demonstrate that the proposed HDMP algorithm can endow a robot with the ability to smoothly execute motor skills and learn autonomously.

• 单位
  Nantong University