"Adaptive RISE Control of Hydraulic Manipulators Using Actor-Critic Architecture"

"Aiming at the difficulty of high-precision motion control of multi-degree-of-freedom (DOF) heavy-duty hydraulic manipulator, an adaptive robust integral of the sign of the error (RISE) controller based on Actor-Critic is developed to achieve high-performance progressive tracking of the system under various unmodeled errors and unknown disturbance. In this article, the proposed design consists of two frameworks: A RISE control method provides a closed-loop system stability framework; A reinforcement learning (RL) with Actor-Critic idea is introduced into the RISE control architecture to improve the system tracking accuracy. Specifically, two multilayer neural network (NN) estimators based on Actor-Critic architecture are designed to deal with uncertain coupled mechanical dynamics and nonlinear hydraulic dynamics, respectively. Specifically, the performance evaluation function is constructed by using the critic NN (CNN) to estimate the cumulative tracking error of the system. The minimum excitation signal of the cumulative error of CNN feedback is integrated in the actor NN (ANN) and the weight update law is learned based on the gradient descent method. Feedforward compensation is performed on the unknown dynamics to reduce the high feedback gain. At the same time, RISE control is used to deal with the reconstruction error and interference of the NN to ensure the asymptotic stability of the system. Finally, an experimental study is carried out based on a 3-DOF heavy-duty hydraulic manipulator to verify the effectiveness of the proposed controller."

针对多自由度（multi-degree-of-freedom, DOF）重载液压机械臂的高精度运动控制难题，本文设计了一种基于演员-评论家（Actor-Critic）的自适应鲁棒误差符号积分（RISE）控制器，以实现系统在各类未建模误差与未知扰动下的高性能渐进跟踪控制。

本文所提出的设计包含两大框架：其一，RISE控制方法为闭环系统提供稳定性保障框架；其二，融合演员-评论家思想的强化学习（reinforcement learning, RL）被引入RISE控制架构，以提升系统跟踪精度。

具体而言，本文设计了两个基于演员-评论家架构的多层神经网络（multilayer neural network, NN）估计器，分别用于处理不确定耦合机械动力学与非线性液压动力学问题。

具体地，通过评论家神经网络（critic NN, CNN）构建性能评价函数，以估计系统的累积跟踪误差。将CNN反馈的累积误差最小激励信号集成至演员神经网络（actor NN, ANN）中，并基于梯度下降法学习权重更新法则。通过对未知动力学进行前馈补偿，以降低高反馈增益；同时利用RISE控制处理神经网络的重构误差与干扰，保障系统的渐近稳定性。

最后，本文基于一台三自由度（3-DOF）重载液压机械臂开展实验研究，验证了所提控制器的有效性。