"PID Control of Hydraulic Robotic Arm Based on Reinforcement Learning"

"To address the challenge of achieving high-precision control for multi-degree-of-freedom heavy-duty hydraulic robotic arms, this paper proposes a reinforcement learning-based Proportional-Integral-Derivative (PID) self-tuning method for a hydraulic robotic arm characterized by a simple structure and low computational load. The method combines an actor-critic neural network (ANN-CNN) reinforcement learning framework with PID control to dynamically adjust the PID gains for high-precision tracking. The critic neural network (CNN) estimates the cumulative value of position error after PID processing, evaluating the advantages and disadvantages of the control methods. Based on the accumulated error signals, the actor neural network (ANN), integrated with the CNN, learns and adaptively adjusts the PID gain in real time. This adaptive mechanism effectively compensates for strong nonlinearities, unknown perturbations, and multi-joint coupling in the electromechanical-liquid coupling system, thereby enhancing overall tracking accuracy. Both the ANN and CNN employ a gradient descent algorithm to update neural network weights online, ensuring fast convergence while avoiding local optima. Finally, the stability of the proposed control method is analytically proven using the Lyapunov stabilization method, and its effectiveness is further validated through experiments on a three-degree-of-freedom heavy-duty hydraulic robotic arm platform."