Reinforcement Learning-Based Hybrid Force/Position Control of Redundant Manipulators under Time Delays

Hybrid force/position control of redundant robotic manipulators under time delays, disturbances, and model uncertainties remains a significant challenge. Conventional methods rely on separating the force and position control spaces and on precise system identification, which increases complexity and reduces robustness. This paper introduces a semi-model-free framework, the Force/Position Reinforcement Learning Super-Twisting Algorithm (F/P-RL-STA), which avoids explicit space separation and reduces dependency on accurate models. By integrating a Super-Twisting Sliding Mode Controller (STSMC) with Deep Deterministic Policy Gradient (DDPG) reinforcement learning, the proposed method enables simultaneous force and position regulation. The reinforcement learning agent adaptively tunes control parameters and null-space torques, while ensuring smooth transitions between free motion and contact interaction. This adaptive mechanism reduces control effort, suppresses chattering, and enhances robustness against disturbances and time delays. Closed-loop stability is guaranteed through Lyapunov analysis. Sensitivity analysis is conducted to guide the tuning of learning parameters and reward design. Simulation results confirm that the proposed approach significantly outperforms conventional methods in terms of accuracy, energy efficiency, smoothness of control signals, and chattering elimination, even in the presence of disturbances and time delays.An agent reinforcement learning architecture is introduced to switch smoothly between position control during free motion and hybrid force/position control during contact tasks. The kinematic redundancy of the OpenManipulator-X robot is exploited using a learned null-space torque to minimize control effort and suppress chattering. A comprehensive sensitivity analysis is performed to guide optimal tuning of learning parameters and the reward function, enhancing training stability and controller robustness. Simulation results confirm that F/P-RL-STA significantly outperforms both SMC-HFPC and F/P-STA in terms of tracking precision, control smoothness, disturbance rejection, and energy efficiency, while maintaining closed-loop stability as verified through Lyapunov-based analysis.