Vision-based single-stage grasp pose estimator with rotated anchors and automatic label generation

Vision-based grasp pose estimation is important and fraught with difficulty in robotics, especially when dealing with household objects of arbitrary shapes and positions. In this paper, we propose a novel single-stage rotated-anchor-based grasp detection method. We adopt a simple feature pyramid network and attach the predefined dense rotated anchors to fit the diverse grasps. To enhance the model’s generalization capabilities, we propose an automatic grasp area labeling strategy that simulates the full grasp attempt process. Our labeling strategy simultaneously considers grasp verification, collision detection, and the robot’s working space limitation. Thus, the reliability and accuracy of ground truth can be improved compared with previous computer-aided labeling that samples grasp points and then does force analysis or hand-only virtual grasp. For the objects, we scanned 98 complex-shaped ones to obtain their 3D models for the simulator rather than using synthetic objects, which can bring a smaller domain gap. We provide our full-process grasp simulation (Fsim) generated dataset for further research. Our detection algorithm achieves state-of-the-art accuracy and efficiency on the Cornell Grasp Dataset, the Jacquard dataset, and the Fsim dataset. Real grasp experiments further demonstrate the superiority of the Fsim dataset and the proposed model.