Robust Tube Localization for Mars Sample Return: Lightweight YOLO-Segmentation with Angle-Guided PnP

One considered approach in the planned Mars Sample Return (MSR) campaign involves accurately identifying and retrieving sample tubes from the Martian surface. This paper presents an innovative approach that utilises lightweight computer-vision techniques to enhance the efficiency and accuracy of the Sample Transfer Arm (STA) aboard the MSR lander. Our methodology employs the YOLOv8 deep learning model for image segmentation, and centroid detection of tubes in the challenging dusty Martian environment. These detected masks and centroids provide the foundation for constructing an outlined representation of the tubes, which is critical for precise spatial orientation. We exploit the knowledge of the object geometry to find key points and match them using their relative positions with respect to the geometry. Subsequently, a Perspective-n-Point (PnP) algorithm with RANSAC utilizes this outline and pre-computed 3D coordinates to ascertain the tube’s pose. This enables the STA’s camera-equipped gripper to locate and retrieve the samples accurately. This process is meticulously tailored for the constrained computational resources available on Martian missions, addressing limitations in processing speed and lack of parallelization capabilities. Extensive simulations under Martian-like conditions demonstrate the robustness and reliability of our approach, which would be a necessary technology to enable a backup tube retrieval concept for a MSR campaign using a robotic arm by ensuring precise and efficient sample collection. This method can achieve sub-degree and sub-centimeter accuracy with a single image.