MLGTT: An Open-Source Tool to Generate Camera-Relative Ground Truth for Monocular Localization

Abstract—Ground truth is essential in Computer Vision problemsas it establishes a baseline for error thresholds in practicalapplications. This paper presents the Monocular LocalizationGround Truth Tool (MLGTT), a modular ground truth seekingtool for camera pose calculations, aimed at obtaining accuratemetrics to verify the performance of localization algorithmswhile isolating intrinsic image error factors. Our tool leveragesa Perspective-n-Point (PnP) algorithm, and a Random SampleConsensus (RANSAC) iterative algorithm, which approximatesthe best fit model, represented by the highest number of inliersand the lowest reprojection error. To utilize this PnP algorithm,we require a predetermined set of 3D points selected on an object’sCAD model, 2D points from an image of the object that aremanually selected in the tool, and a camera model that includesdistortion parameters for handling unrectified images. Theseparameters yield a transformation matrix from the camera tothe object. The MLGTT reprojects these points onto the canvasusing the best transformation matrix from PnP. When pairedwith a rasterizer tool, the MLGTT can synthesize an imageusing the calculated transformation matrix that overlays on topof the original image. A slider tool is also implemented intothe MLGTT which provides the user with a better visual aidfor verifying localization fidelity, by blending the original imagewith the rendered image with a slider. This visual verificationhelps ensure the accuracy of the pose estimation to the level ofaccuracy of the human eye. The MLGTT allows human perceptionto aid in the findings of the ground truth, which removeserror found in other methods. We introduce our characterizationof user and pixel error, quantified through experimentswith individuals both familiar and unfamiliar with the MLGTTor computer vision generally. We will also discuss reprojectionerror on different screen resolutions and how they might affectthe findings. Additionally, we quantify the ground truth errorin free spaces as a function of the resolution and field of view(FOV) of the camera that captured the image. Findings includedemonstrating the tool’s precision and ease of use compared totraditional external ground truthing methods, such as Vicon orAprilTags. These traditional systems typically require externalmarkers and complex setups, whereas the MLGTT operateswithout such additional elements This makes the MLGTT moreversatile and easier to deploy in various environments, demonstratedby testing on Mars 2020 images. Originally designedto provide independent ground truth to verify our localizationalgorithms for the Mars Sample Return (MSR) campaign, theMLGTT has been adapted for multiple different applicationsand can be applied generally to any monocular imagery ofknown hardware with a calibrated camera. This software willbe released as an open-source project along with this paper.