Multi-scale labeled dataset for boulder localization, segmentation and pose estimation for navigation on small bodies

The capability to detect boulders on the surface of small bodies is beneficial for vision-based applications such as hazard detection during critical operations, safety quantification, autonomous planning of scientific operations, and autonomous navigation. This task, however, is challenging due to the wide assortment of irregular shapes, the characteristics of the boulders population, and the rapid variability in the illumination conditions. Moreover, the lack of publicly available labeled datasets damps the research about data-driven algorithms. The following dataset has been designed and made publicly available to tackle these challenges. Its purpose is twofold. First, from the lessons learned from previous datasets, to develop a multi-purpose, high-fidelity dataset with boulders scattered across the surface of a small body. Second, to exploit domain randomization, artificial noise addition, scaling, and post-processing, enabling the design of data-driven pipelines. 

The methodology used to generate the dataset is illustrated in the work "A multi-scale labeled dataset for boulder segmentation and navigation on small bodies" by Mattia Pugliatti and Michele Maestrini, presented at the 74th IAC (International Astronautical Congress), 2024, Baku, Azerbaijan.

The dataset contains the image-label pairs of 45514 samples, organized with the following structure: 

Dataset_PugliattiMaestrini_2023IAC

--img

--labels

--masks

The dataset is comprised of 45514 samples. The "img" folder contains the input, 512x 512 grayscale images. The "labels" folder includes the .txt segmentation labels of the 15 most prominent boulders for each image detected with the methodology illustrated in the IAC paper. The "masks" dataset contains the segmentation masks for all image layers, with the values being encoded between 0 and 17 as uint8. The samples are named as XXXXXX_YYY. XXXXXX stands for the image's original ID during rendering. YYY corresponds to the sub-splits of the original image obtained at rendering: 

001 - Top-Left crop

002 - Top-Right crop

003 - Bottom-Left crop

004 - Bottom-right crop

005 - Whole, resized

The "all_images_data.json" file contains one entry for each image, and for each entry the positions of the centroids of the 15 most prominent boulders are reported as follows:

-- image_name:

filename of the image.        

-- boulder_id:   

ID of the visible boulders in the image. Values are encoded in range [3, 17], following the encoding format of the masks.

-- true_centr_img: 

Image coordinates of the boulders' true centroids. 

-- bb_x0_y0_w_h:       

Image coordinates of the bounding box for each boulder. Each row contains the (x, y) coordinates of the top left corner of the bounding box, its width and height.

-- bb_centr_img:

Geometric center of the bounding box, in image coordinates.

-- mask_centr_img:       

Geometric centers of the pixels which correspond to each boulder in the masks, in image coordinates.

-- label_centr_img:       

Geometrical centers of the polygons defined by the segmentation label points, in image coordinates.

-- body_pos_km: 

3D coordinates of the true boulder centroids in an asteroid body-fixed frame. The origin is the center of the asteroid, and coordinates are in km.

-- centr_camframe_km: 

3D coordinates of the true boulder centroids in the camera space. The origin is the center of the image, the x axis points towards right, the y-axis points downward and the z-axis positive towards the asteroid.

-- label_polygon_img:

Set of (x, y) coordinates pairs representing the points of the segmentation labels' polygon, in image coordinates.

--body2cam_transl_km:

Translation vectors from the body-fixed frame to the camera frame, expressed in km.

-- body2cam_rot:

Rotation matrices from the body-fixed frame to the camera frame.

--cam2body_transl_km:

Translation vectors from the camera frame to the body-fixed frame, expressed in km.

-- cam2body_rot:

Rotation matrices from the camera frame to the body-fixed frame.

It is possible that centroids and bounding boxes are outside the image boundaries.

Some images may contain no major boulders. In this case the centroids data fields are left as [].

The parameters of the camera used for rendering are

focal length: 128.1 mm

sensor width: 36.0 mm

sensor height: 36.0 mm