MER Opportunity and Spirit Rovers Pancam Images Labeled Data Set

Introduction The data set is based on 3,004 images collected by the Pancam instruments mounted on the Opportunity and Spirit rovers from NASA's Mars Exploration Rovers (MER) mission. We used rotation, skewing, and shearing augmentation methods to increase the total collection to 70,864 (see Image Augmentation section for more information). Based on the MER Data Catalog User Survey [1], we identified 25 classes of both scientific (e.g. soil trench, float rocks, etc.) and engineering (e.g. rover deck, Pancam calibration target, etc.) interests (see Classes section for more information). The 3,004 images were labeled on Zooniverse platform, and each image is allowed to be assigned with multiple labels. The images are either 512 x 512 or 1024 x 1024 pixels in size (see Image Sampling section for more information). Classes There is a total of 25 classes for this data set. See the list below for class names, counts, and percentages (the percentages are computed as count divided by 3,004). Note that the total counts don't sum up to 3,004 and the percentages don't sum up to 1.0 because each image may be assigned with more than one class. Class name, count, percentage of dataset Rover Deck, 222, 7.39% Pancam Calibration Target, 14, 0.47% Arm Hardware, 4, 0.13% Other Hardware, 116, 3.86% Rover Tracks, 301, 10.02% Soil Trench, 34, 1.13% RAT Brushed Target, 17, 0.57% RAT Hole, 30, 1.00% Rock Outcrop, 1915, 63.75% Float Rocks, 860, 28.63% Clasts, 1676, 55.79% Rocks (misc), 249, 8.29% Bright Soil, 122, 4.06% Dunes/Ripples, 1000, 33.29% Rock (Linear Features), 943, 31.39% Rock (Round Features), 219, 7.29% Soil, 2891, 96.24% Astronomy, 12, 0.40% Spherules, 868, 28.89% Distant Vista, 903, 30.23% Sky, 954, 31.76% Close-up Rock, 23, 0.77% Nearby Surface, 2006, 66.78% Rover Parts, 301, 10.02% Artifacts, 28, 0.93% Image Sampling Images in the MER rover Pancam archive are of sizes ranging from 64x64 to 1024x1024 pixels. The largest size, 1024x1024, was by far the most common size in the archive. For the deep learning dataset, we elected to sample only 1024x1024 and 512x512 images as the higher resolution would be beneficial to feature extraction. In order to ensure that the data set is representative of the total image archive of 4.3 million images, we elected to sample via "site code". Each Pancam image has a corresponding two-digit alphanumeric "site code" which is used to track location throughout its mission. Since each "site code" corresponds to a different general location, sampling a fixed proportion of images taken from each site ensure that the data set contained some images from each location. In this way, we could ensure that a model performing well on this dataset would generalize well to the unlabeled archive data as a whole. We randomly sampled 20% of the images at each site within the subset of Pancam data fitting all other image criteria, applying a floor function to non-whole number sample sizes, resulting in a dataset of 3,004 images. Train/validation/test sets split The 3,004 images were split into train, validation, and test data sets. The split was done so that roughly 60, 15, and 25 percent of the 3,004 images would end up as train, validation, and test data sets respectively, while ensuing that images from a given site are not split between train/validaiton/test data sets. This resulted in 1,806 train images, 456 validation images, and 742 test images. Augmentation To augment the images in train and validation data sets (note that images in the test data set were not augmented), three augmentation methods were chosen that best represent transformations that could be realistically seen in Pancam images. The three augmentations methods are rotation, skew, and shear. The augmentation methods were applied with random magnitude, followed by a random horizontal flipping, to create 30 augmented images for each image. Since each transformation is followed by a square crop in order to keep input shape consistent, we had to constrict the magnitude limits of each augmentation to avoid cropping out important features at the edges of input images. Thus, rotations were limited to 15 degrees in either direction, the 3-dimensional skew was limited to 45 degrees in any direction, and shearing was limited to 10 degrees in either direction. Note that augmentation was done only on training and validation images. Directory Contents images: contains all 70,864 images train-set-v1.1.0.txt: label file for the training data set val-set-v1.1.0.txt: label file for the validation data set test-set-v1.1.0.txt: label file for the testing data set Images with relatively short file names (e.g., 1p128287181mrd0000p2303l2m1.img.jpg) are original images, and images with long file names (e.g., 1p128287181mrd0000p2303l2m1.img.jpg_04140167-5781-49bd-a913-6d4d0a61dab1.jpg) are augmented images. The label files are formatted as "Image name, Class1, Class2, ..., ClassN". Reference [1] S.B. Cole, J.C. Aubele, B.A. Cohen, S.M. Milkovich, and S.A. Shields, Identifying Community Needs for a Mars Exploration Rovers (MER), Daata Catalog, 51st Lunar and Planetary Science Conference (LPSC), 2020.## 引言 本数据集基于美国国家航空航天局（NASA）火星探测漫游者（Mars Exploration Rovers, MER）任务中，机遇号与勇气号漫游车搭载的全景相机（Pancam）所采集的3004张原始图像构建。我们采用旋转、倾斜与剪切增强方法，将总样本量扩充至70864张（详细信息请参见「图像增强」章节）。基于MER数据目录用户调研[1]，我们确定了25类兼具科学与工程研究价值的目标：其中科学类包括土壤沟槽、漂浮岩石等，工程类包括漫游车甲板、全景相机校准靶标等（详细信息请参见「类别」章节）。该3004张原始图像在Zooniverse平台完成标注，每张图像可被赋予多个类别标签。图像尺寸分为512×512与1024×1024像素两种（详细信息请参见「图像采样」章节）。 ## 类别说明 本数据集共计25个类别，各类别详情、样本计数与占比如下（占比计算方式为该类别计数除以3004）。需注意，各类别计数总和与3004并不相等，占比总和也不为1.0，这是因为每张图像可被分配多个类别标签： 1. 漫游车甲板：222，7.39% 2. 全景相机校准靶标：14，0.47% 3. 机械臂硬件：4，0.13% 4. 其他硬件：116，3.86% 5. 漫游车车辙：301，10.02% 6. 土壤沟槽：34，1.13% 7. RAT打磨靶标：17，0.57% 8. RAT钻孔：30，1.00% 9. 岩石露头：1915，63.75% 10. 漂浮岩石：860，28.63% 11. 岩屑：1676，55.79% 12. 其他岩石：249，8.29% 13. 亮土壤：122，4.06% 14. 沙丘/波纹：1000，33.29% 15. 线性特征岩石：943，31.39% 16. 圆形特征岩石：219，7.29% 17. 土壤：2891，96.24% 18. 天文目标：12，0.40% 19. 球粒：868，28.89% 20. 远景：903，30.23% 21. 天空：954，31.76% 22. 特写岩石：23，0.77% 23. 近地表：2006，66.78% 24. 漫游车部件：301，10.02% 25. 人工制品：28，0.93% ## 图像采样 MER漫游车全景相机档案中的图像尺寸范围为64×64至1024×1024像素，其中1024×1024像素为档案中最常见的尺寸。针对本深度学习数据集，我们仅选取1024×1024与512×512像素的图像，因为更高分辨率的图像有助于特征提取。为确保本数据集能够代表总计430万张的全景相机档案图像整体分布，我们采用「站点代码」进行采样。每一张全景相机图像均对应一个两位字母数字混合的「站点代码」，用于追踪任务全程中的拍摄位置。由于每个站点代码对应不同的大致拍摄区域，从每个站点按固定比例采样可确保数据集覆盖所有拍摄位置的图像。通过该方式，我们可保证在本数据集上表现优异的模型能够良好泛化至未标注的全景相机档案整体数据。我们在符合所有其他图像筛选条件的全景相机数据子集中，对每个站点随机采样20%的图像，并对非整数采样量应用向下取整操作，最终得到包含3004张图像的数据集。 ## 训练/验证/测试集划分 我们将3004张图像划分为训练集、验证集与测试集。划分时需满足：训练集、验证集、测试集的占比分别约为60%、15%与25%，同时确保同一站点的图像不会被拆分至不同的数据集中。最终得到训练集图像1806张，验证集456张，测试集742张。 ## 图像增强 我们对训练集与验证集的图像进行增强（注：测试集图像未进行增强），选取了三种最能代表全景相机图像可能出现的真实变换的增强方法：旋转、倾斜与剪切。我们为每张原始图像生成30张增强图像，具体操作为：先应用随机幅度的上述三种变换，再进行随机水平翻转。由于每次变换后需进行方形裁剪以保持输入尺寸一致，我们需要限制每种增强的幅度上限，避免裁剪掉输入图像边缘的重要特征。因此，旋转角度限制为±15度，三维倾斜角度限制为±45度，剪切角度限制为±10度。需注意，增强操作仅应用于训练集与验证集的图像。 ## 目录结构 本数据集的目录结构如下： - `images/`：包含全部70864张图像 - `train-set-v1.1.0.txt`：训练集标签文件 - `val-set-v1.1.0.txt`：验证集标签文件 - `test-set-v1.1.0.txt`：测试集标签文件 文件名较短的图像（例如`1p128287181mrd0000p2303l2m1.img.jpg`）为原始图像；文件名较长的图像（例如`1p128287181mrd0000p2303l2m1.img.jpg_04140167-5781-49bd-a913-6d4d0a61dab1.jpg`）为增强图像。 标签文件的格式为："Image name, Class1, Class2, ..., ClassN" ## 参考文献 [1] S.B. Cole, J.C. Aubele, B.A. Cohen, S.M. Milkovich, S.A. Shields. 《火星探测漫游者（MER）数据目录的社区需求识别》，第51届月球与行星科学会议（LPSC），2020年。