Dataset for "SpecTf: Transformers Enable Data-Driven Imaging Spectroscopy Cloud Detection"

SpecTf: Transformers Enable Data-Driven Imaging Spectroscopy Cloud Detection J.H. Lee, M. Kiper, D.R. Thompson, P.G. BrodrickProc. Natl. Acad. Sci. U.S.A. 122 (27) e2502903122, https://doi.org/10.1073/pnas.2502903122 (2025).Code repository: https://github.com/emit-sds/SpecTf Summary This repository contains the dataset used to train and evaluate the Spectroscopic Transformer model for EMIT cloud screening. spectf_cloud_labelbox.hdf5 1,841,641 Labeled spectra from 221 EMIT Scenes. spectf_cloud_mmgis.hdf5 1,733,801 Labeled spectra from 313 EMIT Scenes. These scenes were specifically labeled to correct false detections by an earlier version of the model. spectf_cloud_mmgis_2.hdf5 3,606,820 Labeled spectra from 456 EMIT Scenes. These scenes were specifically labeled to correct false detections by an earlier version of the model. train_fids.csv 848 EMIT scenes comprising the training set. test_fids.csv 126 EMIT scenes comprising the held-out validation set. validation_scenes.pdf 126 validation scenes in RGB and Falsecolor, and cloud mask product comparisons. Please note that the ANN and GBT models in these results were not re-trained on the mmgis_2 dataset, as the SpecTf model was. v2 adds validation_scenes.pdf, a PDF displaying the 69 validation scenes in RGB and Falsecolor, their existing baseline cloud masks, as well as their cloud masks produced by the ANN and GBT reference models and the SpecTf model. v3 adds spectf_cloud_mmgis_2.hdf5, a third dataset used to train an improved version of the model. Train and test CSVs were updated to include scenes in this dataset, and validation_scenes.pdf was updated to show the results of the v3 model. The model trained on this dataset is released as v0.1.0 via the GitHub repository. Data Description 221 EMIT Scenes were initially selected for labeling with diversity in mind. After sparse segmentation labeling of confident regions in Labelbox, up to 10,000 spectra were selected per-class per-scene to form the spectf_cloud_labelbox dataset. We deployed a preliminary model trained on these spectra on all EMIT scenes observed in March 2024, then labeled another 313 EMIT Scenes using MMGIS's polygonal labeling tool to correct false positive and false negative detections. After similarly sampling spectra from these scenes, a total of 3,575,442 spectra were labeled and sampled. v3: We deployed the previous model trained on these spectral on all EMIT scenes observed during the first day of each month in 2024, then labeled another 456 EMIT scenes using MMGIS's polygonal labeling tool to correct false positive and false negative detections. After similarly sampling spectra from these scenes, an additional 3,606,820 spectra were labeled and sampled. The train/test split was randomly determined by scene FID to prevent the same EMIT scene from contributing spectra to both the training and validation datasets. Please refer to Section 4.2 in the paper for a complete description, and to our code repository for example usage and a Pytorch dataloader. Each hdf5 file contains the following arrays: 'spectra' Top-of-Atmosphere reflectance calculated from the EMIT L1B Radiance product Float64 of shape (n, 268) 'fids' The FID from which each spectrum was sampled Binary string of shape (n,) 'indices' The (col, row) index from which each spectrum was sampled Int64 of shape (n, 2) 'labels' Annotation label of each spectrum 0 - "Clear" 1 - "Cloud" 2 - "Cloud Shadow" (Only for the Labelbox dataset, and this class was combined with the clear class for this work. See paper for details.) label[label==2] = 0 Int64 of shape (n,2) Each hdf5 file contains the following attribute: 'bands' The band center wavelengths (nm) of the spectrum Float64 of shape (268,) Acknowledgements The EMIT online mapping tool was developed by the JPL MMGIS team. The High Performance Computing resources used in this investigation were provided by funding from the JPL Information and Technology Solutions Directorate. This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004). Government sponsorship acknowledged.