Evaluating Self-Supervised Redundancy-Reduction Representations for Intra-Site and Inter-Site Tropical Tree Species Classification from Airborne Hyperspectral Imagery : Dataset

# Hyperspectral Spectral-Pixel Datasets for Tropical Tree Species Classification This repository accompanies the paper "Evaluating Self-Supervised Redundancy-Reduction Representations for Intra-Site and Inter-Site Tropical Tree Species Classification from Airborne Hyperspectral Imagery" and provides the spectral-pixel data used to train and evaluate the models. All data are stored as Python `.pickle` files. Each spectrum has a spectral dimension of 343 bands (Ndim = 343), with reflectance values scaled by 10,000. Two types of data are provided: supervised datasets for classification and a self-supervised dataset of spectral pairs. ## 1. Supervised datasets: `GroundTruth/` Labeled spectral pixels from manually delineated tree crowns over two research sites in French Guiana, Paracou and Nouragues. Only liana-free crowns matched to their trunk with the highest confidence are retained. Two datasets are provided, with disjoint sets of species. In both, training and test partitions are formed of distinct tree crowns, so that no crown contributes to both. ### 1.1 Inter-site dataset: `GroundTruth/Inter_Site/` Fourteen species represented at both sites. Training crowns come from Paracou and test crowns from Nouragues. | File | Role | Trees ||---|---|---|| `InterSite_Paracou.pickle` | training (Paracou) | 954 || `InterSite_Nouragues.pickle` | test (Nouragues) | 148 | **Species composition** (number of spectra / number of trees; Table III of the paper): Paracou : 1 : Vochysiaceae_Qualea_rosea NTree : 244 | Total Spectra : 357452 : Fabaceae_Eperua_falcata NTree : 176 | Total Spectra : 211503 : Fabaceae_Dicorynia_guianensis NTree : 145 | Total Spectra : 262894 : Fabaceae_Vouacapoua_americana NTree : 59 | Total Spectra : 60995 : Lecythidaceae_Eschweilera_coriacea NTree : 38 | Total Spectra : 57996 : Goupiaceae_Goupia_glabra NTree : 66 | Total Spectra : 167757 : Lauraceae_Sextonia_rubra NTree : 51 | Total Spectra : 119158 : Malvaceae_Sterculia_pruriens NTree : 32 | Total Spectra : 51409 : Chrysobalanaceae_Licania_membranacea NTree : 38 | Total Spectra : 363510 : Myristicaceae_Virola_michelii NTree : 32 | Total Spectra : 296211 : Sapotaceae_Manilkara_bidentata NTree : 26 | Total Spectra : 444012 : Meliaceae_Carapa_surinamensis NTree : 26 | Total Spectra : 189613 : Lecythidaceae_Lecythis_poiteaui NTree : 14 | Total Spectra : 220714 : Lecythidaceae_Lecythis_persistens NTree : 7 | Total Spectra : 603 Nouragues : 1 : Vochysiaceae_Qualea_rosea NTree : 18 | Total Spectra : 51062 : Fabaceae_Eperua_falcata NTree : 5 | Total Spectra : 4203 : Fabaceae_Dicorynia_guianensis NTree : 37 | Total Spectra : 155364 : Fabaceae_Vouacapoua_americana NTree : 20 | Total Spectra : 30795 : Lecythidaceae_Eschweilera_coriacea NTree : 14 | Total Spectra : 11506 : Goupiaceae_Goupia_glabra NTree : 5 | Total Spectra : 12187 : Lauraceae_Sextonia_rubra NTree : 8 | Total Spectra : 15958 : Malvaceae_Sterculia_pruriens NTree : 8 | Total Spectra : 13579 : Chrysobalanaceae_Licania_membranacea NTree : 4 | Total Spectra : 100610 : Myristicaceae_Virola_michelii NTree : 11 | Total Spectra : 102811 : Sapotaceae_Manilkara_bidentata NTree : 6 | Total Spectra : 141612 : Meliaceae_Carapa_surinamensis NTree : 3 | Total Spectra : 27413 : Lecythidaceae_Lecythis_poiteaui NTree : 6 | Total Spectra : 56514 : Lecythidaceae_Lecythis_persistens NTree : 3 | Total Spectra : 257 ### 1.2 Intra-site dataset: `GroundTruth/Intra_Site/` Fourteen other species from Paracou, disjoint from those of the inter-site dataset, each represented by at least 34 crowns. For each species, the crowns are split into training (about 80%) and test (about 20%) sets, with at least ten test crowns per species. The labels of these species were used neither for encoder pretraining, nor for checkpoint selection, nor for any hyperparameter choice. | File | Role | Trees | Spectra ||---|---|---|---|| `Paracou_InterSite_Train.pickle` | training | 844 | 124,908 || `Paracou_InterSite_Test.pickle` | test | 240 | 33,789 | **Species composition** (Table IV of the paper; NTree: total number of trees,Nsample: number of spectral pixels): 1 : Sapotaceae_Pradosia_cochlearia — NTree : 255 — Train NTree : 204 — Test NTree : 51 — Train Nsample : 61300 — Test Nsample : 13183 2 : Lecythidaceae_Eschweilera_sagotiana — NTree : 184 — Train NTree : 147 — Test NTree : 37 — Train Nsample : 10840 — Test Nsample : 3289 3 : Fabaceae_Eperua_grandiflora — NTree : 113 — Train NTree : 90 — Test NTree : 23 — Train Nsample : 10709 — Test Nsample : 3100 4 : Fabaceae_Recordoxylon_speciosum — NTree : 97 — Train NTree : 77 — Test NTree : 20 — Train Nsample : 8847 — Test Nsample : 2575 5 : Fabaceae_Tachigali_melinonii — NTree : 67 — Train NTree : 53 — Test NTree : 14 — Train Nsample : 7373 — Test Nsample : 1521 6 : Chrysobalanaceae_Licania_alba — NTree : 62 — Train NTree : 49 — Test NTree : 13 — Train Nsample : 3313 — Test Nsample : 1045 7 : Lecythidaceae_Couratari_multiflora — NTree : 57 — Train NTree : 45 — Test NTree : 12 — Train Nsample : 4712 — Test Nsample : 1282 8 : Dichapetalaceae_Tapura_capitulifera — NTree : 39 — Train NTree : 29 — Test NTree : 10 — Train Nsample : 1755 — Test Nsample : 644 9 : Chrysobalanaceae_Hymenopus_heteromorphus — NTree : 36 — Train NTree : 26 — Test NTree : 10 — Train Nsample : 1145 — Test Nsample : 368 10 : Fabaceae_Bocoa_prouacensis — NTree : 36 — Train NTree : 26 — Test NTree : 10 — Train Nsample : 1667 — Test Nsample : 880 11 : Bignoniaceae_Jacaranda_copaia — NTree : 35 — Train NTree : 25 — Test NTree : 10 — Train Nsample : 1539 — Test Nsample : 1177 12 : Clusiaceae_Symphonia_globulifera — NTree : 35 — Train NTree : 25 — Test NTree : 10 — Train Nsample : 5140 — Test Nsample : 1696 13 : Chrysobalanaceae_Licania_ovalifolia — NTree : 34 — Train NTree : 24 — Test NTree : 10 — Train Nsample : 3804 — Test Nsample : 1456 14 : Clusiaceae_Moronobea_coccinea — NTree : 34 — Train NTree : 24 — Test NTree : 10 — Train Nsample : 2764 — Test Nsample : 1573 ### File content Each `.pickle` stores a dictionary with the following keys: | Key | Type | Shape / content ||---|---|---|| `x` | tensor | Nsample × Ndim: the spectral pixels (Ndim = 343), reflectance × 10,000 || `y` | tensor | Nsample: integer label assigned to each spectral pixel || `species` | list | (label, species name) pairs mapping each integer label in `y` to its species || `fid` | tensor | Nsample: identifier of the crown each spectral pixel belongs to (when provided) | ## 2. Self-supervised dataset: `SSL_Datasets/` Pairs of spectral acquisitions built with several pairing strategies. Each strategy has its own subfolder, and every subfolder is split into several subsets ("shards") to allow lighter and more dynamic loading. Each shard is a `.pickle` storing a dictionary with: | Key | Type | Shape / content ||---|---|---|| `x` | tensor | Nsample × 2·Ndim: the two spectra of each pair, stacked along the spectral axis || `Dim_Spectra` | int | spectral dimension of a single spectrum (343), used to split `x` into its two halves | To recover the two views of a pair from `x`: import pickle with open(shard_path, "rb") as f: data = pickle.load(f) d = data["Dim_Spectra"] # 343 x1 = data["x"][:, :d] # first view x2 = data["x"][:, d:] # second (positive) view Each pairing strategy contains 10 shards of 100,000 samples, for a total of 1,000,000spectral pairs per strategy. ## Scope of the released data The **supervised datasets** (`GroundTruth/`) are released in full: they are exactly the data used for the classification results reported in the paper, for both the inter-site and the intra-site regimes. The **self-supervised dataset** (`SSL_Datasets/`) is a **subsample** of the pairs used for pretraining, released to keep the archive at a size suitable for distribution. Each strategy is provided as 10 shards of 100,000 pairs (1,000,000 pairs per strategy), whereas the full training sets ranged from 1.2 to 12.7 million pairs depending on the strategy (see Table V of the paper). The subsample preserves the construction logic of each strategy and is intended to allow the pipeline to be reproduced and inspected; retraining on it will not reproduce the reported accuracies exactly. ## Version history - **v2**: the intra-site dataset (`GroundTruth/Intra_Site/`) now contains fourteen Paracou species disjoint from the inter-site species, as used in the revised version of the paper. It replaces the intra-site dataset of v1, which shared its species with the inter-site dataset.- **v1**: initial release. ## Citation If you use these data, please cite the accompanying paper and this archive.