EuroCropsML

EuroCropsML* is a ready-to-use ML dataset combining EuroCrops reference data with Sentinel-2 reflectance data from 2021. It contains data from Latvia, Portugal, and Estonia and is intended for benchmarking few-shot crop type classification. We used Eurostat's GISCO dataset to map the EuroCrops parcels to their NUTS1-3 region.

The provided data comes in two stages:

raw_data.zip (stage 1): One dataframe per country containing a annual time series of observations for each parcel, as well as separate files for the parcels' geometries and classes (EC_hcat_c = 10-digit HCAT code indicating the hierarchy of the crop).

preprocess.zip (stage 2): Read-to-use .npz-files. Each data point is saved in an .npz-file along with its metadata (parcel's centroid in [lon,lan]; observation dates). In addition, we performed some cloud removal steps. Each .npz-file is saved with the following naming convention: __.npz

Furthermore, split.zip contains .json-files that split the files from preprocess.zip into a pre-training/meta-learning (train and validation) and fine-tuning (train, validation, and test) dataset. In total, we provide two use cases:

latvia_vs_estonia: pre-training on Latvia (103 distinct classes) and fine-tuning on Estonia (127 distinct classes, of which 46 have not been seen during pre-training)

latvia_portugal_vs_estonia: pre-training on Latvia and Portugal (142 distinct classes), fine-tuning on Estonia (127 distinct classes, of which 34 have not been seen during pre-training)

overlap_latvia_vs_estonia: pre-training on overlapping classes between Latvia and Estonia (81 distinct classes) and fine-tuning on Estonia (127 distinct classes, of which 46 have not been seen during pre-training)

overlap_latvia_portugal_vs_estonia: pre-training on overlapping classes between Latvia and Estonia as well as Portugal and Estonia (93 distinct classes in total), fine-tuning on Estonia (127 distinct classes, of which 34 have not been seen during pre-training)

For all cases, the fine-tuning split tays consistent and is as follows:

train: 1-, 5-, 10-, 20-, 100-, 200-, 500-shot (for few-shot classification and benchmarking) and all samples

validation: 1000 samples

test: all samples

Changelog

Version 11:

Added new splits: overlap_latvia_vs_estona and overlap_latvia_portugal_vs_estonia.

Rectified file Portugal.parquet that has been inadvertently corrupted in version 10.

Version 10: 

Added back in missing raw_data files 

Raw data geometries _geometries.geojson and labels _labels.geojson are now sourced from the shapefiles and, hence, contain all parcels, even if no Sentinel-2 is available.

Version 9: 

Replacement of some parcel IDs for Latvia and Estonia. These parcels were duplicates within the original source shapefile. Hence, the replacement does not affect the data itself, solely the parcel IDs are replaced. The following IDs are affected:

country
old parcel ID
new parcel ID

Estonia
20548567
22172347

Estonia
21313556
22111331

Latvia
12786929
13203478

Latvia
12297424
12804307

Latvia
12297361
12804296

Latvia
12297423
12803325

Latvia
12297421
12803323

Latvia
12297422
12803324

In the split files, for Latvia only ID 13203478 is affected. The remaining IDs are not part of the splits since they belong to the meadow class, which is downsampled for the pre-training splits.

This version is missing the .parquet files and cannot be used to run pre-processing.

Version 8: Adjustment of Portugal finetuning split such that it matches the Latvia finetuning split

Version 7: Added new few-shot fine-tuning splits: 200 and 500

Version 6: Added new (few-shot) fine-tuning splits: 20, 100, and all samples

Version 4: The EuroCrops shapefiles sometimes contain a couple of parcels that lie outside the national borders. We now map them to the closest NUTS region within the country. Please rely on this version or newer.

Version 3: Some parcels have been clipped incorrectly. 

Version 2: Remove datapoints that contain only cloudy observations (in preprocess.zip).

Version 1: Initial publication

* Contains Copernicus Sentinel data (2024), processed on EOLab

Country-secific data sources for EuroCrops reference data

Estonia:

INSPIRE GEOPORTAL

If link does not work, search for Estonia --> Geospatial Aid Application Estonia Agricultural parcels on the INSPIRE platform.

Latvia:

Lauku atbalsta dienests Updated Source

Portugal:

Download via WFS https://www.ifap.pt/isip/ows/isip.data/wfs or over the IFAP website.