Non productive EFAs detection over the Lithuanian pilot region (2021)

In the context of the EU-funded project DIONE (No. 870378), the Super-Resolved Sentinel-2 data were further used to identify small-sized areas that can be characterised of great importance in terms of environmental sustainability, which in the Common Agricultural Policy (CAP) are denoted as Ecological Focus Area (EFA). EFAs are characterised as a portion of farmland area that has to be designed for environmental purposes. For the needs of DIONE, the EFAs regions were identified with the Sentinel-2 & Very High Resolution (VHR) data to be acquired from 01-08-2020 until 31-05-2021 providing tailored information for the needs of the Lithuanian Paying Agency (e.g. NPA).

Further Details of the provided EFAs dataset

There are only three EFA types present in the dataset given by NPA:

mg0: Tree group, forest

gr0: Ditch, the canal from 1 m wide

ku0: Pond with coastal vegetation

Due to the nature of the data, the majority of EFA polygons do not contain at least one full Sentinel-2 pixel. To circumvent this issue, the 2.5m resolution output of the multi-temporal SR model is used to produce EFA signals. Using 2.5m resolution, around 20% of FOIs belonging to the "Ditch, canal from 1m wide" class do not contain one full super-resolved pixel at 2.5m resolution. Out of "34157" EFA polygons in the sample area, we have signals for "32732" of them. The majority are missing due to pixelization (~1000 EFA FOIs fully contain less than 1 SR pixel), others are missing due to having no cloud-free observation during the observation period.

Details for the model deployed

To classify EFAs an LSTM model is used (same as for crop type classification). The model is trained as a binary classifier using:

LSTM FOIs inside the test area as positive examples

Agricultural parcels inside the test area as negative examples

Five models are trained using a 5-fold split to ensure that the results are predicted on data the model has not seen.

Results

Overall performance of the models

Accuracy = 91.1

Precision 
Recall
  F1 
Count

EFA
92.3 
93.2
92.8
32732

NOT EFA
89.2
87.9
88.6
21005

In reality, we are only interested to detect wrongly / fraudulently claimed EFAs - meaning FOIs that are claimed as EFAs and are in reality some kind of agricultural land. The model is less confident in predicting EFAs as EFAs for the "Ditch/Canal" group. This is probably due to the fact that these polygons are long and narrow and the most susceptible to registration errors and shifts which we know are present in the SR data from which the signals were produced.

This dataset is comprised of one geopackage file, the "efas-w-results-v0-binary.gpkg", which was computed for the Lithuanian pilot region. Descriptions are given below.

classification: Whether the model thinks the FOI is an EFA (`EFA`) or not (`NOT_EFA`)

classification_score: the pseudoprobability of the EFA prediction as assigned by the crop-group model

  a score close to 1 indicates that the model is very confident in the prediction

  a score close to 0 indicates that the model is not confident in the prediction