Non productive EFAs detection over the Cypriotic pilot region (2021)

In the context of the EU-funded project DIONE (No. 870378), the Super-Resolved Sentinel-2 data were further used to identify small-sized areas that can be characterised of great importance in terms of environmental sustainability, which in the Common Agricultural Policy (CAP) are denoted as Ecological Focus Area (EFA). EFAs are characterised as a portion of farmland area that has to be designed for environmental purposes. For the needs of DIONE, the EFAs regions were identified with the Sentinel-2 & Very High Resolution (VHR) data to be acquired from 01-08-2020 until 25-06-2021 providing tailored information for the needs of the Cypriotic Paying Agency (e.g. CAPO).

Further Details of the provided EFAs dataset

The EFA polygons were downloaded from the EFA endpoints provided by CAPO.

There are three EFA types present in the dataset:

Points: Trees

Lines: Ditches

Polygons: Productive EFAs - The current layer will not be exploited in the content of the DIONE project, as it is predominately focused on the detection of *non-productive (stable)* EFA elements.

The points and lines had to be buffered to an extent of 5m so that the signals could be downloaded. Because the elements considered are very small and they would not fit into one full Sentinel-2 pixel, signals were downloaded from the 2.5m resolution output of the multi-temporal Super Resolution model. The observation period for EFAs detection is from 01-08-2020 until 25-06-2021. Results are provided for 52691 point polygons and 454 line polygons located in the test area.

Details for the model deployed

To classify EFAs an LSTM model is used. The model is trained as a binary classifier using:

Signals from EFA (buffered point and line geometries) FOIs inside the test area as positive examples

Signals from agricultural parcels inside the test area as negative examples

Five models are trained using a 5-fold split to ensure that the results are predicted on data the model has not seen.

Results

Overall performance of the models

Accuracy = 82.9

Precision 
Recall
  F1 
Count

EFA
84.9
92.9
88.7
53008

NOT EFA
74.9
56.4
64.3
19992

In reality, we are only interested in one side of the confusion matrix. We want to detect wrongly / fraudulently claimed EFAs - meaning FOIs that are claimed as EFAs and are in reality some kind of agricultural land. In these cases, 93% of the claimed EFAs were confirmed to be EFAs and for 7% of the EFA FOIs, the model disagrees.

This dataset is comprised of one geopackage file, the "efas_summary.gpkg", which was computed for the Cypriotic pilot region. Descriptions are given below.

classification: Whether the model thinks the FOI is an EFA (`EFA`) or not (`NOT_EFA`)

classification_score: The pseudoprobability of the EFA prediction as assigned by the crop-group mode

a score close to 1 indicates that the model is very confident in the prediction

a score close to 0.5 indicates that the model is not confident in the prediction