Match-up Database (MDB) files for the validation of satellite water products from Sentinel-3 OLCI and Sentinel-2 MSI using the initial samples of HYPERNETS in situ data

This dataset provides the Match-up Database (MDB) files used in the validation of Sentinel-3 OLCI Level-2 Water Full Resolution (WFR); Sentinel-2 MSI ACOLITE/DSF and Sentinel-2 MSI Case 2 Regional Coast Colour (C2RCC) satellite water products using reference (in situ) radiometry data acquired between January 2021 and March 2023 at six water sites during the first deployment of the HYPSTAR® (HYperspectral Pointable System for Terrestrial and Aquatic Radiometry) hyperspectral radiometers. Results of this multi-site validation analysis based on the MDB-approach are presented and discussed in Gonzalez Vilas et al. (2024).

The HYPSTAR® was developed in the framework of the European Union’s HORIZON 2020 project HYPERNETS (Ruddick et al., 2024). This radiometer provides high-quality above-water hyperspectral radiometry data in the visible to near infrared (VNIR) from 380 to 1020 nm with a spectral resolution of 5 nm (or better), and in the short-wavelength infrared (SWIR) from 1000 to 1700 nm with a spectral resolution of 10 nm (Goyens et al, 2022; De Vis et al.,2024). The MDB files in this dataset are based on the  HYPSTAR® publicly available initial samples acquired at six water sites: Aqua Alta Oceanographic Tower in Venice (AAOT), Italy (VEIT) (Brando and González Vilas, 2023); Garda Lake, Italy (GAIT) (Brando et al., 2023); Berre, France (BEFR) (Doxaran and Corizzi, 2023a); Magest station – GIRonde estuary (MAGIR), France (MAFR) (Doxaran and Corizzi, 2023b); Rio de la Plata (RdP-EsNM), Argentina (LPAR) (Dogliotti et al., 2023); and Zeebrugge, Belgium (M1BE) (Goyens and Gammaru, 2023).

In the framework of HYPERNETS project, an open-source MDB file structure built on the open-source NetCDF-4 (Network Common Data Form) file format was developed. MDB files are designed to include all the potential match-ups between satellite and reference (in situ) remote-sensing reflectance (Rrs) data and to be used as input for the implementation of validation analysis.  Variables in these files are grouped in four data categories: satellite (satellite prefix); in situ (insitu prefix); match-ups (mu prefix); and flag (flag prefix). Files also include metadata (global attributes) to fully characterize and identify the files, including satellite and in situ sensor, latitude and longitude of the site, creation date, etc. (Gonzalez Vilas et al., 2024).

A set of Python tools was also developed to work with these MDB files and perform validation analysis of satellite water products using HYPSTAR® data as reference (Gonzalez Vilas et al., 2024).  Tools are organized in three modules: SAT_EXTRACT, MDB_builder and MDB_reader, which are included in the hypernets_val repository available from the HYPERNETS project GitHub (https://github.com/HYPERNETS/hypernets_val).

The OLCI WFR product is operationally processed by EUMETSAT for both Sentinel-3A (from April 2016 to present) and 3B (from May 2018 to present) missions using the OLCI L2 processor IPF-OL-2 version 07 (EUMETSAT, 2021; Zibordi et al., 2022).  OLCI WFR provides Rrs for 16 bands (all the OLCI bands except those dedicated to atmospheric measurements) between 400 nm and 1020 nn at 300 m resolution, and with a revisit time of one day combining both missions. It also includes a pixel classification band, i.e., Water Quality and Science Flags (WQSF), which is imported into the MDB file. Data were available from the EUMETSAT Ocean Colour baseline collection OL_L2M.003.

Sentinel-2 Level 1B images available from ESA (revisit time varies between 3 and 5 days combining both missions) were processed to produce Level-2 Rrs products at 20 m resolution using two atmospheric correction processors: C2RCC (Brockmann et al. 2016, Doerffer et al. 2007) and ACOLITE/DSF (Vanhellemont, 2019). ACOLITE provides remote sensing reflectance data for 11 bands between 442 nm and 2200 nm at 20 m spatial resolution. The Version 2 of C2RCC (https://c2rcc.org/neural-nets/) provides Rrs data for 8 MSI bands between 442 nm and 865 nm at 20 m spatial resolution. In addition to the atmospheric correction, the pixels are also classified to flag non-water and/or cloud contaminated pixels. Therefore, the IdePix software (v2.2.10, algorithm update 8.0.3), available as a SNAP processor, is used. How the data has been processed is detailed in Van der Zande et al. (2022). This document also provides a description of both AC algorithms, and their respective assumptions.

The reference data from HYPSTAR® are processed with the “hypernets processor” (https://github.com/HYPERNETS/hypernets_processor) and assumed to be quality-assured Level 2 Rrs data (with site specific quality checks, but without extrapolation of the viewing and illumination geometry). For most sites the NIR similarity correction (Ruddick et al., 2006) has been applied to correct the spectra for remaining glint. However, for some sites the theory of the NIR Similarity correction is not valid, in particular in very turbid waters, explaining why the Rrs data without the NIR similarity spectrum correction is used. Both Rrs variables (insitu_Rrs and insitu_Rrs_nosc) are delivered by the “hypernets processor” and are available in the MDB files. It is at the discretion of the end-user to select the appropriate final product. The HYPSTAR® Level-2 files also provide flags (imported into the MDB file as insitu_quality_flag) that are useful for checking the validity of the Rrs spectra.

This dataset provides the 28 MDB files used in the validation analysis presented in Gonzalez Vilas et al. (2024), including:

- 6 MDB files including the potential match-ups between Sentinel-3A OLCI WFR and HYPSTAR® data from 6 WATERHYPERNET sites (VEIT, GAIT, BEFR, MAFR, LPAR and M1BE).

- 6 MDB files including the potential match-ups between Sentinel-3B OLCI WFR and HYPSTAR® data from 6 WATERHYPERNET sites (VEIT, GAIT, BEFR, MAFR, LPAR and M1BE).

- 4 MDB files including the potential match-ups between Sentinel-2A MSI C2RCC and HYPSTAR® data from 4 WATERHYPERNET sites (VEIT, BEFR, MAFR and LPAR).

- 4 MDB files including the potential match-ups between Sentinel-2B MSI C2RCC and HYPSTAR® data from 4 WATERHYPERNET sites (VEIT, BEFR, MAFR and LPAR).

- 4 MDB files including the potential match-ups between Sentinel-2A MSI ACOLITE and HYPSTAR® data from 4 WATERHYPERNET sites (VEIT, BEFR, MAFR and LPAR).

- 4 MDB files including the potential match-ups between Sentinel-2B MSI ACOLITE and HYPSTAR® data from 4 WATERHYPERNET sites (VEIT, BEFR, MAFR and LPAR).

These files could be used to replicate the validation analysis presented in Gonzalez Vilas et al. (2024) following the tutorial available at https://github.com/HYPERNETS/hypernets_val/.