Project HF‑EOLUS. Task 1. Sentinel‑1 SAR Derived Data Bundle (GeoParquet + STAC)

Overview

This Zenodo record packages a reproducible bundle of derived ocean wind data and auxiliary materials produced from Copernicus Sentinel‑1 SAR Level‑2 OCN (OWI) products. The bundle contains the processed dataset as a GeoParquet data lake with a static STAC catalog, together with the exact scripts, configuration, SQL statements, and inputs used to generate it. It is intended to enable full transparency, re‑use, and re‑execution of the data generation workflow.

The dataset was generated using the HF‑EOLUS Sentinel‑1 SAR Ingestion Pipeline (software record DOI: https://doi.org/10.5281/zenodo.17011823), which automates download and ingestion into partitioned GeoParquet, and builds a STAC Collection and Items describing the outputs. This record includes all relevant artifacts so that other researchers can verify the steps, inspect lineage, and rerun the pipeline if needed.

Dataset summary

- Final table rows: 11,384,420

- Rows with valid wind data: 7,992,086

Contents of this record

- hf_eolus_sar.tar.gz: Tarball containing the processed data outputs. It includes the partitioned GeoParquet dataset (OGC GeoParquet v1.1 metadata) and a static STAC catalog (Collection and Items with the STAC Table Extension and Processing metadata) describing the Parquet assets.

- pipeline.sh: The exact sequence of commands used to generate this dataset (download and ingestion), serving as an executable provenance log for reproducibility.

- stac_properties_collection.json: STAC property definitions/templates applied at Collection level during catalog generation.

- stac_properties_item.json: STAC property definitions/templates applied at Item level during catalog generation.

- scripts/*.sql (e.g., scripts/athena_create_table.sql): SQL statements used for registering the resulting Parquet dataset as an external table (e.g., in AWS Athena) and for validating schema/partitions.

- scripts/downloaded_files.txt: Manifest listing the Sentinel‑1 OCN product identifiers that were downloaded and used as inputs.

- scripts/VILA_PRIO_hull.json: Area of Interest (AOI) polygon used to constrain the search and download of Sentinel‑1 scenes spatially. This file defines a convex hull bounding the intersection between the areas covering the echos of VILA and PRIO stations.

- scripts/files_to_download.csv: Input list and/or search results for Sentinel‑1 OCN products targeted by the download stage (includes product IDs and acquisition metadata).

Reproducibility and re‑execution

- Software pipeline: HF‑EOLUS Sentinel‑1 SAR Ingestion Pipeline (https://doi.org/10.5281/zenodo.17011823). The pipeline uses Dockerized R and Python environments for deterministic runs (no manual dependency setup required).

- How to re‑run: Review pipeline.sh to see the exact commands, arguments, and environment variables used. If desired, clone the referenced pipeline repository, ensure Docker is available, and re‑execute the same steps. The AOI and time range used are captured in scripts/area_boundary.geojson and scripts/files_to_download.csv; the precise upstream inputs are listed in scripts/downloaded_files.txt.

- Optional cloud registration: The SQL files in scripts/ can be used to register the resulting Parquet dataset in AWS Athena (or adapted for other engines like Trino/Spark). This step is optional and not required to read the Parquet files directly with tools such as Python (pyarrow/GeoPandas), R (arrow), or DuckDB.

- Copernicus credentials: To re‑download Sentinel‑1 OCN data, provide your own Copernicus Data Space Ecosystem account credentials. Create a plain‑text file (e.g., `credentials`) with one line containing your key and pass its path via `--credentials-file` to `scripts/download_sar.sh`. For account and API access information, see https://dataspace.copernicus.eu and the documentation at https://documentation.dataspace.copernicus.eu/

Data format and standards

- GeoParquet: Columnar, compressed Parquet files with OGC GeoParquet v1.1 geospatial metadata (geometry column, CRS, encoding). Files are partitioned to support efficient filtering and scalable analytics.

- STAC: A static STAC Collection with Items describes each Parquet asset, including schema via the STAC Table Extension and lineage via Processing metadata. The catalog is suitable for static hosting and is interoperable with common STAC tooling.

Provenance and upstream data

- Upstream source: Copernicus Sentinel‑1 SAR Level‑2 OCN (OWI) products provided by the European Space Agency (ESA) under the Copernicus Programme. The list of specific input products for this dataset is included in scripts/downloaded_files.txt.

- Processing: All derivations (extraction of ocean wind variables, conversion to GeoParquet, and STAC catalog creation) were performed by the HF‑EOLUS pipeline referenced above. The sequence is captured verbatim in pipeline.sh.

- Credit line: Contains modified Copernicus Sentinel‑1 data; we gratefully acknowledge the Copernicus Programme and the European Space Agency (ESA) for providing free and open Sentinel‑1 data.

How to cite

- This data bundle:

Herrera Cortijo, J. L., Fernández‑Baladrón, A., Rosón, G., Gil Coto, M., Dubert, J., & Varela Benvenuto, R. (2025). Project HF‑EOLUS. Task 2. Sentinel‑1 SAR Derived Data Bundle (GeoParquet + STAC) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.17007304

- Software/pipeline used

Herrera Cortijo, J. L., Fernández‑Baladrón, A., Rosón, G., Gil Coto, M., Dubert, J., & Varela Benvenuto, R. (2025). HF‑EOLUS Sentinel‑1 SAR Ingestion Pipeline (GeoParquet + STAC) (v0.1.2). Zenodo. https://doi.org/10.5281/zenodo.17011788

Usage notes

- Local analysis: The GeoParquet files can be opened directly with Python (pyarrow, pandas/GeoPandas), R (arrow), or SQL engines (DuckDB/Trino) without additional ingestion steps.

- Catalog discovery: The STAC catalog in the tarball is static and can be browsed with STAC tools or published on object storage or a simple web server.

- AWS/Athena setup (optional): To use the GeoParquet in AWS, upload the dataset to Amazon S3 and adjust the SQL to your paths and names, then execute in Athena:

1) Upload the GeoParquet (and optionally the STAC catalog) to `s3://<your-bucket>/<your-prefix>/` preserving the folder structure.

2) Edit `scripts/athena_create_table.sql` to set the `LOCATION` to your S3 path and customize the database and table (e.g., change `SAR_INGEST.SAR` to `MY_DB.MY_TABLE`).

3) In Athena, run the SQL to create the database (if needed) and the external table.

4) Load partitions with `MSCK REPAIR TABLE MY_DB.MY_TABLE;` (or add partitions explicitly) and validate with a quick query such as `SELECT COUNT(*) FROM MY_DB.MY_TABLE;`.

Acknowledgements

This work has been funded by the HF‑EOLUS project (TED2021‑129551B‑I00), financed by MICIU/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR - BDNS 598843 - Component 17 - Investment I3. Members of the Marine Research Centre (CIM) of the University of Vigo have participated in the development. We gratefully acknowledge the Copernicus Programme and the European Space Agency (ESA) for providing free and open Sentinel‑1 data used in this work (contains modified Copernicus Sentinel data).

Disclaimer

This content is provided "as is", without warranties of any kind. Users are responsible for verifying fitness for purpose and for complying with the licensing and attribution requirements of upstream Copernicus Sentinel data.