HF-EOLUS. Task 3. MeteoGalicia Wind Interpolation Outputs

MeteoGalicia Interpolation Case Study This record packages the reference outputs for the HF-EOLUS Task 3 interpolation experiment over the Vilano HF-radar footprint (open Atlantic, NW Iberia). The dataset was produced with the hf-eolus-interpolation-toolkit (Herrera Cortijo et al., 2025) by ingesting MeteoGalicia's historical WRF_HIST d03 (4 km) winds, refining the grid by a factor of eight, fitting kriging/IDW models and exporting the resulting GeoParquet layers together with metadata, plots and buoy comparisons. All counts and metrics below were recomputed with duckdb/duckdb:latest (Docker) over the committed assets in March 2025 so the README reflects the current state of the case study.Due to file size and number-of-files restrictions, the dataset has not been uploaded as a single zipped folder. Please copy all the files into a folder and decompress them, then move catalog.json and the contents of the buoy file into a catalogs folder to reproduce the original dataset layout. Dataset highlights Field Value Spatial domain AOI polygon stored in area_boundary.geojson, centred on the PdE Vilano buoy and matching the CONAS HF-radar footprint. Temporal coverage (GeoParquet assets) 2018-01-01 00:00 UTC → 2019-02-01 23:00 UTC (9,506 hourly partitions under local_sync/year=*/month=*/day=*/hour=*/data.parquet (in the toolkit root folder) and mirrored to case_study/catalogs/meteogalicia_interpolation/assets/parquet). Temporal coverage published in STAC 2018-01-01 00:00 UTC → 2018-08-18 20:00 UTC plus 2019-01-01 00:00 UTC → 2019-02-01 23:00 UTC (6,263 hourly items in case_study/catalogs/meteogalicia_interpolation/items/parquet). The missing 3,243 items (2018-08-19 → 2018-12-31) already exist in assets/ and can be appended by re-running the STAC builder with --stac-incremental. MeteoGalicia inputs WRF_HIST d03 (4 km deterministic mesh, 10 m winds) downloaded from the THREDDS service. Interpolation setup 8× refinement grid, 20 km variogram cutoff, 1 km variogram width, kriging model chosen per hour between Gaussian/Exponential/Spherical with IDW fallback when RSR deteriorates. Total wind vectors 652,929,116 records (u, v, wind_speed, wind_dir, diagnostics). Unique grid nodes 68,686 nodes spanning the refined mesh; stored as node_id. Reference hold-out test_points.csv contains the PdE Vilano buoy (node_id = "Vilano_buoy", lon -9.21, lat 43.5) which is excluded from model fitting and used for reports. Buoy comparison window 2018-05-01 00:00 UTC → 2019-02-01 23:00 UTC (3,379 matched samples against the PdE catalog in case_study/catalogs/pde_vilano_buoy). Toolkit release hf-eolus-interpolation-toolkit v0.1.0 (Zenodo https://doi.org/10.5281/zenodo.17598353). Storage and catalog layout area_boundary.geojson – Polygon delimiting the AOI given to the ingestion workflow. test_points.csv – CSV with validation points; currently only the Vilano buoy. catalogs/catalog.json – STAC root pointing to the MeteoGalicia collection and the PdE buoy catalog (never modify pde_vilano_buoy). catalogs/meteogalicia_interpolation/collection.json – Collection metadata (extent, providers, citation). Contents: assets/parquet – Hourly GeoParquet partitions organised as year=YYYY/month=MM/day=DD/hour=HH/data.parquet. assets/metadata – JSON sidecars with CRS, provenance, variogram parameters and AOI footprints. assets/plots – Diagnostic PNGs (quadrant grid + empirical variograms) matching each hour. items/parquet, items/metadata, items/plots – STAC Items per asset type; each hourly ID lives inside items/<type>/<timestamp>/<timestamp>.json. reports/buoys/vilano – CSV metrics, Markdown summary and PNG figures for the PdE comparison. stac_overrides/collection_override.json and stac_overrides/item_override.json – JSON fragments injected into the published collection/items during catalog generation. The pipeline also writes staging directories at the repository root (local_sync, local_sync_metadata, local_sync_plots). These are mirrored into case_study/catalogs/meteogalicia_interpolation/assets/* by run_pipeline_case_study.sh before the integrity check that keeps the committed catalog in sync with fresh runs. Interpolation mix Samples per source Source Samples % of rows Interpolated grid nodes 642,719,672 98.44 Original WRF nodes retained 10,199,938 1.56 Hold-out/test point emissions 9,506 ≪0.01 Model selection across hourly runs Component Gaussian Exponential Spherical U component 6,039 hours (63.5 %) 2,712 hours (28.5 %) 755 hours (7.9 %) V component 5,043 hours (53.0 %) 3,015 hours (31.7 %) 1,448 hours (15.2 %) Validation summary Average metrics computed per hour (distinct timestamp) and then averaged: Metric U component V component Cross-validation RSR 0.154 0.172 Cross-validation bias (m/s) +0.00006 -0.00032 Hold-out RSR (test) 0.081 0.090 Hold-out bias (m/s) +0.00010 -0.00149 Figures are based on the hourly diagnostics embedded in each partition (cv_* and test_* columns) and computed with duckdb/duckdb:latest so they remain reproducible. PdE Vilano buoy comparison Observed-versus-predicted metrics (3,379 matched hours) are stored in reports/buoys/vilano: Samples RMSE (m/s) MAE (m/s) Bias (m/s) Corr R² Scatter index Dir. RMSE (deg) Comp. corr 3,379 4.56 3.70 +1.52 0.208 -0.86 0.754 88.77 0.269 Component-level scores: Variable RMSE (m/s) RSR Bias (m/s) Wind speed 4.56 1.36 +1.52 U component 5.74 1.31 +0.41 V component 6.02 1.47 -0.61 The Markdown report (vilano_report.md) and the figures (wind_speed_timeseries.png, wind_speed_scatter.png) provide quick visual checks and cite the exact observation/prediction windows (2018-05-01 → 2019-02-01). Reproducing or extending the run Clone & restore the toolkit (outside of this repository) and make sure renv::restore() has been executed. The AWS prerequisites described in the root README.md also apply here. Place this case-study folder alongside the toolkit checkout or unpack the Zenodo bundle (hf-eolus-task-3-interpolation-outputs.zip, DOI https://doi.org/10.5281/zenodo.17490873) into the toolkit's case_study/ directory. Launch the wrapper so it exports all PIPELINE_* variables and copies the outputs back into this repository: ./run_pipeline_case_study.sh \ --toolkit-dir /path/to/hf-eolus-interpolation-toolkit \ --bucket hf-eolus \ --region-name VILA-PRIO-HF \ --ingest-start 2018-01-01 --ingest-end 2019-02-01 \ --interp-start 2018-01-01 --interp-end 2019-02-01 \ --res-factor 8 --cutoff-km 20 --width-km 1 \ --stac-incremental The wrapper sequentially calls every toolkit script (tests, IAM, ingestion, Batch interpolation, STAC builder) unless you opt into --skip-* flags. To keep the committed STAC untouched while verifying a run, omit --force-overwrite-stac. When you need to append newly generated partitions, keep --stac-incremental so only the missing items are added. Buoy comparisons are enabled automatically via case_study/buoy_comparison_config.json; disable them with --skip-buoy-comparison or by setting PIPELINE_ENABLE_BUOY_COMPARISON=0. At the end the script prints whether hashes of the regenerated catalog match the tracked files and where the buoy reports were written. Regenerate STAC items for September–December 2018 (if needed) by ensuring the ingestion/interpolation windows include those months and re-running the builder with --stac-incremental. The assets are already present, so the command will only emit the missing Item JSON files. Inspecting the GeoParquet data (Dockerised DuckDB) Per project rules, both quicklooks and audits should rely on the official DuckDB image. Examples: # Total vectors docker run --rm -v "$(pwd)":/work -w /work duckdb/duckdb:latest duckdb -c " SELECT count(*) AS rows FROM read_parquet('local_sync/year=*/month=*/day=*/hour=*/data.parquet'); " # Unique grid nodes docker run --rm -v "$(pwd)":/work -w /work duckdb/duckdb:latest duckdb -c " SELECT count(DISTINCT node_id) AS unique_nodes FROM read_parquet('local_sync/year=*/month=*/day=*/hour=*/data.parquet'); " # Hourly diagnostics (averaged later) docker run --rm -v "$(pwd)":/work -w /work duckdb/duckdb:latest duckdb -c " WITH hourly AS ( SELECT timestamp, max(cv_rsr_u) AS cv_rsr_u, max(test_rsr_u) AS test_rsr_u FROM read_parquet('local_sync/year=*/month=*/day=*/hour=*/data.parquet') GROUP BY 1 ) SELECT avg(cv_rsr_u) AS avg_cv_rsr_u, avg(test_rsr_u) AS avg_test_rsr_u FROM hourly; " Adapt the queries as needed for additional diagnostics; remember to clean up any temporary files the container might leave behind before committing. Acknowledgements This work has been funded by the HF-EOLUS project (TED2021-129551B-I00), financed by MICIU/AEI /10.13039/501100011033 and by the European Union NextGenerationEU/PRTR - BDNS 598843 - Component 17 - Investment I3. Members of the Marine Research Centre (CIM) of the University of Vigo have participated in the development of this repository. The Government of Galicia, through MeteoGalicia, runs and publishes outputs from its WRF numerical model at various resolutions. This resource uses the 4 km domain (named 'd03') from the WRF_HIST historical dataset. The files were downloaded from MeteoGalicia's THREDDS server (WRF_HIST/d03 directory). MeteoGalicia makes these data freely available; the only condition for using them is to cite MeteoGalicia as the data source. We thank MeteoGalicia for facilitating open access to these atmospheric model simulations. We thank Puertos del Estado for providing public access to the buoy data through https://portus.puertos.es Disclaimer This software is provided "as is", without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose, and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the software or the use or other dealings in the software. References Herrera Cortijo, J. L., Fernández-Baladrón, A., Rosón, G., Gil Coto, M., Dubert, J., & Varela Benvenuto, R. (2025). HF-EOLUS Wind Interpolation Toolkit for MeteoGalicia models output (v0.1.0). Zenodo. https://doi.org/10.5281/zenodo.17598353 Herrera Cortijo, J. L., Fernández-Baladrón, A., Rosón, G., Gil Coto, M., Dubert, J., & Varela Benvenuto, R. (2025). HF-EOLUS. Task 3. MeteoGalicia Wind Interpolation Outputs. Zenodo. https://doi.org/10.5281/zenodo.17490873 Fernández-Baladrón, A., Varela Benvenuto, R., & Herrera Cortijo, J. L. (2020). Interrelationships between Surface Circulation and Wind in the Ría de Vigo. Zenodo. https://doi.org/10.5281/zenodo.17490675