IBTrACS + ERA5 Tropical Cyclone Benchmark (North Atlantic, 2000–2024) — Multi-Horizon Track, Intensity, RI, & Lifecycle

🚀 Release v2 — IBTrACS + ERA5 Tropical Cyclone Benchmark (North Atlantic, 2000–2024) Scope: North Atlantic basin • Years: 2000–2024 • Cadence: 6-hourly (00/06/12/18 UTC) Includes: ML-ready master table, task subsets (intensity, track, RI-24h, lifecycle), splits, and dictionaries. ✨ Highlights (v2) Unified master (2000–2024) at 6-hour cadence with harmonized IBTrACS tracks and collocated ERA5 fields. Forecast targets at +6 / +12 / +24 h for wind and position (e.g., wmo_wind_next6, lat_next24, lon_next24). Ready-made tasks: Intensity regression (evaluate with MAE/RMSE) Track regression (evaluate with great-circle error) Rapid Intensification (24 h) classification Lifecycle classification (dissipation / ET within 48 h) Leakage-safe storm-level splits (train / val / test) and sample weights for imbalance. Schemas in human & machine-readable form (DATA_DICTIONARY.md, *_dictionary.csv). Reproducible build with two scripts: scripts/build_dataset.py → scripts/add_targets.py. 📦 Release Assets outputs/model_ready_2000_2024_v2.parquet — Master benchmark table (6-hour). IBTrACS + ERA5 features + multi-horizon targets. outputs/merged_ibtracs_era5_2000_2024_clean.parquet — Clean merged intermediate (pre-target engineering) for custom target design. outputs/tasks/*.parquet — Slim task-specific subsets (intensity, track, RI-24h, lifecycle). outputs/DATA_DICTIONARY.md, outputs/model_ready_2000_2024_v2_dictionary.csv — Variable names, units, descriptions. [!TIP] Use the task subsets for baselines/benchmarks; use the master for feature engineering & R&D. 🧪 Tasks & Targets (at a glance) Intensity (regression): wmo_wind_next{6,12,24} (kt) → MAE / RMSE Track (regression): lat_next{6,12,24}, lon_next{6,12,24} (deg) → Haversine km Rapid Intensification (24 h): ri_next24 (Δwind ≥ 30 kt) → AUROC / AUPRC / F1 Lifecycle (48 h horizon): dissipates_within48, et_within48 → F1 / AUROC Anti-leakage: use the provided storm-level split; never mix fixes from the same storm across sets. ⚙️ ERA5 Region & Variables (Benchmark Cut) Bounds: N 50°, S 0°, W 100°, E 10° • Cadence: 6-hourly • Format: NetCDF4 (source) Variables: u10, v10, wind10, t2m_c, sst_c, msl_hpa, tp (+ IBTrACS: wmo_wind, wmo_pres, basin, metadata) [!IMPORTANT] ERA5 raw data are not included (size/licensing). Fetch from Copernicus CDS and rebuild with the scripts if needed. 🏃 Quickstart The following examples show how to load the dataset, apply the provided splits, and run simple baselines for intensity and track forecasting. These baselines help verify correctness before building advanced ML models. Load the master dataset import pandas as pd, numpy as np df = pd.read_parquet("outputs/model_ready_2000_2024_v2.parquet") print(df.shape, df.columns[:12]) Splits train = df[df["split"] == "train"] val = df[df["split"] == "val"] test = df[df["split"] == "test"] Intensity baseline: persistence @ +6 h tmp = df[["storm_id","iso_time","wmo_wind","wmo_wind_next6"]].dropna() y = tmp["wmo_wind_next6"].to_numpy() yhat = tmp["wmo_wind"].to_numpy() mae = np.mean(np.abs(y - yhat)) print(f"Persistence MAE (+6h): {mae:.2f} kt") Track error (Haversine, +12 h): import numpy as np R = 6371.0 # km def haversine(lat1, lon1, lat2, lon2): p = np.pi/180 dlat, dlon = (lat2-lat1)*p, (lon2-lon1)*p a = np.sin(dlat/2)**2 + np.cos(lat1*p)*np.cos(lat2*p)*np.sin(dlon/2)**2 return 2*R*np.arcsin(np.sqrt(a)) tt = df[["lat","lon","lat_next12","lon_next12"]].dropna() err_km = haversine(tt["lat"], tt["lon"], tt["lat_next12"], tt["lon_next12"]) print(f"Median great-circle error (+12h): {np.median(err_km):.1f} km") 🔁 Build Pipeline Raw ingestion — IBTrACS v04r01 (global tracks) + ERA5 (regional cut), 6-hour alignment. Cleaning & normalization — keys: storm_id, iso_time (UTC), lat, lon; units; duplicates. Feature & target engineering — multi-horizon shifts {6,12,24} for wind/position; masks has_next{6,12,24,48}; ri_next24; lifecycle flags; splits; sample_weight. Export — model_ready_2000_2024_v2.parquet, merged_*_clean.parquet, tasks/*.parquet. Recreate: python scripts/build_dataset.py python scripts/add_targets.py 📊 Versioning & Changelog Version: v2 Added: unified 2000–2024 master; refreshed task subsets; improved data dictionary. Improved: leakage-safe storm-level splits and sample weighting. Clarified: licensing and ERA5 acquisition notes. v1: initial merge & targets. 📜 Licensing Code: MIT License Processed Data: CC-BY 4.0 Raw ERA5: acquire from Copernicus CDS per C3S/ECMWF terms. 🧾 Cite This Dataset DOI: https://doi.org/10.5281/zenodo.17239540 @dataset{hasan_ibtracs_era5_tc_benchmark_2000_2024_v2, author = {Saifur Rahman Mehedi}, title = {IBTrACS + ERA5 Tropical Cyclone Benchmark (2000–2024), v2}, year = {2025}, publisher = {Zenodo}, doi = {10.5281/zenodo.17239540}, url = {https://doi.org/10.5281/zenodo.17239540} } ⚠️ Intended Use & Ethics This dataset is for research and education in climate science and ML. Do not use for real-time hazard guidance — rely on official sources (e.g., NOAA/JTWC). Always disclose uncertainties and limitations when publishing results. 👤 Maintainer Saifur Rahman Mehedi — Issues and questions welcome via the repository's Issue Tracker. LinkedIn