fMoW-ERA5: A Tri-Modal Satellite Image Dataset with Textual and Environmental Metadata for Diffusion-Based Generation

This dataset accompanies the paper "Environment-Aware Satellite Image Generation with Diffusion Models" and provides one of the first publicly available tri-modal annotation files for satellite imagery, combining textual scene descriptions with both static geographic metadata and continuous numerical environmental metadata derived from the ERA5 climate reanalysis. The dataset is released as a JSON Lines annotation file (.jsonl) designed for use in conjunction with the fMoW-RGB dataset (Functional Map of the World, Christie et al. 2018, https://github.com/fMoW/dataset), which must be downloaded separately under its own license. Each record in this file corresponds to one fMoW image and contains the following fields: file_name: filename of the corresponding fMoW image text: scene caption of the form "a satellite image of a <class> in <country> during <season>, <year>" lon, lat: centroid coordinates of the depicted scene year, month, day: image capture date gsd: ground sampling distance (m/pixel) t2: 2 m temperature, 5-day average (K) [ERA5: 2t] tp: total precipitation, 5-day average (mm) [ERA5: tp] 10u: 10 m U wind component, 5-day average (m/s) [ERA5: 10u] 10v: 10 m V wind component, 5-day average (m/s) [ERA5: 10v] ssr: surface net solar radiation, 5-day sum (J/m²) [ERA5: ssr] tcc: total cloud cover, instantaneous (0–1) [ERA5: tcc] t2d: 2 m dewpoint temperature, 5-day average (K) [ERA5: 2d] ERA5 variables were retrieved from the Copernicus Climate Data Store (CDS) using the cdsapi Python library in NetCDF format (Hersbach et al. 2020, https://doi.org/10.1002/qj.3803). For each fMoW image, the nearest ERA5 25×25 km grid cell to the image centroid was identified, and dynamic variables were aggregated over a 5-day window prior to the image capture time, reflecting the characteristic timescale over which weather conditions influence surface-level visual attributes. Total cloud cover was not aggregated, as it directly affects image acquisition quality rather than cumulative surface processes. The annotation file is provided for the fMoW training split. It is intended for use with diffusion-based generative models for satellite imagery, data augmentation in imbalanced remote sensing classification tasks, and multimodal representation learning.