Harmonised aircraft observation database for upper-tropospheric humidity and cirrus/contrail studies (FAAM & IAGOS, 2019-2025)

Overview This dataset provides a harmonised aircraft observation database (2019-2025) designed for upper-tropospheric/lower-stratospheric research, focusing on relative humidity with respect to ice (RHi) measurements in the range 80-100% (pre-saturated). The database merges two in-situ aircraft data streams into a consistent Parquet schema: FAAM (Facility for Airborne Atmospheric Measurements) BAe-146 research aircraft core measurements (1 Hz). IAGOS (In-service Aircraft for a Global Observing System) commercial aircraft time-series observations (CORE package products). The database is optimised for high-performance computing and is intended for filtering to upper-tropospheric conditions, computing diagnostic quality control (QC) metrics, and collocating aircraft tracks with satellite or reanalysis data for cirrus/contrail analyses. What is included The released database contains harmonised Parquet files partitioned by year. Each file contains a standard set of core geolocation/altitude variables, thermodynamic variables (pressure/temperature/humidity, where available), wind components (where available), and derived QC diagnostics. Time is stored as time_utc at millisecond resolution (Parquet TIMESTAMP(MILLIS)) for consistency across workflows. See the table below for the full list of variables What is not included ADS-B-derived trajectory data were used during internal development and testing of the harmonisation approach but are not included in this Zenodo release because the underlying ADS-B products are commercial and not licensed for public redistribution. The repository documentation explains how ADS-B data fit into the broader workflow without distributing the proprietary source data. Quality control and uncertainties Where available, upstream instrument QC flags and combined uncertainty fields are retained or documented alongside the variables (especially for FAAM humidity/temperature products). Additional derived QC diagnostics (e.g., RHI maxima and extreme-duration checks) are included as separate fields to support reproducible filtering and sensitivity tests. File organisation The dataset is stored as Parquet under a year-based hierarchy (e.g., .../FAAM/2019/ and .../IAGOS/2019/), with filenames based on the first timestamp in each output file. Supplementary metadata files (variable lists, sample heads, uncertainty tables, and dictionary lookups used to reduce path-string redundancy in processing) are provided alongside the Parquet outputs. Intended use The primary intended use is scientific analysis of UTLS humidity and cirrus/contrail-relevant conditions, including building aircraft–satellite intersection datasets, evaluating the prevalence of ice supersaturation, and supporting sensitivity analyses that account for measurement uncertainty and QC flags. * An asterisk denotes that an associated QC flag exists for the upstream FAAM measurement variable and is included in the dataset.† IAGOS vertical wind products are model-derived (e.g., ERA5/ECMWF) and are not included as in-situ data in this database. Table 1: Harmonised variables Category Harmonised variable FAAM IAGOS Core flight_id Flight code (e.g. C278) — Core time_utc FAAM time coordinate UTC_time Core latitude_deg LAT_GIN* lat Core longitude_deg LON_GIN* lon Core altitude_m ALT_GIN or PALT_RVS* baro_alt_AC* Core altitude_source Derived(values: 'd' (direct) or 'b' (barometric)) Derived (value: 'b') Thermodynamics pressure_hpa PS_RVSM* air_press_AC (→ hPa)* Thermodynamics temperature_k TAT_DI_R / TAT_ND_R* air_temp_P1 / air_temp_AC* Thermodynamics temperature_source Derived (values: ‘R102BL’ or ‘R102AL’) Derived (value: ‘PT100’) Thermodynamics specific_humidity_kgkg Derived (WVSS, P, T) — Thermodynamics wvss_vmr_ppm WVSS2F_VMR_C* — Thermodynamics neph_rh_percent NEPH_HUM* — Thermodynamics rhi RH_ICE (or derived)* rhi_P1* Thermodynamics rhi_error RH_ICE_CU* rhi_P1_error Winds u_east_ms U_C* zon_wind_AC* Winds v_north_ms V_C* mer_wind_AC* Winds w_vertical_ms W_C* —† Cloud microphysics lwc_kgkg_raw LWC (raw; probe dependent)* — Cloud microphysics lwc_kgkg LWC (cleaned/clipped)* — Cloud microphysics iwc_kgkg IWC — Table 2: QC diagnostics and provenance fields Category QC harmonised variable FAAM IAGOS QC diagnostics rhi_source Source tag (dataset-specific) ICH where RHI present QC diagnostics rhi_max Derived per flight/segment maximum Derived per segment maximum QC diagnostics rhi_gt200_seconds Duration above 200% threshold Duration above 200% threshold QC diagnostics exclude_rhi200_gt5min Derived exclusion flag Derived exclusion flag QC diagnostics lwc_was_negative Derived flag — Metadata faam_flight_dir FAAM-specific flight directory for provenance Metadata core_filename FAAM-specific flight filename for provenance