Multi-Dimensional Coupling of Kinematic Anomalies in ADS-B Surveillance Telemetry

Multi-Dimensional Coupling of Kinematic Anomalies in ADS-B Surveillance Telemetry Eugene Pik 20 June 2026 Data License: Creative Commons Attribution 4.0 International (CC BY 4.0) Code License: MIT Dataset DOI (concept, always resolves to the latest version): 10.5281/zenodo.20387308 Overview Welcome to the replication dataset for the manuscript “Cross-Channel Anomaly Coupling in ADS-B: A Surveillance-Integrity Monitor for Air Traffic Management”. This repository contains the complete open-science codebase, raw extraction shards, intermediate analytical products, statistical artifacts, and publication figures required to fully reproduce the conditional-dependence analysis of 2025 global ADS-B kinematic telemetry. The pipeline analyses 117,542,731 aircraft-hour aggregates clustered by 240,001 unique airframes, and 791,862,868 non-overlapping 300-second tumbling windows, all extracted from the OpenSky Network state_vectors_data4 table for calendar year 2025 (1 January through 31 December 2025 UTC). For the year-over-year temporal-anomaly diagnostic, a calendar-aligned 50-day window of Spring 2024 (2 April through 22 May 2024) is re-extracted, yielding an additional 15,733,909 aircraft-hours across 192,747 airframes. A pair-level pilot analysis on 24 sampled hours yields a further 1,176,250,672 consecutive-message pairs across 76,023 airframes, with velocity_data4-level re-extraction exposing the DF17 BDS 0,9 vertical-rate source bit (VrSrc). The pipeline executes rate-normalised per-step anomaly flagging anchored in 14 CFR Part 25 limit load factors, RTCA DO-260B baro-altitude quantisation, and the standard rate of turn; three-tier data-quality stratification by within-hour Δt-coverage; cluster-bootstrap confidence intervals resampling by airframe; an exact within-cluster permutation null distribution preserving per-airframe marginal counts; cluster-robust phase-stratified logistic regression with symmetric-threshold sensitivity; an explicit cluster-robust regression specification controlling for per-hour message density with a DVR × log(total_pairs) interaction; VrSrc-stratified pair-level analysis directly resolving the barometric / GNSS-source decomposition; altitude-band (cruise vs terminal) stratification of the coupling; a deployable multi-channel consistency monitor with detection-performance characterisation; and cross-references against the NOAA Storm Events Database and the GFZ Potsdam daily geomagnetic and solar indices archive. Changes in This Version (v2) This version extends the original deposit: Seven new analysis scripts: stage2_3b_airframe_coverage_audit.py (unique-airframe manufacturer-metadata coverage audit), phase3_10_altitude_stratification_V1.4.py (cruise-vs-terminal altitude stratification), phase3_11_monitor_windows_V1.1.py and phase3_11_monitor_figure_V1.2.py (the multi-channel consistency monitor and its figure), phase3_12_monitor_detection.py (monitor detection-performance characterisation), phase3_14_dv_dh_threshold_audit.py (DV/DH threshold-sweep audit), and verify_residual_items.py (pre-submission QA / traceability guard). Two new publication figures: Figure 5 (the monitor on the coupling/burden state plane) and Figure 6 (monitor detection performance across calendar 2025). New manuscript tables: Table 6 (altitude-band stratification), Table 8 (multi-channel consistency monitor), and Table A.5 (indicator-definition robustness), plus a DVR threshold-sweep audit table. Pipeline updates: Stage 2.3 gains an airframe-class universality block (now V2.7); all source/target directories are centralised in config.py; per-script version history has been consolidated into CHANGELOG.md; and requirements.txt now additionally lists psutil, tabulate, and certifi. Figures are now provided as genuine vector PDFs. Quick Start & Reproducibility To replicate the conditional-dependence analysis locally: Install dependencies: pip install -r requirements.txt Execute the pipeline: Stages are decoupled and write intermediate Parquet shards to allow per-stage reruns. All source/target directories are resolved from config.py. CODE/stage1_master_metadata_extraction_V5.3.py — Builds the enriched aircraft-metadata registry by merging the OpenSky aircraft database (base icao24, type-designator, registration, icaoAircraftClass), the FAA Aircraft Registry (US tail-number built_year), Transport Canada CARS (Canadian tail-number built_year), and the Rikgale ICAO Type designators (typecode-to-engine and typecode-to-airframe-category resolution). CODE/stage2_1_pphysical_extraction_V2.2.py — Aggregates per-step rate-normalised flags to aircraft-hour granularity (Dataset 3A). Runs against the OpenSky Trino interface; requires OpenSky credentials. CODE/stage2_2_window_extraction_V2.2.py — Aggregates per-step flags to 300-second tumbling windows (Dataset 3B) and classifies each window’s manoeuvring envelope. CODE/stage2_3_metadata_enrichment_V2.7.py — Merges Dataset 3A with the metadata registry and computes pairwise enrichment, conditional-independence violation ratios, and engine-type / airframe-class universality statistics. CODE/stage2_4_aerodynamic_masking_regression_V1.2.py — Fits the phase-stratified cluster-robust logistic regression on Dataset 3B. CODE/phase3_2_*.py through CODE/phase3_14_*.py — Phase 3 robustness analyses (data-quality decomposition, DVR threshold sweep, geographic and seasonal stratification, NACv/NIC/receiver-redundancy stratification, pair-level NACv pilot, Spring 2025 elevation diagnostic, year-over-year Spring 2024 re-extraction, space-weather cross-reference, forest-plot generation, publication figures, VrSrc-stratified pair-level pilot, log(total_pairs)-controlled cluster-robust regression, altitude-band stratification, multi-channel consistency monitor and its detection-performance characterisation, and the DV/DH threshold-sweep audit). Dataset Structure This Zenodo dataset consists of three ZIP archives and four loose files: . ├── README.md <- this file ├── requirements.txt <- Python dependencies ├── LICENSE-CODE-MIT.txt ├── LICENSE-DATA-CC-BY-4.0.txt │ ├── replication_kit.zip <- code + results (~1 MB) │ └── replication_kit/ │ ├── CODE/ │ │ ├── config.py │ │ ├── CHANGELOG.md │ │ ├── stage1_master_metadata_extraction_V5.3.py │ │ ├── stage2_1_pphysical_extraction_V2.2.py │ │ ├── stage2_2_window_extraction_V2.2.py │ │ ├── stage2_3_metadata_enrichment_V2.7.py │ │ ├── stage2_3b_airframe_coverage_audit.py │ │ ├── stage2_4_aerodynamic_masking_regression_V1.2.py │ │ ├── phase3_2_robustness_analyses.py │ │ ├── phase3_3_integrity_extraction_V2.2.py │ │ ├── phase3_3_nac_stratification.py │ │ ├── phase3_4_spring_diagnostic.py │ │ ├── phase3_4b_spring2024_full_window_V1.1.py │ │ ├── phase3_5_combined_robustness.py │ │ ├── phase3_6_space_weather_V1.2.py │ │ ├── phase3_7_forest_data_builder_V1.2.py │ │ ├── phase3_7_figure_generation_V1.5.py │ │ ├── phase3_8_vrsrc_pilot_V1.2.py │ │ ├── phase3_9_npair_regression_V1.4.py │ │ ├── phase3_10_altitude_stratification_V1.4.py │ │ ├── phase3_11_monitor_windows_V1.1.py │ │ ├── phase3_11_monitor_figure_V1.2.py │ │ ├── phase3_12_monitor_detection.py │ │ ├── phase3_14_dv_dh_threshold_audit.py │ │ ├── phase3_engine_breakdown_audit.py │ │ ├── rename_dq_tier_columns.py │ │ └── verify_residual_items.py │ └── RESULTS/ │ ├── summaries/ <- JSON summaries from every phase │ ├── manuscript_tables/ <- one CSV per LaTeX tab:* label │ ├── spring2025_diagnostics/ <- d1-d8 + storm-events tables │ ├── space_weather/ <- GFZ Potsdam cross-reference │ ├── permutation_null/ <- 1,000-iteration null distribution │ ├── forest_data/ <- enrichment-forest input │ └── figures/ <- Figures 1-6 PDFs + plot data │ ├── source_registries.zip <- registries (~187 MB) │ └── source_registries/ │ └── aircraft_db/ │ ├── FAA/ <- MASTER.txt, ACFTREF.txt │ ├── ICAO/ <- rikgale_icao_types.csv │ ├── OpenSky/ <- 8 monthly snapshots │ └── Transport_Canada/ <- carscurr.txt │ └── extractions.zip <- parquet shards (~20 GB) └── extractions/ ├── Dataset_3A_aircraft_hour/ <- 12 monthly shards ├── Dataset_3B_300s_windows/ <- 2 monthly shards (Apr, May) ├── Integrity_NACv_NIC_receiver/ <- 12 monthly shards ├── pphysical_monthly_aggregates/ <- 12 monthly shards (Stage 2.1) ├── Windows_monthly_aggregates/ <- 12 monthly shards (Stage 2.2) ├── Spring_2024_re_extraction/ <- year-over-year diagnostic ├── Seasonal_checkpoint_extracts/ <- spring/summer/autumn/winter v2 └── Aircraft_metadata_registry/ <- enriched_aircraft_metadata.parquet File Manifest & Data Dictionary 1. Root Files README.md — This documentation file. requirements.txt — Minimal Python dependency list (Python ≥ 3.9; pandas, numpy, scipy, statsmodels, pyarrow, matplotlib, trino-python-client, certifi, psutil, tabulate). LICENSE-CODE-MIT.txt — MIT license covering all Python source code. LICENSE-DATA-CC-BY-4.0.txt — Creative Commons Attribution 4.0 covering all data files. 2. replication_kit.zip / CODE/ config.py — Shared configuration module: pipeline thresholds (DV > 10 m/s², DVR > 8 m/s², DH > 15 °/s), centralised data-directory constants (including AIRCRAFT_DB_DIR and PROCESSED_DIR), OpenSky Trino credentials helper, seasonal checkpoints, exclusion lists for known broadcaster pathologies. CHANGELOG.md — Consolidated, newest-first version history for every script (previously embedded in the individual file headers). stage1_master_metadata_extraction_V5.3.py — Builds the cross-referenced aircraft-metadata registry from four sources: the OpenSky aircraft-database snapshots (base table), the FAA Aircraft Registry, Transport Canada CARS, and the Rikgale ICAO Type designators. Input/output directories resolved from config.py. stage2_1_pphysical_extraction_V2.2.py — Aggregates per-step DV/DVR/DH flags to aircraft-hour granularity with the alternate-threshold columns pre-extracted for the DVR sweep. Embeds the production Trino SQL filter WHERE dt_seconds > 0 AND dt_seconds <= 60. stage2_2_window_extraction_V2.2.py — Produces Dataset 3B; classifies each 300-second window as Climb/Cruise/Descent/Transition. stage2_3_metadata_enrichment_V2.7.py — Merges metadata into Dataset 3A and computes pairwise enrichment, conditional-independence violations, and engine-type universality. (V2.7) Adds an airframe_universality block (whole-class conditional enrichment + cluster-bootstrap CIs for Fixed-Wing and Rotary-Wing on the full merged frame). stage2_3b_airframe_coverage_audit.py (NEW) — Unique-airframe metadata-coverage audit. Recomputes manufacturer-metadata coverage over the distinct icao24 of the low-latency Dataset 3A (the population the manuscript names) to confirm the coverage figure reported in §3.1 (77.3% of unique airframes). With --deposit, injects one key into stage2_3_enrichment_summary.json. stage2_4_aerodynamic_masking_regression_V1.2.py — Phase-stratified cluster-robust logistic regression and the rotary-wing saturation grid. phase3_2_robustness_analyses.py — Data-quality tier decomposition, DVR threshold sweep, seasonal-week robustness, geographic robustness, strict-Fixed-Wing replication, helicopter saturation grid. phase3_3_integrity_extraction_V2.2.py — Aircraft-hour-level NACv/NIC/receiver-redundancy stratification. Strict-high NACv definition is nacv_velocity_min >= 3. phase3_3_nac_stratification.py — Pair-level NACv pilot. Pair classification is LEAST(nac, p_nac); high-integrity stratum is NACv ≥ 2. phase3_4_spring_diagnostic.py — 365-day temporal breakdown identifying the Spring 2025 elevation episode and its UTC diurnal, geographic, airframe, and engine localisation. phase3_4b_spring2024_full_window_V1.1.py — Re-extraction of the calendar-aligned 50-day Spring 2024 window using the identical SQL pipeline. phase3_5_combined_robustness.py — Permutation null model, Stage 2.4 symmetric-threshold sensitivity, and NOAA Storm Events cross-reference. phase3_6_space_weather_V1.2.py — GFZ Potsdam daily geomagnetic and solar indices cross-reference against the daily P(DV) trajectory. phase3_7_forest_data_builder_V1.2.py and phase3_7_figure_generation_V1.5.py — Assemble the forest-plot input and generate publication Figures 1–4. (forest V1.2) Airframe rows re-enabled from stage2_3 V2.7 airframe_universality. phase3_8_vrsrc_pilot_V1.2.py — VrSrc-stratified pair-level pilot. Re-extracts the 24-partition sample directly from velocity_data4, exposing the DF17 BDS 0,9 source bit as the baro column (TRUE = barometric, FALSE = geometric/GNSS). Pairs are classified into strict-barometric / strict-geometric / mixed-source strata; enrichment and its within-cluster bootstrap CI are computed per stratum. phase3_9_npair_regression_V1.4.py — total_pairs-controlled cluster-robust logistic regression. Fits Model A (flag_dv_rate ~ flag_dvr_rate + flag_dh_rate) and Model B (Model A plus log(total_pairs) covariate and flag_dvr_rate:log(total_pairs) interaction), plus a quintile-stratified pairwise enrichment table. Uses the canonical smf.logit(...).fit(cov_type='cluster', cov_kwds={'groups': icao24}, method='lbfgs') pattern. phase3_10_altitude_stratification_V1.4.py (NEW) — Altitude-band re-extraction and cruise-vs-terminal stratification of the DV/DVR/DH coupling. Pushes per-(icao24, altitude-band) sufficient statistics server-side; one hour per query with atomic per-shard resume. Tests whether the coupling persists at cruise, where coordinated real severe manoeuvre is implausible. phase3_11_monitor_windows_V1.1.py (NEW) — Multi-channel consistency monitor: computes pairwise enrichment R and the Pathway-B violation VB (with marginals) over three monitoring windows (episode_2025, baseline_2025, y2024), each with a within-airframe cluster-bootstrap 95% CI. phase3_11_monitor_figure_V1.2.py (NEW) — Renders Figure 5 (the monitor on the coupling/burden state plane, plus single-channel anomaly prevalences per window). phase3_12_monitor_detection.py (NEW) — Detection-performance characterisation of the monitor at daily resolution across 2025: control-limit calibration, realised in-control false-alarm rate and ARL0, episode-onset detection latency. Renders Figure 6. phase3_14_dv_dh_threshold_audit.py (NEW) — DV/DH threshold-sweep audit. Schema audit of alternate-threshold columns, a DVR-sweep validation, and track/definition robustness recomputed directly from the deposited Dataset 3A parquet (no re-extraction). phase3_engine_breakdown_audit.py — Strict-Fixed-Wing engine-type breakdown audit. rename_dq_tier_columns.py — Utility for harmonising Stage 2.1 DQ-tier column naming across pipeline versions (now defaults its --csv path from config.py). verify_residual_items.py (NEW) — Phase-3 QA: re-derives the numbers behind residual manuscript / traceability-matrix items directly from the deposited data files and audits LaTeX citation usage. Non-zero exit on any failed check, so it doubles as a pre-submission CI guard. 3. extractions.zip (Parquet shards) Dataset_3A_aircraft_hour/ — Per-month Parquet shards of the 117,542,731 aircraft-hour aggregates, with eight integer counters per cluster {n, n_DV, n_DVR, n_DH, n_DV,DVR, n_DV,DH, n_DVR,DH, n_DV,DVR,DH}, median latitude/longitude, latency tier, fraction Δt = 1 s, and alternate-threshold columns for the DVR sweep. Dataset_3B_300s_windows/ — Local two-month export of the 300-second tumbling-window aggregates. The canonical 12-month set is in Windows_monthly_aggregates/. Integrity_NACv_NIC_receiver/ — Per-month NACv/NIC/receiver-redundancy stratification source data. pphysical_monthly_aggregates/ — Stage 2.1 output, identical schema to Dataset 3A. Windows_monthly_aggregates/ — Stage 2.2 output, 12 monthly shards aggregating the 791,862,868 windows. Spring_2024_re_extraction/ — Calendar-aligned 50-day Spring 2024 aircraft-hour aggregates (15,733,909 rows over 192,747 airframes). Seasonal_checkpoint_extracts/ — Four one-week seasonal-checkpoint extracts (Winter, Spring, Summer, Autumn 2025) plus the Stage 2.4 / Phase 3.5 seasonal-windows extraction. Aircraft_metadata_registry/enriched_aircraft_metadata.parquet — Stage 1 output: enriched aircraft registry cross-referenced across the OpenSky aircraft-database snapshots, FAA, Transport Canada, and Rikgale. Note: phase3_10 (altitude-strat) and phase3_11 (monitor-windows) re-extract their own intermediate shards directly from the OpenSky Trino interface and do not require additional deposited parquet. 4. source_registries.zip (Source Registries) aircraft_db/FAA/ — U.S. FAA Aircraft Registry (MASTER.txt, ACFTREF.txt). Public-domain US-government work (17 USC §105). aircraft_db/ICAO/ — Rikgale-compiled ICAO Type Designators (rikgale_icao_types.csv). aircraft_db/OpenSky/ — Eight monthly snapshots (May 2024 through February 2025) of the OpenSky aircraft database. aircraft_db/Transport_Canada/ — Transport Canada Civil Aircraft Register (carscurr.txt). Government of Canada Open Government Licence. 5. RESULTS/ (Analytical Outputs, inside replication_kit.zip) Summaries (summaries/*.json): Per-phase machine-readable summaries containing population counts, headline enrichments, violation ratios with bootstrap confidence intervals, and per-stratum statistics. Includes the V38 additions (phase3_8_vrsrc_pilot_summary.json, phase3_9_npair_regression_summary.json) and the new phase3_10_altitude_summary.json, phase3_11_monitor_summary.json, phase3_12_monitor_detection_summary.json, phase3_14_threshold_audit_summary.json, and stage2_3b_airframe_coverage_summary.json (the §3.1 manufacturer-metadata coverage audit). Manuscript Tables (manuscript_tables/*.csv): Direct mapping to the \label{tab:...} tags in the manuscript: CSV file Manuscript reference phase3_2_dq_tier_table.csv data-quality tier decomposition phase3_2_seasonal_table.csv seasonal robustness phase3_2_geographic_table.csv geographic robustness phase3_2_threshold_sweep.csv DVR threshold sweep phase3_2_strict_fw_regression.csv strict-Fixed-Wing regression phase3_2_helicopter_saturation_grid.csv rotary-wing saturation grid phase3_3_nacv_tier_table.csv, phase3_3_nic_tier_table.csv, phase3_3_receiver_tier_table.csv, phase3_3_joint_strict_table.csv, phase3_3_triple_intersection_table.csv NACv/NIC/receiver-redundancy stratification phase3_3_nac_strata.csv pair-level NACv pilot phase3_5_M1_observed_vs_null.csv permutation null observed-vs-null phase3_5_M2_threshold_sensitivity_table.csv symmetric-threshold sensitivity phase3_4b_d8b_spring2024_full_window.csv Spring 2024 year-over-year comparison phase3_engine_breakdown_strict.csv strict-Fixed-Wing engine breakdown phase3_8_vrsrc_strata.csv tab:pair_vrsrc — VrSrc-stratified pair-level enrichment (strict-barometric / strict-geometric / mixed-source) phase3_9_npair_regression_coefficients.csv tab:npair_regression — Model A baseline and Model B with log(total_pairs) + interaction, cluster-robust ORs and CIs phase3_9_npair_quintile_strata.csv tab:npair_quintile — pairwise enrichment stratified by total_pairs quintile phase3_10_altitude_strata.csv (NEW) Table 6 — altitude-band (cruise vs terminal) stratification of R and VB (§4.14) phase3_11_monitor_windows.csv (NEW) Table 8 — multi-channel consistency monitor: per-window burden, coupling (VB) and R over the episode + two nominal windows (§6.3) phase3_14_dvr_sweep_table.csv (NEW) DVR threshold-sweep audit reproduction (§4.6 / App. A.1) — validation cross-check of the deposited parquet; the canonical sweep table is phase3_2_threshold_sweep.csv phase3_14_track_robustness_table.csv (NEW) Table A.5 — indicator-definition (Track 1/2/3) robustness of R and VB (§4.6 / App. A.1) Also present (supplementary / audit tables, not in the v1 manuscript-table mapping): h7_latency_tier_gradient.csv, h8_two_pathway_decomposition.csv, h9_engine_lineage_results.csv, global_volume_census_2025.csv, and stage2_3b_airframe_coverage_by_source.csv (per-source manufacturer-metadata coverage, §3.1). Spring 2025 Diagnostics (spring2025_diagnostics/*.csv): Daily, weekly, hourly, geographic, altitude, airframe, and storm-events stratification of the 50-day elevation episode. Space-Weather Cross-Reference (space_weather/*.csv): Daily joined P(DV) and GFZ Potsdam indices, Pearson and Spearman correlations, storm-day coincidence summaries, and the top-ten extreme P(DV) days. Permutation Null (permutation_null/*.csv): Full 1,000-iteration null distribution of enrichment, VA, VI, VB. Figures (figures/): File Manuscript reference Figure_1_permutation_null.pdf Figure 1 — permutation null distribution for the enrichment statistic Figure_2_saturation_grid_2panel.pdf Figure 2 — rotary-wing saturation grid (empirical vs Model 3B predicted) Figure_3_enrichment_forest.pdf Figure 3 — forest plot of pairwise enrichment across all stratifications Figure_4_spring_2025_timeline.pdf Figure 4 — daily P(DV), P(DVR), P(DH) for calendar 2025 with the 50-day elevation window and the daily sunspot number overlay Figure_5_monitor.pdf (NEW) Figure 5 — multi-channel consistency monitor on the (coupling, burden) state plane Figure_6_monitor_detection.pdf (NEW) Figure 6 — monitor detection-performance characterisation across calendar 2025 Accompanying *_data.csv files contain the X-Y coordinate data plotted in each figure. Notes and Exclusions Manuscript LaTeX source is not included in this deposit. It is distributed separately via the journal upon acceptance. Pipeline execution logs are not included. They contain no information required for reproducibility; re-running the pipeline against the parquet shards regenerates all analytical outputs deterministically. icao_firs_global.geojson (FIR boundary polygons) is licensed and therefore not redistributed here. The geographic stratification is reproducible from the median latitude/longitude column retained in each Dataset 3A parquet shard, using the continental bounding boxes specified in config.py. Phase 3.8 / 3.10 / 3.11 re-runs require live access to the OpenSky Trino interface (raw-message or hourly re-extraction); supply OpenSky credentials via the standard config.py mechanism. Phase 3.9, 3.12, and 3.14 read directly from the deposited parquet shards and require no network access. How to Cite If you use this dataset, pipeline, or any of its analytical outputs, please cite the accompanying manuscript: Pik, E. (2026). Cross-Channel Anomaly Coupling in ADS-B: A Surveillance-Integrity Monitor for Air Traffic Management. Aerospace Science and Technology (under review). And the dataset itself: Pik, E. (2026). Replication dataset for “Multi-Dimensional Coupling of Kinematic Anomalies in ADS-B Surveillance Telemetry” (Version v2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.20387308 Dataset DOI (concept): 10.5281/zenodo.20387308 Acknowledgements The underlying state-vector telemetry is provided by the OpenSky Network (https://opensky-network.org) under their published terms of service. The NOAA Storm Events Database is publicly distributed by the U.S. National Centers for Environmental Information. The GFZ Potsdam daily geomagnetic and solar indices archive (Kp, ap, Ap, sunspot number, F10.7) is publicly distributed by GFZ Helmholtz Centre Potsdam. The FAA Aircraft Registry is a public-domain U.S.-government work; the Transport Canada Civil Aircraft Register is published under the Government of Canada Open Government Licence.