Analysis package for the article: How Much Does Architecture Really Matter? A Statistical and Cost-Aware Benchmark of Deep Learning Models for Sentinel-1 Water Segmentation

Complete, self-contained analysis package accompanying the manuscript “How Much Does Architecture Really Matter? A Statistical and Cost-Aware Benchmark of Deep Learning Models for Sentinel-1 Water Segmentation” (2026). The study benchmarks nine convolutional encoder–decoder architectures for segmenting water from Sentinel-1 Synthetic Aperture Radar imagery of the Poechos reservoir, northern Peru. No deep-learning model was trained here: the study reuses 45 pre-trained checkpoints (three decoders — U-Net, U-Net++ and Feature Pyramid Network — crossed with three encoders — ResNet50, EfficientNet-B7 and Inception-v4 — each as a five-fold ensemble) developed by a companion reservoir-monitoring study, and interrogates them with a purely analytical protocol. Research questions Are the accuracy differences among the nine models statistically real, and is the systematic signal driven by the encoder or by the decoder? Does the internal cross-validation ranking predict performance on independent external data? Once accuracy is accounted for, how large is the spread in computational cost, and which models are Pareto-optimal? Does a statistical tie translate into interchangeable water masks? Version 2 (August 2026) extends the protocol with two further analyses on the same fixed checkpoints: formal equivalence testing (paired two one-sided tests with Holm correction, Bayesian region-of-practical-equivalence posteriors, and a Nemenyi critical-difference analysis) and measured GPU inference energy (NVML power integration on an RTX 1000 Ada). Together these show that the nine models are not merely statistically indistinguishable but provably equivalent within one percentage point of mIoU, while spanning a five-fold range in energy per patch. The companion-study citation is updated to its published version. Main findings Across five cross-validation folds the nine models span only 0.53 percentage points of mean Intersection-over-Union. A two-way analysis of variance blocked by fold shows that the data partition explains 98.0% of the accuracy variance, the encoder a small but significant 0.71% (p = 0.0002), and the decoder architecture a non-significant 0.07% (p = 0.334). A Friedman test confirms a reproducible but marginal ordering, while Wilcoxon–Holm post-hoc tests resolve no individual pair. The internal ranking is essentially uncorrelated with the independent external-test ranking (Spearman ρ = −0.03; bootstrap 95% CI [−0.35, +0.68]); EfficientNet-B7 is the only backbone ranked first in both regimes. Computational cost varies five-fold in inference time (76–369 ms per patch) and seven-fold in floating-point operations (15.6–114.9 GFLOPs); all three Pareto-optimal models use Feature-Pyramid-Network decoders, and every U-Net and U-Net++ variant is dominated. A faithful replication of the companion study’s threshold-calibration pipeline reproduces its published optimum (0.64) exactly; per-model calibration adds only 0.4–1.2 points of pooled IoU. On a held-out scene, the best- and worst-ranked calibrated models disagree on only 0.27% of pixels, their water masks overlapping at an IoU of 98.2% — the statistical tie is also a practical one. Conclusion. When segmentation accuracy is statistically indistinguishable, model selection should be governed by computational cost. The protocol is transferable to other applied segmentation benchmarks. Contents scripts/ — the methodology scripts (Python, 8 files) that produce every reported number. Neither the figure images nor the code that draws them are included: this deposit archives the analysis, not its rendering. Every quantity shown in the manuscript figures comes from the tables in derived_tables/. derived_tables/ — all result tables: consolidated cross-validation metrics, ANOVA and Friedman outputs, rank-correlation results, computational-cost profile, Pareto set, per-model threshold-calibration curves and the mask-agreement statistics. base_study_data/ — the lightweight inputs reused from the companion study: per-fold cross-validation metric records, ensemble results and configurations for the nine models, and three held-out Sentinel-1 test scenes with their PlanetScope ground-truth masks. What is and is not reused. This package reuses only the artefacts of the companion study’s segmentation stage: the 45 trained checkpoints, their per-fold validation records, and the held-out test scenes with their ground-truth masks. The SWOT altimetry, the stochastic coupling, the water-area time series and the elevation–area–volume curve are outputs of the companion study and are not used in this analysis; the two spreadsheets under base_study_data/spreadsheets/ are included for provenance and context only. No new imagery, model or map was generated. Data provenance. All benchmarked models, cross-validation metrics and Sentinel-1 patches originate from the companion study, now published as: Breña Aliaga, J. C., Bourrel, L., Cruz Machacuay, J., Breña Oré, J. L., Felipe, O., Rau, P., & Lavado-Casimiro, W. (2026). Continuous Satellite Monitoring of Reservoir Capacity Loss Using Deep Learning and Stochastic Mapping: The Poechos Reservoir and Regional Transferability in Northern Peru. Remote Sensing, 18(17), 2901. https://doi.org/10.3390/rs18172901 Its data and model weights are openly archived at https://doi.org/10.5281/zenodo.21253273. The 45 model checkpoints (~8.7 GB) and the training patches are not duplicated here, to avoid a redundant multi-gigabyte re-upload: they are required only to reproduce the computational-cost, calibration and mask-agreement analyses, while the statistical benchmark is fully reproducible from the cross-validation metric files provided in this deposit. See base_study_data/MODEL_WEIGHTS_NOTE.txt. This deposit: https://doi.org/10.5281/zenodo.22210215It supersedes an earlier release of the same analysis package, https://doi.org/10.5281/zenodo.21466955. License: Creative Commons Attribution 4.0 International (CC BY 4.0).