Data for study "A Structural Preservation Framework for Denoiser Selection in YOLO-Based Pedestrian Detection under Sensor Noise"

Basic information---------------------------1. Journal article: "A Structural Preservation Framework for Denoiser Selection in YOLO-Based Pedestrian Detection under Sensor Noise" Authors: Vo Thanh Kiet, Rene Jaros, Minh Ly Duc, Petr Bilik, Radek Martinek 2. DOI: concept DOI 10.5281/zenodo.22871454 (resolves to the latest version) this version: 10.5281/zenodo.23726720 3. Contact information Name: Vo Thanh Kiet Institution: VSB - Technical University of Ostrava E-mail: [e-mail] ORCID: https://orcid.org/0009-0002-3278-8755 4. Dataset publication date: the publication date of this version on Zenodo 5. Place of publication: Ostrava, Czechia --------------------------------------------------------------------------------6. Dataset Description================================================================================ This dataset contains the data and code behind the article named above, in the state of its secondrevision. The study asks whether a reference-based measure of structural preservation can tell whichdenoiser should be placed in front of a You Only Look Once (YOLO) pedestrian detector when theincoming frames carry additive Gaussian noise. It introduces the Structural Preservation Score(SPS), a family of three reference-based variants -- edge overlap (Variant A, SPS_edge),gradient-magnitude correlation (Variant B, SPS_grad) and high-frequency energy retention (Variant C)-- and their unweighted combination, and validates them against the detection shift that denoisingcauses: the change in F1 score (Delta-F1) and in mean Average Precision at anintersection-over-union threshold of 0.5 (mAP50) of five YOLO medium detectors (YOLOv8m, YOLOv9m,YOLOv10m, YOLOv11m, YOLOv12m), in two protocols. In the retraining protocol each detector is trainedand evaluated on the processed images of one condition (noise level sigma in 0, 1, 5, 10, 20, 30, in8-bit intensity units, where sigma 0 gives replicate trainings on the clean images and the 25 activecells use sigma 1-30; five denoisers: Gaussian smoothing, Block-Matching and 3D filtering (BM3D),the denoising convolutional neural network DnCNN, a convolutional autoencoder and the sameautoencoder with particle-swarm hyperparameters, CAE+PSO; plus the noisy baseline). In thefrozen-detector protocol five clean-trained detectors are evaluated on the processed images withoutretraining. All experiments use the Person and Person-Like Objects (PnPLO) pedestrian dataset ofKarthika and Chandran (also referred to as the Karthy dataset): 944 training, 160 validation and 235test images, classes person and person-like. Version 3.0 accompanies the second revision. It keeps every data and code file of version 1 (twofolders moved, see "Files" below) and adds the second revision round (folder round2/): a tilingartefact of the three learned denoisers was found and corrected, the affected images wereregenerated, the affected detector runs were repeated, every checkpoint was evaluated on both thetest split (235 images, primary) and the validation split (160 images, sensitivity), and everyimage-level metric was recomputed. The image-level metrics are nine: SPS_edge, SPS_grad, thehigh-frequency similarity hf_sim (Variant C), SPS_combined, the peak signal-to-noise ratio (PSNR),the structural similarity index (SSIM), the gradient magnitude similarity deviation (GMSD), thefeature similarity index (FSIM) and the deep image structure and texture similarity (DISTS). Therecord contains the detection results on both splits of the 180 archived retraining checkpoints(re-evaluated) and of 100 new runs, the training logs of all runs, the frozen-detector results, theper-image and per-cell values of the nine image-level metrics, the statistical outputs behind everytable and figure of the revised article, a checksum manifest of the regenerated denoised images, animportable Python implementation of SPS (source_code/sps.py), the revision notebooks, the analysisscripts of both revision rounds, the first-round notebooks, and the checkpoints of the three learneddenoisers that produced the images. 6.1 What changed since version 1 (2026-09-21) and version 2 (2026-09-29)--------------------------------------------------------------------------------The changes below are stated against the files of version 1; version 2 updated the description ofthe record. Everything below is new in version 3.0 unless stated otherwise. - Tiling artefact of the learned denoisers. The tiled inference of the Autoencoder, DnCNN and CAE+PSO (128-pixel patches, stride 112) left the bottom and right border of most images uncovered, and the original code wrote those pixels as black. Measured on the clean images: 196 of 235 test images (83 %) and 127 of 160 validation images (79 %) have such a band; it covers 7.3 % of the image area on average over all test images (7.1 % over the validation images). Gaussian smoothing and BM3D are not tiled and are unaffected. Correction: the last window is clamped to the border and overlaps are averaged as before; denoising runs in strict fp32. The 15 learned-denoiser cells were regenerated (train, validation and test) from the archived noisy images with the denoiser checkpoints that produced them (checkpoints_denoisers/ver_1/); the band is gone in every image (check T4), and on the area the original tiling covered the regenerated images agree with the archived ones to 0.041-0.812 grey levels (mean absolute difference, 36 images per cell, check T3). - Evaluation split. The archived detection metrics of version 1 (data/detection/) were computed with Ultralytics' default split of the data file, which is the validation split (160 images), while the image metrics were computed on the test split (235 images). Version 3.0 evaluates every checkpoint on both splits; the article reports the test split as primary and the validation split as a sensitivity analysis. - New runs. 100 new retraining runs: 75 runs on the regenerated learned-denoiser sets and 25 same-campaign noisy baselines (sigma 1-30), Ultralytics 8.4.19 (the training version of the archived runs), NVIDIA A100-SXM4-80GB, the batch of the archived counterpart of each run (18 runs with batch 48, 82 runs with batch 64), stop reason early_stop 100, epochs 133-386 (median 215.5), 57.0 GPU-hours in total. The archived runs were trained with Ultralytics 8.4.19 (120 runs), 8.4.20 (39 runs), 8.4.21 (21 runs); the training wall-clock of the 180 archived runs is 106 GPU-hours (round2/image_metrics/timing_and_validation/train_time.json). - Ultralytics versions and the mAP. Ultralytics 8.4.37 changed the AP integral (pull request 24175): with conf = 0.25 the older versions extend the last precision over the recall range the truncated precision-recall curve never reaches and inflate mAP, more on noisy images. The confusion matrix, and therefore F1, is the same in every version. In version 3.0 every mAP comes from Ultralytics 8.4.58: the 180 archived checkpoints were re-evaluated with it (their validation F1 reproduces the archived F1 within 0.01 in 180 of 180 rows), the frozen-detector protocol was re-scored with it, and the new runs were evaluated with it after training. F1 of the new runs comes from the inline evaluation of the training session; in 5 of the 200 new-run rows the 8.4.58 evaluation on an A100 40 GB GPU differs from it by a single detection (rows flagged INVALID_T8, identical in all three sessions; their mAP is used as it is). The per-run JSON files of the new runs carry the pre-fix 8.4.19 mAP of the training session: use the tables instead. - Frozen-detector protocol. The five clean-trained detectors (Ultralytics 8.4.58) were evaluated on 31 conditions and on 15 reference conditions (the archived banded learned-denoiser sets), on both splits, in one session (460 rows). The archived frozen table of version 1 was computed with plots=False, which leaves every confusion-matrix column empty (see data/detection/cross_eval_clean_model.csv below); version 3.0 has F1 for the frozen protocol as well. - Image metrics. SPS, PSNR, SSIM and GMSD were recomputed per image on both splits for the regenerated learned-denoiser sets, the unchanged classical sets, the archived banded sets and the noisy inputs, with a NumPy / SciPy implementation that reproduces the OpenCV pipeline of the first-round notebooks bit for bit (released as source_code/sps.py); FSIM and DISTS were recomputed on Colab with piq 0.8.0 (the first-round library). - Empty training label. noise_1/labels/train/image (835).txt is empty in the stored noisy sigma-1 set (the person-like box of that image is missing: 959 instead of 960 person-like training boxes). The regenerated sigma-1 sets copy the stored labels, so they carry the same empty file (0 bytes in all three manifests), as did the archived noisy sigma-1 runs. Validation and test labels are unaffected. - Image (159). In the first-round per-image table revision_reanalysis/new_metrics/gaussian_filter_s30.csv the row of test image (159).jpg was computed on other pixels than the stored image: GMSD 0.086366 there against 0.080590 recomputed, FSIM 0.844765 against 0.879123, DISTS 0.161199 against 0.160318; the cell mean FSIM moves from 0.849327 to 0.849473. The version-1 file is kept unchanged; the round-2 tables (round2/image_metrics/) are authoritative. - Denoiser checkpoints. Version 1 shipped the checkpoint set dated 1-2 March 2026 and left open which set produced the images. Resolved: the set dated 22-23 February 2026 (ver_1) produced the stored images (check T3 reproduces them from the noisy inputs) and the regenerated ones (sha256-pinned in the revision notebook); it is new in checkpoints_denoisers/ver_1/. The March set (0 of 20 files identical) is a later retraining, kept as checkpoints_denoisers/march_2026_set/ for continuity with the earlier versions. Parameter counts of ver_1: Autoencoder 417,795; DnCNN 558,336; CAE+PSO 1,079,103 / 106,115 / 686,015 / 277,475 / 310,148 at sigma 1 / 5 / 10 / 20 / 30. - Corrections to statements of the version-1 readme: (a) the paragraph on library versions, GPU-hours and training hyperparameters (now recorded; see "Note on library versions and recorded settings" below); (b) data/detection/cross_eval_clean_model.csv has 0 in every confusion-matrix column (plots=False; its mAP columns are valid); (c) data/detection/classification_metrics.csv stores the transposed confusion-matrix reading of the parent pipeline (F1 is unaffected and is the only column this study uses; version 2 of the parent-study record corrected that file); (d) the archived detection metrics were computed on the validation split; (e) the 180-row identity with the parent-study record holds for version 1 of that record (10.5281/zenodo.20842880); (f) the provenance of the denoiser checkpoints (above). - Files: figures/ now holds the figures of the revised article; the version-1 figures moved to figures_round1/. New: round2/, checkpoints_denoisers/ver_1/, source_code/sps.py, source_code/test_sps.py, source_code/sps_selftest_expected.json. readme.txt, LICENSE.txt, source_code/00_PATHS_README.txt and manifest/MANIFEST.csv were rewritten; every other version-1 file is unchanged. 6.2 Notes carried over from version 1 (updated)--------------------------------------------------------------------------------Note on the image data: this package provides the quantitative results, the own-trained denoisercheckpoints and the code, not the underlying images. PnPLO is a third-party dataset distributed onKaggle under the slug "karthika95/pedestrian-detection"(https://www.kaggle.com/datasets/karthika95/pedestrian-detection) and described in Karthika andChandran, "Addressing the False Positives in Pedestrian Detection", Electronic Systems andIntelligent Computing, Lecture Notes in Electrical Engineering vol. 686, Springer Singapore, 2020,pp. 1083-1092 (https://doi.org/10.1007/978-981-15-7031-5_103). Its images and annotations are NOTredistributed here. The noisy and denoised image sets are derivative works of them and are likewisenot redistributed; each is a complete YOLO dataset (data.yaml, images/ and labels/ for 944 training,160 validation and 235 test frames). They can be regenerated from the original images: the noisy andthe classical denoised sets with the pipeline notebook of the parent study (Zenodo record10.5281/zenodo.20842879), the learned-denoiser sets with the revision notebook of this deposit(round2/optionC_kit/) and the checkpoints in checkpoints_denoisers/ver_1/. A regeneratedlearned-denoiser set can be checked file by file against round2/regenerated_image_checksums/ (sizeand SHA-256 of every member of the 15 regenerated zips, 20,085 images). Regenerated noisy images areexpected to match the stored ones only if the noise is drawn in the same order and the JPEG encoderis the same; the revision notebook itself regenerated the learned-denoiser sets from the storednoisy JPEGs. In place of the images the package releases the measurement layer: the per-image tablesof round2/image_metrics/ (second round) and of revision_reanalysis/new_metrics/ (first round). Note on the parent experiment and the earlier Zenodo record: the 180 archived detection runs wereproduced by the pipeline of our earlier study on YOLO noise robustness (Zenodo concept record10.5281/zenodo.20842879). In version 1 of that record (10.5281/zenodo.20842880),results/all_results.csv (180 rows) has SHA-256550d440daabbfea7e5aa1219e48f98fbd2ba3fa779a6562bcaa020fd28fc7f96 andresults/classification_metrics.csv (180 rows) has SHA-256bb82167398444162191fcde96902afa4030f0e948cf9338ca0a534ef995f77c2; every one of those values appearsunchanged in data/detection/all_results.csv and data/detection/classification_metrics.csv here.Version 2 of that record corrected the confusion-matrix orientation of classification_metrics.csv(F1 unchanged), so the byte identity holds for its version 1 only. The two files shipped here arethe superset of the parent grid: all_results.csv carries 323 rows and classification_metrics.csv314, because the parent experiment also trained YOLO26m and three ResNet-backbone YOLOv8 variants,and all_results.csv also carries one clean-baseline row per detector at noise level minus one. Torecover the 180-row grid keep the rows whose model is yolov8m, yolov9m, yolov10m, yolo11m or yolo12mand drop the rows at noise level minus one. Note on the superseded runs and the smoke outputs: as in version 1, only the run of the first-roundanalysis notebook that produced the figures of the first submission (results/run_20260419_135335/)and the shared top-level tables (results/full_exp_shared/, results/e8_cutoff_sweep/) are released;see the version-1 entries below. The smoke-test outputs of the round-2 notebooks and scripts, theinternal audit scripts of the revision and the trainer's plots of the new runs are not included. Note on library versions and recorded settings (corrects version 1): the version-1 readme repeatedvalues from the submitted article (Python 3.10, PyTorch 2.1, Ultralytics 8.2, about 720 GPU-hours,batch size 16, cosine learning-rate decay) as if no artefact recorded them. They are recorded, andseveral differ: the archived runs were trained with Ultralytics 8.4.19-8.4.21(round2/analysis/inputs/s2b_run_completeness.csv, column version) with batch 64 or 48 (same file,column batch), optimizer 'auto', lr0 0.01, lrf 0.01 (linear decay, cos_lr False), momentum 0.937,weight decay 0.0005, 500 epochs with patience 50 (the train_args of the archived checkpoints; thenew runs reproduce them, round2/new_runs/<run>/args.yaml and check T7); the 180 archived runs took106 GPU-hours of training. The versions of the second round (Ultralytics, torch, Python, GPU persession) are in round2/ENVIRONMENT.txt and round2/kit_outputs/env/. The BM3D package version of thefirst round is still not recorded. Two implementation details remain: the high-frequency variantapplies a two-dimensional Hann window before the Fourier transform, and the image-listing helper ofthe first-round notebooks globs lower- and upper-case extensions, which double-counts on acase-insensitive file system (the round-2 code removes the duplicates). Using the code: every path constant of the released notebooks and scripts resolves relative to theroot of this package (or to an image folder the user supplies), so the programs that need only thereleased tables run from a fresh extraction as they stand; see section "How to re-run" below,source_code/00_PATHS_README.txt (first round) and round2/00_ROUND2_PATHS_README.txt (second round),which list every path constant, where each program writes, the exact edits that separate thereleased copies from the files as run, and the released SHA-256 of each. Those edits touch paths,comments and printed text; the few declared code changes of the second round (none changes acomputed value) are listed there with their diff. Some comments of the first-round notebooks are inVietnamese, and so are three docstrings of the denoiser classes in the revision notebook, copiedverbatim from the parent-study notebook. The manifest folder holds a SHA-256 listing of every fileand the script that produced it; run "python make_manifest.py --check" from inside that folder afterdownload and before any re-run (the _rerun/ folder of a re-run is reported as EXTRA; delete itbefore checking). IMPORTANT: Do not modify the file locations within the folder structure. Folder Structure-------------------------------------------------------------------------------- 03_sps_denoiser_selection_dataset (readme.txt, LICENSE.txt and 9 folders; 1,371 files, 105.40 MB = 105,399,382 bytes, in total)|+-- data (335 files, 9.23 MB): measured inputs of the first revision round (version 1, unchanged)| +-- pilot (7 files, 0.07 MB)| +-- detection (5 files, 0.06 MB)| +-- training_curves (323 files, 9.11 MB): per-run training logs, archived runs+-- results (47 files, 1.21 MB): outputs of the first-round analysis notebook (version 1)+-- revision_reanalysis (36 files, 0.53 MB): first revision round (version 1)+-- checkpoints_denoisers (41 files, 59.29 MB)| +-- ver_1 (21 files, 29.69 MB): the set that produced the images (new)| +-- march_2026_set (20 files, 29.60 MB): the later set shipped with version 1+-- source_code (8 files, 1.42 MB): first-round notebooks, sps.py (new), test_sps.py (new)+-- figures (7 files, 0.35 MB): figures of the revised article (new)+-- figures_round1 (6 files, 0.15 MB): figures of version 1 (moved from figures/)+-- round2 (887 files, 32.98 MB): the second revision round (new)| +-- optionC_kit (3 files, 0.37 MB): revision notebook, FSIM/DISTS notebook, release notes| +-- kit_outputs (182 files, 2.94 MB): tables and records of the notebook| +-- new_runs (400 files, 3.16 MB): 100 run folders: training log, args, evaluations| +-- regenerated_image_checksums (32 files, 10.32 MB): 15 manifests + DONE records + 2 CSV| +-- image_metrics (204 files, 12.81 MB)| | +-- sps_psnr_ssim_gmsd (22 files, 10.46 MB): SPS, PSNR, SSIM, GMSD| | +-- fsim_dists (174 files, 2.32 MB): FSIM, DISTS| | +-- timing_and_validation (8 files, 0.03 MB)| +-- analysis (64 files, 3.34 MB): statistics, final tables, figures| +-- outputs (23 files, 1.97 MB)| +-- collect_tables (10 files, 0.70 MB)| +-- figures (11 files, 0.04 MB)| +-- inputs (8 files, 0.49 MB)+-- manifest (2 files, 0.18 MB): MANIFEST.csv and make_manifest.py+-- LICENSE.txt+-- readme.txt Figure 1 of the article is a schematic and Figure 3 a bar chart whose five values are typed into themanuscript; they have no file here. The values of Figure 3 are inround2/analysis/outputs/f4_extras.json (sps/test/hf_ratio_s20_pct). File Descriptions -- second revision round (new in version 3.0)-------------------------------------------------------------------------------- * round2/optionC_kit/ The two notebooks of the second round, as release copies, and their release notes. P3_OptionC_tilefix_retrain_v1.2_release.ipynb regenerates the learned-denoiser outputs with the corrected tiling, retrains and evaluates the detectors on both splits and re-scores the frozen-detector protocol (Google Colab, A100 GPU; stages selected in Cell 1). P3_OptionC_fsim_dists_v1.0_release.ipynb computes FSIM and DISTS. Both need EXPERIMENT_ROOT and PAPER_ROOT set in Cell 1 and stop otherwise. KIT_NOTES_release.txt describes the stages, the integrity checks T1-T8, the deliberate deviations from the original pipeline, what the October 2026 campaign produced and the known issues. * round2/kit_outputs/ The small outputs of the revision notebook (stage collect, session 20261026T143812Z-5447b0) with their inputs. new_runs_eval.csv: 200 rows (100 new runs x 2 splits); F1, TP, FP, FN, TN from the inline evaluation, mAP50 / mAP50_95 / AP50_person / AP50_plike from the 8.4.58 eval_new row (map_source); never use the *_pre_fix columns (8.4.19 mAP). In the five INVALID_T8 rows new_runs_eval.csv leaves the mAP columns blank (map_source 'none yet: run STAGE=eval_new'); their mAP is in new_runs_eval_v58.csv and in round2/analysis/collect_tables/final_new_runs.csv (map_flag FILLED_INVALID_T8). new_runs_eval_v58.csv: the 200 eval_new rows (status OK or INVALID_T8). archived_eval_valtest.csv: 360 rows (180 archived checkpoints x 2 splits, Ultralytics 8.4.58; arch_F1 = the archived value of data/detection/classification_metrics.csv). frozen_valtest.csv: 460 rows (5 detectors x 46 conditions x 2 splits; stage B1_clean / B2_noisy / B3_denoised; image_set regen_tilefix, archived or archived_banded). runs_done.csv: the 100 new runs (epochs, best epoch, stop reason, batch and the archived batch, GPU, versions, best.pt sha256). regen_check.csv and regen_t3_perimage.csv: checks T3 / T4 of the 15 regenerated cells. canary_T5*.json: check T5. bridge_offset.json: check T6. RUN_SUMMARY.md: the summary the notebook wrote. parts/: the per-session tables the collect stage merges. env/: one environment record per GPU session (17: versions, GPU, pip freeze; the collect session wrote a log only). logs/: one log per session (18; the logs of the train sessions print the inline, pre-fix 8.4.19 mAP). invalid/: the three INVALID_T8 records. runs_done/: the DONE record of every new run (its eval block carries the pre-fix mAP). Column dataset holds the image folder of a row as <EXPERIMENT_ROOT>/...; see round2/00_ROUND2_PATHS_README.txt. * round2/new_runs/<run>/ One folder per new run (100): results.csv (the Ultralytics training log, one row per epoch), args.yaml (the training arguments), eval_val.json and eval_test.json (the inline evaluation of the training session, Ultralytics 8.4.19: confusion matrix and F1 final, mAP pre-fix). The DONE record of each run is round2/kit_outputs/runs_done/<run>.json. best.pt, the trainer's plots and image mosaics and predictions.json are not included. * round2/regenerated_image_checksums/ For each of the 15 regenerated learned-denoiser cells (<method>_noise_<sigma>): <cell>.manifest.json (size and SHA-256 of each of the 2,679 zip members: images, labels and data.yaml of the three splits, and the zip's own SHA-256) and <cell>.DONE.json (zip size and SHA-256, the results of checks T2-T4, versions). regenerated_images_sha256.csv lists every member in one table (columns cell, zip, member, bytes, sha256); regenerated_zips_sha256.csv lists the 15 zips (cell, zip, zip_bytes, zip_sha256, n_members). The noisy and the classical denoised sets were not regenerated and have no checksum here. * round2/image_metrics/sps_psnr_ssim_gmsd/ Per-image and per-cell values of SPS_edge, SPS_grad, the Variant C ratio (cutoff r_c = 0.5 r_max plus 0.3, 0.4, 0.6, 0.7), PSNR, SSIM and GMSD. metrics_perimage_test.csv (7,285 rows) and metrics_perimage_val.csv (7,200 rows): one row per image pair; columns split, source (regen = regenerated learned-denoiser images, arch = archived images: the classical cells and the banded learned cells, noisy = the noisy input), method, sigma, image, the image size and band geometry (H, W, Hc, Wc, band_frac), the metrics on the full image (sps_edge, sps_grad, hf_ratio, hf_r03 ... hf_r07, psnr, ssim, gmsd) and on the area the original tiling covered (cov_*), band diagnostics, the pixel comparison with the archived image of the same cell (ref_*, regenerated rows only) and the time per metric (t_*). cells_25_<split>.csv: the 25 active cells of the article (learned = regenerated, classical = archived): mean, SD (ddof 1), SD (ddof 0), median and number of valid images per metric, hf_sim (of the cell-mean ratio), combined, gmsd_neg, the cutoff grid. cells_all_<split>.csv: every computed cell. compare_learned_<split>.csv: archived (full and covered area) against regenerated learned cells. noisy_cells.csv, cutoff_grid_<split>.csv (Figure 2 inputs), perimg_sigma20_sps_grad_test.csv (Figure 4 input), timing_summary.json, summary.json (gates of the recomputation: the archived test cells reproduce the published values within 1e-6), cv2_crosscheck.json (OpenCV 4.11 against the NumPy code: all differences 0). Scripts: sps_regen.py (computes the per-image rows; needs the images), aggregate_sps.py (aggregates them), cv2_crosscheck.py, check_cells_from_perimage.py (recomputes the cell tables from the per-image tables; runs from the package alone). inputs/: two tables of the first-round values that aggregate_sps.py compares against. run_timing/: the single-process timing pass. * round2/image_metrics/fsim_dists/ FSIM and DISTS (piq 0.8.0, Colab session 20261026T163414Z-7cf1bf; GMSD recomputed as a provenance check). cells_25_fsim_dists_<split>.csv: the 25 cells (learned = regenerated, classical = archived); *_banded.csv: the same with the archived banded learned cells; cells_fsim_dists_all_units.csv: every unit; perimg/: 80 per-image tables (<split>__<source>__<method>_s<sigma>.csv, columns image, gmsd, fsim, dists) with their done records; gate_*.json, _canary/: the gate against the first-round values; ref/gmsd_numpy_reference.csv: the NumPy GMSD per image (written by build_ref.py from the per-image tables of the folder above); env/, logs/, DONE.json. * round2/image_metrics/timing_and_validation/ timing.py / timing.json and timing_total.py / timing_total.json: measured time of each metric per image pair (BM3D sigma 20, test images, one CPU thread); validate_extraction.py / .json: the validation of the NumPy metric code against the archived test values (the grey conversion variant "15" reproduces them); train_time.py / .json: training wall-clock of the archived runs from data/training_curves/. * round2/analysis/ The statistics of the revised article. run_all.py runs f1_load (tidy detection tables and data checks), f2_analysis (cell shifts, run-to-run reference, campaign offset, correlations with bootstrap intervals, Holm / Benjamini-Hochberg, Meng-Rosenthal-Rubin tests, leave-one-method-out cross-validation, cluster bootstrap, mixed model, ANOVA at every noise level, frozen protocol), f3_sens (sensitivity designs), f4_extras (descriptive values), f6_tables (final_tables.md), f7_keynumbers (key_numbers.md) and the extras x01 (robustness of the Meng-Rosenthal-Rubin result over 21 variants), x04 (band effect on FSIM / DISTS / GMSD) and x02 (FSIM / DISTS on the banded images); f0_common.py and f_stats.py hold the paths, conventions and statistics helpers. outputs/: the 23 outputs, rebuilt by the released code from the files of this package (byte-identical to a re-run from this package with the versions in round2/ENVIRONMENT.txt; identical to the outputs as run up to the declared path and text substitutions, see round2/00_ROUND2_PATHS_README.txt, section 6). final_tables.md lists every number of the article's results with its source; key_numbers.md is a compact version. * round2/analysis/collect_tables/ build_final_tables.py writes the four tidy tables of the paper: final_new_runs.csv (200 rows; F1 / TP / FP / FN / TN from the inline evaluation, mAP from eval_new, map_flag OK or FILLED_INVALID_T8), final_archived_runs.csv (360 rows), final_frozen.csv (460 rows) and final_image_metrics_cells.csv (50 rows: the nine image metrics per cell and split, FSIM / DISTS from fsim_dists/), with final_tables_manifest.json (SHA-256 of every input and output, every assertion). analyze_eval_new.py writes eval_new_checks.json (inline vs eval_new vs the per-run records), prefix_vs_postfix_by_group.csv (mAP before and after the AP fix) and bridge_noisy_vs_archived_noisy.csv (campaign offset of mAP). * round2/analysis/figures/ make_figures_D.py draws the six data figures of the revised article (figures/ of this package) from the analysis outputs and the image-metric tables, asserting every plotted statistic against the outputs; the CSV files are the data of each figure; manifest_sha256.csv lists the files one run writes, with paths relative to the script's output folder as written on the build machine (Windows separators): out\ holds the files of figures/ of this package, png\ further PNG renders and options\ two renders of an alternative layout of Figure 2; the png\ and options\ files are not released. * round2/analysis/inputs/ s2b_run_completeness.csv (archived runs: epochs, best epochs, whether the run stopped early, batch, Ultralytics version), t8_check.json and t8_check.py (the INVALID_T8 rows across the three eval_new sessions), archived_banded_val_cells.csv (the seven image metrics on the archived banded validation images) and provisional/ (four tables of an earlier pipeline of this revision; f1_load.py checks that the final numbers equal them). * round2/00_ROUND2_PATHS_README.txt, round2/ENVIRONMENT.txt The path note of the second round (roots, environment variables, re-runnability, the per-file edit list with the released SHA-256, the declared code changes, the path tokens and the checks of the build) and the software and hardware of every session. * checkpoints_denoisers/ver_1/ The 15 denoiser checkpoints (autoencoder_sigma<N>.pt, dncnn_sigma<N>.pt, cae_pso_sigma<N>.pt) and the five cae_pso_sigma<N>_params.yaml of the set that produced the images, plus cae_pso_params_decoded.csv: 21 files, 29.69 MB. Plain PyTorch state dictionaries saved from a CUDA device: load with torch.load(path, map_location="cpu", weights_only=True) on a machine without a GPU; the classes are in Cell 5 of the revision notebook. The params.yaml files store the learning rate as a NumPy scalar tag that only an unsafe YAML loader reads; cae_pso_params_decoded.csv gives base_filters, kernel_size and lr decoded without evaluating the YAML. * checkpoints_denoisers/march_2026_set/ The 20 files that version 1 shipped in checkpoints_denoisers/ (dated 1-2 March 2026), 29.60 MB, unchanged; a later retraining, not the set that produced the images. Twelve of them are also part of the companion deposit of our frozen-detector study of modern denoisers. In this set the CAE+PSO parameter files store lr as a plain number; the .pt files load as those of ver_1 (map_location="cpu" without a GPU). * source_code/sps.py, source_code/test_sps.py, source_code/sps_selftest_expected.json The importable SPS module (NumPy, SciPy, Pillow; scikit-image for PSNR / SSIM; OpenCV optional): load_rgb, to_gray, canny, sobel_magnitude, sps_edge, sps_grad, hf_ratio, hf_sim, sps_combined, psnr, ssim, gmsd, score_pair (every per-image value, keys as in metrics_perimage_*.csv), score_cell (a folder pair, conventions of cells_25_*.csv) and a command line (python sps.py CLEAN_DIR DENOISED_DIR [--out per_image.csv]). The metric functions are those the round-2 values were computed with, copied byte for byte; on real images it reproduces the released per-image values (round2/00_ROUND2_PATHS_README.txt, section 6). gray_cv keeps the default grey-conversion variant "14" of the as-run code, whereas to_gray, score_pair and score_cell use variant "15", which reproduces the released values; call gray_cv(rgb, "15") or to_gray(rgb) when using gray_cv directly. test_sps.py checks an installation on synthetic images against sps_selftest_expected.json. * figures/ The data figures of the revised article: fig_02_cutoff_sensitivity.pdf, fig_04_perimg_violin.pdf, fig_05_per_yolo_version.pdf, fig_06_vs_classical.pdf, fig_07_loocv.pdf, fig_08_infer_only.pdf; fig_08 also as a PNG preview. Drawn by round2/analysis/figures/make_figures_D.py (all seven files byte-identical to a re-run of it from this package with the versions in round2/ENVIRONMENT.txt). * figures_round1/ The figure files of version 1 (the first revised manuscript): fig_02_cutoff_sensitivity.pdf, fig_04_perimg_violin.pdf, fig_05_per_yolo_version.pdf, fig_06_vs_classical.pdf, fig_07_loocv.pdf, fig_08_infer_only.pdf. Five of them are byte-identical to their copies in results/run_20260419_135335/. File Descriptions -- first revision round (version 1, unchanged files; corrected statements)-------------------------------------------------------------------------------- * data/pilot/ pilot_all_variants.csv (30 rows = 5 methods x 6 noise levels; columns method, sigma, sps_edge_mean, sps_grad_mean, sps_hf_mean, delta_f1_mean, delta_f1_std, sps_hf_sim, sps_combined), the three component tables pilot_sps_edge.csv, pilot_sps_grad.csv, pilot_sps_hf.csv (standard deviations with divisor n), pilot_delta_f1_v2.csv (the first-round detection-shift target, from the validation split), pilot_v2_summary.json and pilot_v2_variants_scatter.pdf. These are the first-round values (archived banded learned-denoiser images, validation F1); the second-round equivalents are round2/image_metrics/ and round2/analysis/. The rows at sigma = 0 are placeholders (no denoised set at sigma = 0). * data/detection/classification_metrics.csv The classification metrics of every training run of the parent grid, 314 rows (columns model, noise_sigma, denoise_method, Accuracy, F1-Score, Sensitivity, Specificity, Precision, TP, FP, FN, TN), computed on the validation split (160 images) with conf 0.25, iou 0.5. Correction: the TP / FP / FN / TN, Sensitivity, Specificity, Precision and Accuracy columns follow the transposed reading of the Ultralytics confusion matrix of the parent pipeline (its FP is the true FN and vice versa); F1 is unaffected. This study uses F1 only. Version 2 of the parent-study record ships the corrected file. At noise level zero the noisy / gaussian_filter rows and the autoencoder / dncnn rows are identical for every medium detector (ten pairs) although their training logs differ; no analysis uses the sigma = 0 cells. * data/detection/all_results.csv The detection metrics of the same runs, 323 rows (model, noise_sigma, denoise_method, mAP50, mAP50-95, Precision, Recall, Composite), from the trainer's validation (pre-fix mAP of the parent pipeline); not used by the analysis; includes one clean row per detector at noise level minus one. * data/detection/cross_eval_clean_model.csv The archived frozen-detector grid, 192 rows (6 detectors x 32 cells: B1_clean, B2_noisy, B3_denoised), validation split, Ultralytics 8.4.58. Correction: it was computed with plots=False, so every confusion-matrix column (Accuracy, F1-Score, Sensitivity, Specificity, Precision, TP, FP, FN, TN) is 0 in all rows; the mAP50 and mAP50-95 columns are valid. Its B3_denoised learned-denoiser rows are on the archived banded images. The second-round frozen table is round2/kit_outputs/frozen_valtest.csv. * data/detection/origin_baseline_metrics.csv, experiment_meta.json Clean baselines of the five medium detectors and the description the parent experiment wrote for itself (its model list predates the ResNet-backbone additions). * data/training_curves/ 323 per-run training logs of the parent grid (Ultralytics results.csv), 9.11 MB: <detector>_<method>_noise_<sigma>_results.csv and <detector>_clean_results.csv; the latter include the five clean-trained detectors of the frozen protocol. One row per epoch actually run. * results/run_20260419_135335/, results/full_exp_shared/, results/e8_cutoff_sweep/ The outputs of the first-round analysis notebook (e3 PSNR / SSIM, e4 / e6 cross-validation, e5 per-image SPS_grad at sigma 20, e7 per-detector, e8 cutoff sweep, summaries and the figure exports of the first submission); unchanged. They were computed on the archived banded learned-denoiser images and the validation-split F1. * revision_reanalysis/ The first revision round: stats_reanalysis.py and its outputs (gate, multiplicity corrections, dependent-correlation tests, cluster bootstrap, mixed model, ANOVA, frozen-detector reduction), infer_only_25cells.csv and infer_only_125obs.csv (frozen protocol, first round), new_metrics_gmsd_fsim_dists.py and new_metrics/ (per-image GMSD, FSIM, DISTS on the test split; see the image (159) note above), new_metrics_correlate.py, loocv_verify.py and make_fig_infer_only.py; unchanged. * source_code/ (first round) full_experiments_final.ipynb (the first-round analysis notebook), full_experiments__4_fixed.ipynb and full_experiments__3_fixed.ipynb (earlier builds; the latter is the only copy of the cutoff-sweep computation), pilot_generator/pilot_sps_v3_fixed.ipynb (generator of data/pilot) and 00_PATHS_README.txt (how they find their inputs and what was edited for release); unchanged except 00_PATHS_README.txt (two notes added). * manifest/MANIFEST.csv, manifest/make_manifest.py Path, size in bytes and SHA-256 of every file (1,370 rows), and the standard-library script that writes it; "python make_manifest.py --check" verifies a downloaded copy (run it before any re-run, or delete the _rerun/ folder first, which it would report as EXTRA). * Names in the files gaussian_filter, bm3d, autoencoder, dncnn and cae_pso are the article's Gaussian smoothing, BM3D, Autoencoder, DnCNN and CAE+PSO; noisy is the noisy baseline; yolo11m and yolo12m are the article's YOLOv11m and YOLOv12m. noise_sigma = -1 (method clean) is the clean condition, distinct from noise_sigma = 0. In the round-2 tables split is test or val, and source / image_set say which images a value was computed on: regen / regen_tilefix = the regenerated learned-denoiser images (corrected tiling), arch / archived = the stored images (classical methods unchanged; learned methods banded), archived_banded = the archived learned-denoiser images used as reference conditions of the frozen protocol. Stages of the frozen protocol: B1_clean, B2_noisy, B3_denoised. Labels of the parent study and of the working folders: P2 is the parent study (Zenodo 10.5281/zenodo.20842879); "P2 notebook", "P2 Final_Ver_4" and "4STEP Ver_9" are its pipeline notebook and two of its versions, and cell numbers next to them refer to that notebook (for example the note "P2 cell 21 convention" in final_archived_runs.csv); ver_1, ver_6 and ver_7 are folders of the parent experiment (ver_1 holds the denoiser checkpoints released in checkpoints_denoisers/ver_1/, ver_7 and ver_6 the archived detector checkpoints, ver_7 also the archived tables of data/detection/); a5 (A5, --a5) is round2/image_metrics/sps_psnr_ssim_gmsd/; s1 is the provisional pipeline whose tables are in round2/analysis/inputs/provisional/. Table and figure provenance (revised article)-------------------------------------------------------------------------------- Table 4 (baseline F1) round2/analysis/outputs/f4_table4.csv, f4_extras.json (table4) Table 5 (correlations) f2_results.json retrain_corr: dF1|test|primary and dmap50|test|primary; cells: f2_cells_retrain.csv; image metrics: cells_25_test.csv Tables 6-7 (cross-validation) f2_results.json retrain_corr .../loocv Table 8 (frozen protocol) f2_frozen_cells.csv, f2_results.json frozen Table 9 (ANOVA) f2_results.json anova Table 10 (paired t-tests) f2_results.json anova .../paired_t_dF1 Figure 2 f4_cutoff_grid.csv -> fig_02_cutoff_sensitivity.pdf Figure 3 (typed values) f4_extras.json sps/test/hf_ratio_s20_pct Figure 4 perimg_sigma20_sps_grad_test.csv -> fig_04_perimg_violin.pdf Figure 5 f2_results.json retrain_corr .../per_detector, f1_perdet.csv -> fig_05_per_yolo_version.pdf Figure 6 cells_25_test.csv, f2_cells_retrain.csv -> fig_06_vs_classical.pdf Figure 7 f2_results.json retrain_corr .../loocv -> fig_07_loocv.pdf Figure 8 f2_frozen_cells.csv, f2_results.json frozen -> fig_08_infer_only.pdf final_tables.md (sections T0-T8) prints every number of these tables and figures with its source. How to re-run--------------------------------------------------------------------------------From the package alone (CPU; Python 3 with numpy, scipy, pandas, statsmodels, scikit-learn,matplotlib, scikit-image, Pillow; the versions of the build are in round2/ENVIRONMENT.txt): "pythonround2/analysis/run_all.py" writes every output of round2/analysis/outputs/ into _rerun/analysis/ atthe package root (about 23 minutes on the build machine; f2_analysis.py is the slow step); "pythonround2/analysis/figures/make_figures_D.py" draws the figures into _rerun/figures/out/ (their dataCSVs and manifest_sha256.csv into _rerun/figures/); build_final_tables.py, analyze_eval_new.py,check_cells_from_perimage.py, train_time.py and source_code/test_sps.py run as they stand, whilet8_check.py takes the kit outputs folder ("python round2/analysis/inputs/t8_check.pyround2/kit_outputs") and build_ref.py an output file (for example_rerun/build_ref/gmsd_numpy_reference.csv). Set SPS_OUT to write elsewhere and SPS_PKG_ROOT to run ascript from outside the package. No round-2 script writes into a released folder, and neither doessource_code/test_sps.py run without options; the first-round scripts in revision_reanalysis/ writebeside themselves and overwrite the released copies (source_code/00_PATHS_README.txt), so copy thatfolder before running them, and manifest/make_manifest.py run without --check rewritesmanifest/MANIFEST.csv. Run python with -B so that no __pycache__ folder is written into the package(make_manifest.py ignores such folders). With the images (set SPS_IMAGE_ROOT to a folder with dataset_yolo/images/<split>/ andyolo_denoise_experiment/{noisy,denoised}_datasets/, and SPS_REGEN_ROOT to the regenerated zips):sps_regen.py (then aggregate_sps.py), cv2_crosscheck.py and the timing / validation scripts. OnGoogle Colab (GPU): the FSIM / DISTS notebook and the revision notebook (regen and train stages;eval_old, frozen and eval_new also need the detector checkpoints, which are available on request).The first-round notebooks and scripts are described in source_code/00_PATHS_README.txt. What this package does not contain-------------------------------------------------------------------------------- - Image data from the dataset: no original, noisy, denoised or regenerated frame and none of the image mosaics or prediction files the YOLO trainer writes (see the note on the image data). - The trained YOLO detector checkpoints: the 180 archived retraining runs (8.74 GB), one intact substitute copy of a truncated archived checkpoint (40.54 MB), the 100 new runs (4.15 GB) and the 5 clean-trained detectors of the frozen protocol (0.21 GB), together 13.14 GB (13,142,529,903 bytes); Ultralytics-produced with the AGPL-3.0 string in their metadata; available from the corresponding authors on reasonable request. - The pipeline notebook of the parent study (noise injection, the original denoising with the uncorrected tiling, training of the archived detectors): Zenodo record 10.5281/zenodo.20842879. - The COCO-pretrained YOLO weights and the DISTS weights that piq downloads at run time. - Smoke-test outputs, the internal audit scripts of the revision (they compare numbers with the manuscript and the response letter), the superseded executions of the first-round notebook and the correspondence of the review. -------------------------------------------------------------------------------- 7. Funding The authors are grateful to VSB - Technical University of Ostrava, Czechia, and Van Lang University, Vietnam, for their expertise and support throughout this research. This article was funded by the European Union under the REFRESH - Research Excellence For REgion Sustainability and High-tech Industries project number CZ.10.03.01/00/22_003/0000048 via the Operational Programme Just Transition. This article was funded by the Ministry of Education of the Czechia (Project No. SP2026/012, SP2026/075). The views and conclusions in this article are entirely those of the authors and do not necessarily reflect the views of the funding organizations. 8. Licence and reuse Everything in this archive is released under the Creative Commons Attribution 4.0 International licence (CC BY 4.0), https://creativecommons.org/licenses/by/4.0/. LICENSE.txt in the archive root is authoritative: one licence tier covers everything in the archive, and LICENSE.txt lists separately what is not redistributed and therefore not licensed by us (all image data, the trained detector checkpoints, the parent-study pipeline notebook, pretrained and downloaded weights). One file installs and imports the AGPL-3.0 licensed Ultralytics toolkit, the revision notebook round2/optionC_kit/P3_OptionC_tilefix_retrain_v1.2_release.ipynb; see LICENSE.txt. This file states what we believe the licensing position to be; it is not legal advice. Last reviewed: 2026-10-27.