Matched-Wait evaluation of retrieval-augmented transaction timing under Ethereum EIP-1559: Benchmarking kNN Analogs, Trailing-Rank Heuristics, and 7B Language-Model Planners

This repository contains the complete computational replication package, benchmark datasets, empirical evaluation logs, and diagnostic analysis code accompanying our manuscript submitted to PeerJ Computer Science: "Matched-Wait Evaluation of Retrieval-Augmented Transaction Timing under Ethereum EIP-1559: Benchmarking kNN Analogs, Trailing-Rank Heuristics, and 7B Language-Model Planners". ### AbstractBackground: Execution-timing decisions, such as deferring an Ethereum transaction within a short deadline, present an operational testbed for retrieval-augmented sequential decision making. An autonomous agent can query similar historical market states and act upon subsequent fee dynamics. However, whether analogue retrieval provides alpha beyond simple trailing-price heuristics once policies are evaluated at identical average waiting budgets remains largely untested. Methods: We evaluated 316,800 contiguous Ethereum mainnet blocks (2024-01-01 to 2024-02-14 UTC; BigQuery public datasets) partitioned chronologically into library (LIB), calibration (CAL), and holdout test (TEST) windows. A swap transaction arriving at block t0 could defer execution up to a deadline of D = 50 blocks (10.0 minutes) and paid the base fee of its submission block plus a fixed 1.5 gwei tip (150,000 gas). We benchmarked submit-immediately, random-wait, a trailing percentile-rank heuristic, non-parametric kNN retrieval over causal state features, an ex-post oracle, and an open-weight 7B language model (qwen2.5:7b) with and without retrieved context. Heuristic and retrieval thresholds were matched to equal mean wait, and uncertainty was quantified using window-clustered bootstrap percentile intervals (3,000 resamples). Results: On the calibration split, retrieved historical neighbors carried genuine predictive signal regarding achievable savings beyond trailing rank (incremental correlation r = +0.227, 95% cluster CI [+0.167, +0.267]). On holdout test windows, unconstrained kNN retrieval saved 5.25% [3.19, 7.31] but expanded wait to 261s, while the heuristic saved 4.61% [3.78, 5.40] at 165s. When deconfounded across the common support frontier (60s to 400s), the mean Pareto frontier gap was Delta = -0.14 pp (95% CI [-0.70, +0.46]). Because the confidence interval half-width (+/-0.58 pp) exceeds our pre-specified economic equivalence margin of +/-0.40 pp, the comparison is statistically inconclusive rather than an established tie. The 7B language model failed to outperform matched-wait baselines across all prompt configurations (retrieval vs. grid-matched heuristic: Delta = -1.05 pp [-2.61, +0.40]) and exhibited severe wording fragility on calibration ablations. Conclusions: Although historical state retrieval captures non-redundant predictive signal, we found no statistical evidence that it improves the fee–wait frontier over a matched-wait trailing-rank heuristic. Apparent performance gains in unconstrained retrieval and LLM agents are confounded by waiting budget differences. ### Archive Contents- Evaluation logs and reconciled outcome tables (Table 2 to Table 5)- Frontier sweep data, landmark diagnostics, and deconfounding scripts- Python scripts and environment lockfiles for full algorithmic reproducibility