Adaptive Reinforcement Learning for Autonomous Quantum Error Correction in Surface Codes

Quantum error correction (QEC) is essential for fault-tolerant quantum computing in noisy intermediate-scale quantum (NISQ) devices, which exhibit decoherence times $T_1, T_2 \sim \SIrange{50}{100}{\micro\second}$ and gate infidelities of $10^{-2}$ to $10^{-3}$. We introduce \textbf{AQuaRC}, an autonomous QEC framework that integrates deep reinforcement learning (DRL), dynamically scalable rotated surface codes, and a Transformer-based decoder to mitigate errors under non-independent and identically distributed (non-i.i.d.) noise models, such as $1/f$ flux noise with power spectral density $S(\omega) \propto 1/\omega^\alpha$ ($\alpha \approx 1$). Modeled as a partially observable Markov decision process (POMDP) with state space $\mathcal{S} \subseteq \hilbertH^{\otimes d^2}$, action space $\mathcal{A}$ of Pauli corrections, and observation space $\Omega = \{0,1\}^{2(d-1)^2}$, AQuaRC dynamically adjusts code distance $d_t \in \{3,5,\dots,11\}$, syndrome measurement frequency $f_t \in [\SI{1}{\hertz}, \SI{100}{\hertz}]$, and correction policies via a policy $\pi(a_t \mid \hvec_t; \theta)$ based on real-time noise estimates $\hat{p}_t = \norm{\svec_t}_1 / m + 0.1 \norm{\hvec_t - \hvec_{t-1}}_2$, where $m = 2(d-1)^2$ is the number of stabilizers. Monte Carlo simulations ($10^5$ trials, seed=42) under depolarizing noise ($p=10^{-3}$) and $1/f$ noise ($\alpha=1.0$, spatial correlation $\rho \leq 0.9$) achieve a logical error rate $P_L \approx 3 \times 10^{-4}$, outperforming minimum-weight perfect matching (MWPM) baselines ($P_L \approx 1.5 \times 10^{-2}$ at fixed $d=7$) with $\sim 60\%$ fewer qubits (weighted average $\bar{d} \approx 5$, 57 qubits vs. 121) and approximately $87\%$ lower decoding latency (\SI{15}{\micro\second} vs. \SI{120}{\micro\second}). The Transformer decoder ($N=2$ layers, $h=4$ heads, $d_{\text{model}}=128$) handles syndrome histories $\hvec_t \in \{0,1\}^{10 \times m}$ using multi-head self-attention, trained via REINFORCE with discount factor $\gamma=0.99$ and variance-reduced baseline. Theoretical bounds confirm efficacy: $P_L \lesssim \binom{d}{d/2} p^{d/2}$ for i.i.d. noise and $P_L \sim p^{d_{\text{eff}}/2} \exp\left(\kappa \int_0^{\omega_{\max}} S(\omega) \, d\omega\right)$ for correlated noise ($\kappa \approx 0.1$). Comparisons with MWPM, RNN, and GNN decoders highlight AQuaRC's advantages in resource efficiency for platforms like IBM Quantum and IonQ.