Temporal-Resilience Reinforcement Learning for Autonomous Cyber Defense: Adversarial Red\u2013Blue Co-Evolution and SOC-Aligned Evalu

Abstract\u2014Reinforcement learning (RL) has emerged as apromising foundation for autonomous cyber defense, enablingadaptive decision-making under dynamic and adversarial conditions.However, most existing RL-based defense systems areevaluated using cumulative reward, accuracy, or episode-levelsuccess metrics that fail to capture operationally critical temporalproperties such as detection latency and recovery time. Thisdisconnect limits the relevance of RL evaluation to real-worldsecurity operations centers (SOCs), where Mean Time to Detect(MTTD) and Mean Time to Recover (MTTR) are primaryindicators of defensive effectiveness.This paper introduces a temporal-resilience evaluation frameworkfor autonomous cyber defense that integrates adversarialRed-Blue co-evolution, structured episode-timeline logging, andSOC-aligned temporal metrics. A lightweight cyber-range environmentis developed to support scalable adversarial interactionwhile preserving uncertainty, visibility dynamics, and recoveryprocesses essential for resilience measurement. Temporal metricsare computed directly from time-indexed logs, enabling reproducibleevaluation of detection latency, recovery latency, and anepisode-level Resilience Index.Experiments across 200 training episodes demonstrate thatRL design choices\u2014including exploration scheduling, rewardshaping, and policy stochasticity\u2014materially influence temporalresilience outcomes. Controlled ablation studies show that rewardformulations explicitly encoding detection and recovery objectivesyield significantly lower MTTD and MTTR, while robustpolicy execution improves resilience stability under uncertainty.The results highlight the necessity of temporal-resilience\u2013alignedevaluation for advancing autonomous cyber defense towardoperational deployment.Index Terms\u2014Autonomous cyber defense, reinforcement learning,cyber resilience, adversarial learning, security operations,MTTD, MTTR.I.