Replication Package for "Architecture-Dependent Behavior in Deep Reinforcement Learning for Equity Statistical Arbitrage: Rolling Walk-Forward Evidence from PPO, A2C, and SAC"

This repository provides the public replication package for the study “Architecture-Dependent Behavior in Deep Reinforcement Learning for Equity Statistical Arbitrage: Rolling Walk-Forward Evidence from PPO, A2C, and SAC.” The study evaluates how different deep reinforcement learning architectures behave in an equity statistical arbitrage setting using a constituent-based U.S. large-cap equity universe, cointegration-based pair selection, engineered reinforcement-learning state variables, transaction-cost-aware rewards, rolling walk-forward validation, trade-level attribution, and VIX-conditioned market-regime analysis. The empirical framework compares three reinforcement-learning algorithms: Proximal Policy Optimization (PPO), Advantage Actor–Critic (A2C), and Soft Actor–Critic (SAC). The models are evaluated under a common statistical arbitrage environment using six rolling walk-forward test windows spanning 2020–2025. The package supports verification of the main empirical workflow, including pair-selection diagnostics, engineered feature summaries, rolling walk-forward model results, trade-level behavior summaries, pair-level reward attribution, market-regime robustness outputs, manuscript figures, manuscript tables, and source code used in the experimental pipeline. Package Contents The public replication package includes: Python scripts for data processing, pair selection, feature engineering, model training, evaluation, and robustness analysis. Cointegrated pair-selection diagnostics. Rolling walk-forward performance results. Trade-level output summaries for PPO, A2C, and SAC. Pair-level reward attribution summaries. VIX-regime robustness summaries. APA-style manuscript tables and figures. Metadata files, run-order documentation, requirements file, and checksums for verification. Bloomberg Data Restriction Notice The original empirical analysis uses market data obtained through Bloomberg. Raw Bloomberg-derived equity data are not redistributed in this public Zenodo archive. This includes, but is not limited to, raw security-level price, volume, market-capitalization, sector-classification, and full engineered datasets constructed directly from licensed Bloomberg data. The following types of files are intentionally excluded from public release: Raw Bloomberg equity data. Full row-level market datasets. Full Bloomberg-derived sector classification files. Full engineered datasets directly reconstructable from Bloomberg data. Any proprietary data that may be restricted under Bloomberg or institutional data-license agreements. Researchers who wish to fully reconstruct the raw data layer must obtain appropriate access to Bloomberg through their own institutional or commercial license. The package provides the relevant data-acquisition and processing scripts so that authorized users can regenerate the restricted files in their own environment. Reproducibility Scope This archive supports result verification and partial computational reproduction. Users can inspect the code, evaluation outputs, trade logs, summary diagnostics, figures, tables, and metadata. Full raw-data reproduction is subject to Bloomberg data-access restrictions and therefore cannot be guaranteed from the public archive alone. Because deep reinforcement learning models can be sensitive to random seeds, software versions, and computational environments, regenerated model outputs may differ slightly from the reported results unless the same configuration, dependencies, data access, and training conditions are used. Suggested Use This replication package is intended for: Verifying the empirical workflow of the study. Inspecting the rolling walk-forward evaluation design. Examining trade-level behavior of PPO, A2C, and SAC. Reproducing summary tables and figures from public outputs. Extending the framework to other equity universes, alternative pair-selection procedures, or additional reinforcement-learning algorithms. Keywords Statistical arbitrage; pair trading; deep reinforcement learning; Proximal Policy Optimization; Advantage Actor–Critic; Soft Actor–Critic; cointegration; rolling walk-forward validation; VIX regimes; market-regime analysis; Bloomberg data; replication package; financial machine learning.本仓库为论文《深度强化学习在股票统计套利中的架构依赖行为：来自PPO、A2C与SAC的滚动窗口前瞻验证证据》的公开复现包。该研究针对股票统计套利场景，基于美国大盘股成分股池、协整法配对选择、人工构造的强化学习状态变量、考虑交易成本的奖励函数、滚动窗口前瞻验证、交易层面归因以及基于波动率指数（Volatility Index, VIX）的市场状态分析，评估不同深度强化学习架构的行为表现。 本实证框架对比了三类强化学习算法：近端策略优化（Proximal Policy Optimization, PPO）、优势演员-评论家（Advantage Actor–Critic, A2C）以及软演员-评论家（Soft Actor–Critic, SAC）。研究在统一的统计套利环境下，采用覆盖2020至2025年的六个滚动窗口前瞻测试窗口对模型进行评估。本复现包支持验证核心实证流程，涵盖配对选择诊断、人工构造特征汇总、滚动窗口前瞻模型结果、交易层面行为汇总、配对层面奖励归因、市场状态鲁棒性输出、论文配图、论文附表以及实验流水线所用的源代码。 复现包内容 本公开复现包包含： - 用于数据处理、配对选择、特征工程、模型训练、评估与鲁棒性分析的Python脚本； - 协整配对选择诊断结果； - 滚动窗口前瞻性能结果； - PPO、A2C与SAC的交易层面输出汇总； - 配对层面奖励归因汇总； - 基于VIX的市场状态鲁棒性汇总； - 美国心理学会（APA）格式的论文附表与配图； - 元数据文件、运行顺序文档、依赖需求文件以及用于校验的校验和文件。 彭博（Bloomberg）数据使用限制声明 本原始实证分析所用的市场数据均通过彭博获取。本公开Zenodo存档未分发原始彭博衍生股票数据，此类数据包括但不限于：原始证券级价格、成交量、市值、行业分类数据，以及直接基于授权彭博数据构造的完整人工构造数据集。 以下类型的文件已被故意排除在公开发布范围之外： - 原始彭博股票数据； - 完整行级市场数据集； - 完整彭博衍生行业分类文件； - 可直接由彭博数据重构的完整人工构造数据集； - 受彭博或机构数据许可协议限制的任何专有数据。 若研究者需完整重构原始数据层，需通过自身所属机构或商业渠道获取彭博数据访问权限。本复现包提供了相关的数据获取与处理脚本，经授权的用户可在自有环境中重新生成受限文件。 可复现性范围 本存档支持结果验证与部分计算复现。用户可查看代码、评估输出、交易日志、汇总诊断结果、配图、附表以及元数据。完整原始数据复现需受彭博数据访问限制约束，因此仅通过本公开存档无法保证完成完整复现。 由于深度强化学习模型对随机种子、软件版本与计算环境较为敏感，若未采用完全一致的配置、依赖项、数据访问权限与训练条件，重新生成的模型输出可能与报告结果存在细微差异。 建议使用场景 本复现包适用于以下场景： 1. 验证本研究的实证流程； 2. 检视滚动窗口前瞻评估设计； 3. 分析PPO、A2C与SAC的交易层面行为； 4. 基于公开输出复现论文附表与配图； 5. 将本框架拓展至其他股票池、替代配对选择方法或更多强化学习算法。 关键词 统计套利；配对交易；深度强化学习；近端策略优化；优势演员-评论家；软演员-评论家；协整；滚动窗口前瞻验证；VIX市场状态；市场状态分析；彭博数据；复现包；金融机器学习