Data for Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness

IntroductionThis is the dataset for the paper titled ‘Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness’ that has been accepted for publication at IEEE INFOCOM 2024. The paper’s accepted version will be available following publication in May 2024 at https://sussex.figshare.com/articles/conference_contribution/Reinforcement_learningbased_congestion_control_a_systematic_evaluation_of_fairness_efficiency_and_responsiveness/24711033. The dataset is meant to be used in conjunction with the codebase that is also made available at https://doi.org/10.25377/sussex.24978162.However, the dataset itself is of value to researchers as it contains an extensive set of metrics captured during experimentation with Reinforcement Learning-based Congestion control as discussed in the ‘Experimental Evaluation’ section of the paper. Our study is the result of a 160-hour long experimentation during which 1950 Orca, Aurora and TCP Cubic flows were measured. We have collected approximately 500GB of data encompassing diverse metrics related to network interfaces (e.g., utilisation, retransmissions, packet drops), CPU and memory parameters (such as CPU load and memory usage), as well as the data transport layer (e.g., congestion window, round trip time). Reinforcement learning (RL)-based congestion control (CC) promises efficient CC in a fast-changing networking landscape, where evolving communication technologies, applications and traffic workloads pose severe challenges to human-derived, static CC algorithms. RL-based CC is in its early days and substantial research is required to understand existing limitations, identify research challenges and, eventually, yield deployable solutions for real-world networks. In this paper we present the first reproducible and systematic study of RL-based CC with the aim to highlight strengths and uncover fundamental limitations of the state-of-the-art. We identify challenges in evaluating RL-based CC, establish a methodology for studying said approaches and perform large-scale experimentation with RL-based CC approaches that are publicly available. We show that existing approaches can acquire all available bandwidth swiftly and are resistant to non-congestive loss, however, this is commonly at the cost of excessive packet loss in normal operation. We show that, as fairness is not embedded directly into reward functions, existing approaches exhibit unfairness in almost all tested network setups. Finally, we provide evidence that existing RL-based CC approaches under-perform when the available bandwidth and end-to-end latency dynamically change. Our experimentation codebase and datasets are publicly available with the aim to galvanise the community towards transparency and reproducibility, which have been recognised as crucial for researching and evaluating machine-generated policies.The datasetThe dataset contains an extensive set of metrics captured during experimentation. It is composed of eight different experiments. Please refer to the paper for a detailed explanation of the experimental set-up. Note that some experiments are the source of more than one plot in the paper. Refer to the codebase for the relationship between plots and experiments.The data from each experiment is organised into multiple folders and files. Just under the root folder, the data is divided by the type of packet scheduling adopted by the bottleneck queue, i.e. fifo, codel, fq, fq-codel.The next folder contains all variations of bottleneck bandwidth, propagation delay, and buffer size for the same experiment. Each of the folders is named following the same pattern:{topology}_{bottleneck_bandwidth}mbps_{one_way_delay}ms_{buffer_size}pkts_{loss_rate}loss_{number_of_flows}flows_22tcpbuf_{protocol}For example: Dumbell_100mbit_80ms_279pkts_0loss_2flows_22tcpbuf_orca contains all the runs of two flows in a dumbbell topology with 100mbps bottlenck bandwidth, 160ms rtt and a buffer size of 279 MSS. Note that the semantic meaning of {protocol} depends on the specific experimental setup. In the intra-protocol fairness experiment, all flows will be {protocol}, whereas in the friendliness experiment, one flow will always be cubic and one {protocol}.Under the experiment variation folder, multiple folders contain different runs of the same variation. Depending on the experiment, a different seed may be used. However, in most of the cases, all runs are identical.Finally, each run folder contains all the raw data captured during the experiment, plus some processed data.The files and folders contained in each run folder are:tcp_probe.txt: The raw output of tcp_probe modulex[N]_output.txt: The raw output of the Nth server applicationc[N]_output.txt: The raw output of the Nth client application.emulation_info.json: JSON representation of all flows in the experiment. Each flow is represented by the source and destination IP addresses, the protocol used, and the starting time.queues (folder): Contains a text file for each of the bottleneck queue. Each file contains the output of ‘tc -s qdisc show dev’ applied to the dev in the filename.sysstat (folder): Contains the raw binary files (datafile_*.log) generated with sysstat. It also contains some log files generated using sadf processing utility on the binary datafiles. ]csvs (folder): contains processed data in csv format.<br>

### 引言
本数据集对应已被IEEE INFOCOM 2024接收的论文《基于强化学习的拥塞控制：公平性、效率与响应性的系统评估》（Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness）。论文终稿将于2024年5月发表后，通过以下链接公开：https://sussex.figshare.com/articles/conference_contribution/Reinforcement_learningbased_congestion_control_a_systematic_evaluation_of_fairness_efficiency_and_responsiveness/24711033。本数据集需与配套代码库配合使用，代码库的公开链接为：https://doi.org/10.25377/sussex.24978162。

尽管如此，本数据集本身即具备独立研究价值：其包含了论文“实验评估”章节中所述的、基于强化学习的拥塞控制（Reinforcement Learning-based Congestion Control, RL-based CC）实验过程中采集的多维度指标。本研究历时160小时完成大规模实验，共采集了1950条Orca、Aurora与TCP Cubic流的相关数据，总数据量约500GB，涵盖网络接口（如利用率、重传次数、丢包数）、CPU与内存参数（如CPU负载、内存使用率）以及数据传输层（如拥塞窗口、往返时延（Round Trip Time, RTT））等多类指标。

基于强化学习的拥塞控制（RL-based CC）有望在快速迭代的网络环境中实现高效的拥塞控制：当前演进中的通信技术、应用形态与流量负载，对人工设计的静态拥塞控制算法提出了严峻挑战。目前RL-based CC仍处于早期发展阶段，亟需开展大量研究以明晰其现有局限、明确研究方向，并最终产出可落地的真实网络部署方案。

本论文首次针对RL-based CC开展可复现的系统性研究，旨在揭示现有方案的优势与核心局限。研究团队识别了RL-based CC评估过程中面临的挑战，建立了一套标准化的研究方法，并对公开可用的RL-based CC方案开展了大规模实验验证。实验结果表明，现有RL-based CC方案可快速抢占全部可用带宽资源，且具备抗非拥塞丢包的能力，但这一优势通常以正常运行时的过度丢包为代价；同时，由于公平性未直接嵌入奖励函数，现有方案在几乎所有测试的网络配置下均存在公平性缺失问题；此外，当可用带宽与端到端时延动态变化时，现有RL-based CC方案的性能会显著下降。本研究的实验代码库与数据集均已公开，旨在推动机器学习生成策略研究与评估领域的社区提升研究透明度与可复现性——这两点已被证实为相关研究的核心关键。

### 数据集说明
本数据集包含实验过程中采集的多维度指标，共涵盖8组独立实验。实验的详细配置请参阅论文正文。需注意，部分实验可对应论文中的多张结果图；图表与实验的对应关系请查阅配套代码库。

#### 数据组织方式
每组实验的数据以多级文件夹与文件形式组织：
1. **根目录层级**：数据首先按瓶颈队列采用的数据包调度算法分类，分为先进先出（First In First Out, FIFO）、CoDel、公平队列（Fair Queue, FQ）、FQ-CoDel四类。
2. **调度算法子目录**：按同一实验下的瓶颈带宽、传播时延与缓冲区大小的不同组合创建子文件夹，所有子文件夹均遵循统一命名格式：`{拓扑结构}_{瓶颈带宽}mbps_{单向时延}ms_{缓冲区大小}pkts_{丢包率}loss_{流数量}flows_22tcpbuf_{协议类型}`。例如，`Dumbell_100mbit_80ms_279pkts_0loss_2flows_22tcpbuf_orca` 文件夹包含了哑铃拓扑（Dumbbell Topology）下，瓶颈带宽为100Mbps、单向时延80ms、缓冲区大小为279个最大段大小（Maximum Segment Size, MSS）、无丢包、共2条Orca协议流的所有实验运行数据。

需注意，`{协议类型}`字段的具体含义取决于实验配置：在协议内公平性实验中，所有流均采用该协议类型；而在协议友好性实验中，其中一条流固定为Cubic TCP，另一条流则为该协议类型。
3. **参数组合子目录**：按实验的不同运行次数创建子文件夹。部分实验会采用不同的随机种子，但绝大多数情况下，所有运行的配置完全一致。
4. **运行子目录**：每个运行子文件夹包含实验采集的全部原始数据与部分预处理后的数据，具体包含以下文件与文件夹：
- `tcp_probe.txt`：tcp_probe模块的原始输出日志
- `x[N]_output.txt`：第N个服务器应用的原始输出日志
- `c[N]_output.txt`：第N个客户端应用的原始输出日志
- `emulation_info.json`：实验中所有流的JSON格式描述，每条流包含源IP地址、目的IP地址、使用的协议与启动时间
- `queues/`：包含每个瓶颈队列对应的文本文件，文件内容为针对文件名中指定网卡执行`tc -s qdisc show dev`命令的输出结果
- `sysstat/`：包含sysstat工具生成的原始二进制日志文件（`datafile_*.log`），以及使用sadf处理工具对二进制日志文件解析得到的日志文件
- `csvs/`：包含预处理后的CSV格式数据