Data for Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness

IntroductionThis is the dataset for the paper titled ‘Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness’ that has been accepted for publication at IEEE INFOCOM 2024. The paper’s accepted version will be available following publication in May 2024 at https://sussex.figshare.com/articles/conference_contribution/Reinforcement_learningbased_congestion_control_a_systematic_evaluation_of_fairness_efficiency_and_responsiveness/24711033. The dataset is meant to be used in conjunction with the codebase that is also made available at https://doi.org/10.25377/sussex.24978162.However, the dataset itself is of value to researchers as it contains an extensive set of metrics captured during experimentation with Reinforcement Learning-based Congestion control as discussed in the ‘Experimental Evaluation’ section of the paper. Our study is the result of a 160-hour long experimentation during which 1950 Orca, Aurora and TCP Cubic flows were measured. We have collected approximately 500GB of data encompassing diverse metrics related to network interfaces (e.g., utilisation, retransmissions, packet drops), CPU and memory parameters (such as CPU load and memory usage), as well as the data transport layer (e.g., congestion window, round trip time). Reinforcement learning (RL)-based congestion control (CC) promises efficient CC in a fast-changing networking landscape, where evolving communication technologies, applications and traffic workloads pose severe challenges to human-derived, static CC algorithms. RL-based CC is in its early days and substantial research is required to understand existing limitations, identify research challenges and, eventually, yield deployable solutions for real-world networks. In this paper we present the first reproducible and systematic study of RL-based CC with the aim to highlight strengths and uncover fundamental limitations of the state-of-the-art. We identify challenges in evaluating RL-based CC, establish a methodology for studying said approaches and perform large-scale experimentation with RL-based CC approaches that are publicly available. We show that existing approaches can acquire all available bandwidth swiftly and are resistant to non-congestive loss, however, this is commonly at the cost of excessive packet loss in normal operation. We show that, as fairness is not embedded directly into reward functions, existing approaches exhibit unfairness in almost all tested network setups. Finally, we provide evidence that existing RL-based CC approaches under-perform when the available bandwidth and end-to-end latency dynamically change. Our experimentation codebase and datasets are publicly available with the aim to galvanise the community towards transparency and reproducibility, which have been recognised as crucial for researching and evaluating machine-generated policies.The datasetThe dataset contains an extensive set of metrics captured during experimentation. It is composed of eight different experiments. Please refer to the paper for a detailed explanation of the experimental set-up. Note that some experiments are the source of more than one plot in the paper. Refer to the codebase for the relationship between plots and experiments.The data from each experiment is organised into multiple folders and files. Just under the root folder, the data is divided by the type of packet scheduling adopted by the bottleneck queue, i.e. fifo, codel, fq, fq-codel.The next folder contains all variations of bottleneck bandwidth, propagation delay, and buffer size for the same experiment. Each of the folders is named following the same pattern:{topology}_{bottleneck_bandwidth}mbps_{one_way_delay}ms_{buffer_size}pkts_{loss_rate}loss_{number_of_flows}flows_22tcpbuf_{protocol}For example: Dumbell_100mbit_80ms_279pkts_0loss_2flows_22tcpbuf_orca contains all the runs of two flows in a dumbbell topology with 100mbps bottlenck bandwidth, 160ms rtt and a buffer size of 279 MSS. Note that the semantic meaning of {protocol} depends on the specific experimental setup. In the intra-protocol fairness experiment, all flows will be {protocol}, whereas in the friendliness experiment, one flow will always be cubic and one {protocol}.Under the experiment variation folder, multiple folders contain different runs of the same variation. Depending on the experiment, a different seed may be used. However, in most of the cases, all runs are identical.Finally, each run folder contains all the raw data captured during the experiment, plus some processed data.The files and folders contained in each run folder are:tcp_probe.txt: The raw output of tcp_probe modulex[N]_output.txt: The raw output of the Nth server applicationc[N]_output.txt: The raw output of the Nth client application.emulation_info.json: JSON representation of all flows in the experiment. Each flow is represented by the source and destination IP addresses, the protocol used, and the starting time.queues (folder): Contains a text file for each of the bottleneck queue. Each file contains the output of ‘tc -s qdisc show dev’ applied to the dev in the filename.sysstat (folder): Contains the raw binary files (datafile_*.log) generated with sysstat. It also contains some log files generated using sadf processing utility on the binary datafiles. ]csvs (folder): contains processed data in csv format.<br>

引言
本数据集对应已被IEEE INFOCOM 2024会议录用的论文《基于强化学习的拥塞控制：公平性、效率与响应性的系统性评估》（Reinforcement Learning-based Congestion Control: A Systematic Evaluation of Fairness, Efficiency and Responsiveness）。该论文的录用版本将于2024年5月发表后在https://sussex.figshare.com/articles/conference_contribution/Reinforcement_learningbased_congestion_control_a_systematic_evaluation_of_fairness_efficiency_and_responsiveness/24711033 公开获取。
本数据集需与同期公开的代码库配合使用，代码库的公开地址为https://doi.org/10.25377/sussex.24978162。不过，本数据集本身对研究人员具有较高价值，因其包含了论文"实验评估"章节中提及的、基于强化学习（Reinforcement Learning，RL）的拥塞控制（Congestion Control，CC）实验过程中采集的大量指标数据。
本研究基于时长160小时的实验构建而成，共采集了1950条Orca、Aurora及TCP Cubic流的实验数据。本次实验共收集约500GB数据，涵盖网络接口相关的各类指标（如利用率、重传次数、丢包数）、CPU与内存参数（如CPU负载、内存使用率），以及数据传输层指标（如拥塞窗口、往返时延（Round-Trip Time，RTT））。
基于强化学习的拥塞控制（RL-based CC）有望在快速变化的网络环境中实现高效的拥塞控制——当前通信技术、应用场景与流量负载不断演进，给人工设计的静态拥塞控制算法带来了严峻挑战。当前基于强化学习的拥塞控制仍处于早期阶段，需要开展大量研究以明晰其现存局限、明确研究挑战，并最终产出可部署于真实网络的解决方案。
本论文首次针对基于强化学习的拥塞控制开展可复现的系统性研究，旨在阐明现有前沿方案的优势，并揭示其根本性局限。研究团队明确了基于强化学习的拥塞控制的评估挑战，建立了针对这类方案的研究方法，并对公开可用的基于强化学习的拥塞控制方案开展了大规模实验。
实验结果表明，现有方案可快速抢占全部可用带宽，且能抵御非拥塞丢包，但这通常会以正常运行时出现过量丢包为代价。此外，由于公平性未直接嵌入奖励函数，现有方案在几乎所有测试的网络配置中均表现出不公平性。最后，本研究证实，当可用带宽与端到端时延动态变化时，现有基于强化学习的拥塞控制方案性能表现不佳。
本研究将实验代码库与数据集公开，旨在推动社区重视透明性与可复现性——这两点已被证实为研究与评估机器学习生成策略的核心要素。

数据集
本数据集包含实验过程中采集的大量指标数据，共涵盖8组不同实验。实验配置的详细说明请参见论文，需注意部分实验可对应论文中的多张绘图结果，绘图与实验的对应关系请查阅代码库。
每组实验的数据会被组织为多层文件夹与文件。在根目录下，数据将根据瓶颈队列采用的分组调度类型进行划分，分别为：先入先出（First-In First-Out，FIFO）、CoDel、FQ、FQ-CoDel。
下一级文件夹则对应同一实验下的所有参数变体，包括瓶颈带宽、传播时延与缓冲区大小。所有文件夹均遵循统一命名格式：{拓扑}_{瓶颈带宽}mbps_{单向时延}ms_{缓冲区大小}pkts_{丢包率}loss_{流数量}flows_22tcpbuf_{协议}。
例如：`Dumbell_100mbit_80ms_279pkts_0loss_2flows_22tcpbuf_orca` 文件夹包含了哑铃拓扑下的2条流的所有实验运行数据，该拓扑的瓶颈带宽为100Mbps、单向时延为80ms、往返时延（RTT）为160ms、缓冲区大小为279个最大段大小（Maximum Segment Size，MSS）。需注意，`{protocol}` 的语义取决于具体实验配置：在协议内公平性实验中，所有流均使用`{protocol}` 协议；而在友好性实验中，一条流始终为Cubic协议，另一条流则为`{protocol}` 协议。
在参数变体文件夹下，会存在多个子文件夹，分别对应同一参数配置下的不同实验运行。部分实验会使用不同的随机种子，但在绝大多数场景下，所有运行的实验配置均完全一致。
最终，每个实验运行文件夹将包含实验过程中采集的全部原始数据，以及部分预处理后的数据。
每个实验运行文件夹包含的文件与子文件夹如下：
tcp_probe.txt：tcp_probe模块的原始输出；
x[N]_output.txt：第N个服务器应用程序的原始输出；
c[N]_output.txt：第N个客户端应用程序的原始输出；
emulation_info.json：实验中所有流的"JSON"格式信息，每条流包含源IP地址、目的IP地址、使用的协议以及启动时间；
queues（文件夹）：包含每个瓶颈队列对应的文本文件，每个文件存储了针对文件名中指定网卡执行`tc -s qdisc show dev` 命令的输出结果；
sysstat（文件夹）：包含由sysstat生成的原始二进制文件（datafile_*.log），同时也包含了使用sadf工具对二进制数据文件进行处理后生成的部分日志文件；
csvs（文件夹）：包含预处理后的CSV格式数据。<br>