SMART - Self-adaptive Machine Learning Approach for Real-time Tuning of IEEE 802.11 PHY and MAC layers
收藏资源简介：
<strong>Introduction</strong> Worldwide the demand for wireless access networks providing very high throughputs has been increasing exponentially, namely due to bandwidth-hungry applications such as high definition video streaming and augmented reality. In order to fulfil these requirements, the Wi-Fi standard was enriched with new amendments, such as IEEE 802.11n, IEEE 802.11ac, and recently IEEE 802.11ax (Wi-Fi 6). New parameters have been proposed for both physical (PHY) and media access control (MAC) layers, including channel bonding, short guard interval (SGI), and advanced modulation and coding schemes (MCS). However, the high variability of the signal strength in the wireless radio channel, allied to the channel asymmetry, makes the selection of optimal configurations for these parameters a challenge. Typically, these parameters are configured with a default value. For runtime optimization, some algorithms have already been proposed. Still, they were designed considering legacy IEEE 802.11 releases and static scenarios. Besides, these parameters have their trade-offs that need to be properly managed. To help dealing with this, machine learning has been recently introduced in wireless networks, providing the intelligence that networks need in order to be smart and self-adaptive. SWOP (Smart Wireless Optimization) is a cross-layer optimization approach for Wi-Fi networks extending the current Rate Adaptation (RA) approach, for instance, followed by the well-known Minstrel algorithm widely used in practice. Our approach takes advantage of Deep Reinforcement Learning (DRL) in order to learn the optimal Wi-Fi link configuration. By considering the wireless channel as the environment, the transmitter node (the agent) chooses the best link parameters (the action) in order to maximize the throughput (the reward) based on the channel metrics captured from the environment (the state). In this work we propose a simple DRL-based Wi-Fi Rate Adaptation (RA) algorithm, named Data-driven Algorithm for Rate Adaptation (DARA), which is one of the modules of Smart Wireless Optimization (SWOP) SMART aimed to run a set of wireless experiments on top of w-iLab.t testbeds provided by the Fed4FIRE+ project to directly validate our DRL model and learn a policy from the wireless experiments executed in a controlled environment. However, after facing difficulties with the scenarios we could achieve on the real testbed, we decided to train and test DARA using a trace-based simulation approach. In simulation, we could train our model in scenarios that are more complex and diverse whilst easy to configure, when compared to real testbeds. The w-iLab.t testbeds were still used to capture data traces (e.g. Signal-to-Noise Ratio, position of nodes, transmission power and link distance) that were then injected in ns-3 for validating DARA. With this work, we concluded that DARA performance is impacted when operating in scenarios with asymmetric links, which is common in the highly dynamic and unpredictable wireless environments. Furthermore, the asymmetry offset varies between scenarios and it may also change for the same scenario, as time progresses. This randomness is not addressed when solely considering the SNR as the link metric, posing a challenge in the learning phase of DARA. Despite these limitations, the results obtained show that DARA still achieves up to 14.9% higher throughput higher than Minstrel [1] and slightly lower than Ideal [2] for most of the scenarios. The results obtained will serve as a basis to support our ongoing and future research. <strong>Folder Organization</strong> The following dataset presents the results of the SMART project, organized in different folders for each Rate Adaptation Algorithm, as well as the traces that were used to obtain such results: <strong>DARA: </strong>Results obtained using our solution <strong>(Naming Convention #1, Folder Content #1)</strong> <strong>MIN: </strong>Results obtained using Minstrel-HT <strong>(Naming Convention #1, Folder Content #2)</strong> <strong>ID: </strong>Results obtained using Ideal <strong>(Naming Convention #1, Folder Content #2)</strong> <strong>TRACES: </strong>Trace files used to obtain the results present in this dataset <strong>(Naming Convention #2, Folder Content #3)</strong> <strong>Naming Convention #1 – RAA TID TP TO:</strong> Rate Adaptation Algorithm<strong> (RAA) </strong> <strong>drl </strong>– Data Driven Algorithm for Rate Adaptation <strong>min </strong>– MinstrelHTWifiManager <strong>id </strong>– IdealWifiManager Trace ID<strong> (TID)</strong> <strong>3 </strong>up to<strong> 8</strong> Transport Protocol<strong> (TP)</strong> <strong>udp </strong>– User Datagram Protocol Traffic Orientation<strong> (TO)</strong> <strong>normal </strong>– A<strong>-></strong>B <strong>reversed </strong>– B<strong>-></strong>A <strong>Naming Convention #2 – TID_TXP:</strong> Trace ID<strong> (TID) </strong> <strong>3 </strong>up to<strong> 8</strong> Transmitting Power in dBm <strong>(TXP) </strong> <strong>3, 5, 7, 9, 12 dBm </strong> <strong>Folder Content #1: </strong> <em>checkpoint_ RAA TID TP TO</em><strong> (Folder)</strong> <strong>Policy Checkpoint</strong> with which the results were obtained <strong>Flowmonitor </strong>output for the configured scenario Column 1 – <strong>Step Counter</strong> Column 2 – <strong>Reward Value</strong> Column 3 – <strong>Observation Value</strong> Column 4 – <strong>Action Value</strong> Column 1 – <strong>Simulation Time </strong>(seconds) Column 2 – <strong>Throughput </strong>(Mbit/100ms) <strong>Folder Content #2: </strong> <strong>Flowmonitor </strong>output for the configured scenario Column 1 – <strong>Simulation Time </strong>(seconds) Column 2 – <strong>Throughput </strong>(Mbit/100ms) <strong>Folder Content #3 - </strong>Source: https://zenodo.org/record/3713271#.YjjBVDXLdhE<strong>: </strong> · <em>date_time</em><strong>.cfg </strong>configuration details of the experiment · <em>date_time_NodeID</em><strong><em><sup>[1]</sup></em></strong><em>_SenderID</em><strong><em><sup>[2]</sup></em></strong><em>_ReceiverID</em><strong><em><sup>[3]</sup></em></strong><em>_FlowType</em><strong><em><sup>[4]</sup></em></strong><em>_Params</em><strong><em><sup>[5]</sup></em></strong><strong>.snr </strong>– logs of the Signal/Noise ratio (1 file per node/flow) · <em>date_time_NodeID_SenderID_ReceiverID_FlowType_Params</em><strong>.stats</strong> – logs of the packets received (1 file per node/flow) <sub>[1]</sub><sub> ID of the node Logging node</sub> <sub>[2]</sub><sub> ID of the Sender node</sub> <sub>[3]</sub><sub> ID of the Receiver node</sub> <sub>[4]</sub><sub> Flow type: Unidirectional, Bidirectional or Unidirectional with Multiple Access</sub> <sub>[5]</sub><sub> Configurable parameters: Sender/Receiver Transmission Power and Data Rate (when applicable)</sub> <strong>References</strong> 1. F. FietKau, “Minstrel_HT: New rate control module for 802.11n [LWN.net]”. Mrt-2010. 2. “ns-3: ns3::IdealWifiManager Class Reference,” Jan 2021, [Online; accessed 23. Jun. 2021]. Available: https://www.nsnam.org/docs/release/3.33/doxygen/classns3_1_1_ideal_wifi manager.html
<strong>引言</strong> 全球范围内，支持极高吞吐量的无线接入网需求呈指数级增长，这主要源于高清视频流、增强现实等带宽密集型应用。为满足此类需求，Wi-Fi标准新增了多项修订案，例如IEEE 802.11n、IEEE 802.11ac，以及最新的IEEE 802.11ax（Wi-Fi 6）。研究人员为物理层（Physical Layer, PHY）与媒体访问控制层（Media Access Control, MAC）提出了多项新参数，包括信道绑定、短保护间隔（Short Guard Interval, SGI）以及高级调制编码方案（Modulation and Coding Schemes, MCS）。然而，无线信道中信号强度的高度可变性与信道非对称性，使得为这些参数选择最优配置成为一项挑战。通常情况下，这类参数会采用默认值进行配置。针对运行时优化，已有多项算法被提出，但这些算法均是针对传统IEEE 802.11版本与静态场景设计的。此外，这些参数之间存在权衡关系，需要进行妥善管理。 为解决上述问题，机器学习近年来被引入无线网络，为网络提供了实现智能化与自适应所需的智能。SWOP（Smart Wireless Optimization，智能无线优化）是一种面向Wi-Fi网络的跨层优化方法，它扩展了当前的速率适配（Rate Adaptation, RA）方法——例如广泛应用于实际场景的经典Minstrel算法。本方法利用深度强化学习（Deep Reinforcement Learning, DRL）来学习最优的Wi-Fi链路配置：将无线信道视为环境，发送节点（智能体）根据从环境中获取的信道指标（状态），选择最优的链路参数（动作）以最大化吞吐量（奖励）。 在本研究中，我们提出了一种基于深度强化学习的简易Wi-Fi速率适配算法，命名为数据驱动型速率适配算法（Data-driven Algorithm for Rate Adaptation, DARA），它是Smart Wireless Optimization（SWOP）SMART项目的模块之一，旨在通过Fed4FIRE+项目提供的w-iLab.t测试床开展一系列无线实验，以直接验证我们的深度强化学习模型，并从受控环境下执行的无线实验中学习策略。然而，在实际测试床中遇到可用场景受限的问题后，我们决定采用基于轨迹的仿真方法来训练与测试DARA。相较于真实测试床，仿真环境下我们可以在更复杂多样且易于配置的场景中训练模型。我们仍使用w-iLab.t测试床采集数据轨迹（例如信噪比（Signal-to-Noise Ratio, SNR）、节点位置、发射功率与链路距离），随后将这些轨迹注入ns-3仿真器以验证DARA。 通过本研究，我们得出结论：当在存在非对称链路的场景中运行时，DARA的性能会受到影响——而此类场景在高度动态且不可预测的无线环境中十分常见。此外，非对称偏移量在不同场景中存在差异，且同一场景下也可能随时间推移发生变化。若仅将信噪比作为链路指标，此类随机性无法被有效处理，这为DARA的学习阶段带来了挑战。尽管存在上述局限性，实验结果表明，在大多数场景中，DARA的吞吐量较Minstrel算法最高可提升14.9%，且略低于理想（Ideal）算法。本研究所得结果将作为基础，支撑我们正在进行与未来的研究工作。 <strong>文件夹组织结构</strong> 本数据集展示了SMART项目的实验结果，我们为每种速率适配算法划分了独立文件夹，同时收录了用于生成上述结果的轨迹数据： <strong>DARA：</strong> 采用本研究提出的解决方案所得的实验结果 <strong>（命名规则#1，文件夹内容#1）</strong> <strong>MIN：</strong> 采用Minstrel-HT算法所得的实验结果 <strong>（命名规则#1，文件夹内容#2）</strong> <strong>ID：</strong> 采用理想（Ideal）算法所得的实验结果 <strong>（命名规则#1，文件夹内容#2）</strong> <strong>TRACES：</strong> 用于生成本数据集实验结果的轨迹文件 <strong>（命名规则#2，文件夹内容#3）</strong> <strong>命名规则#1 – RAA TID TP TO:</strong> 速率适配算法（Rate Adaptation Algorithm, RAA） <strong>drl</strong> – 数据驱动型速率适配算法（Data Driven Algorithm for Rate Adaptation） <strong>min</strong> – MinstrelHTWifiManager <strong>id</strong> – IdealWifiManager 轨迹ID（Trace ID, TID）<strong>（TID）</strong>：取值范围为3至8 传输协议（Transport Protocol, TP）<strong>（TP）</strong>： <strong>udp</strong> – 用户数据报协议（User Datagram Protocol） 流量方向（Traffic Orientation, TO）<strong>（TO）</strong>： <strong>normal</strong> – A→B <strong>reversed</strong> – B→A <strong>命名规则#2 – TID_TXP:</strong> 轨迹ID（Trace ID, TID）：取值范围为3至8 发射功率（Transmitting Power, TXP），单位为dBm：<strong>（TXP）</strong>：3、5、7、9、12 dBm <strong>文件夹内容#1：</strong> <em>checkpoint_ RAA TID TP TO</em><strong>（文件夹）</strong>：用于生成实验结果的<strong>策略检查点</strong> Flowmonitor输出：对应配置场景的运行结果，各列含义如下： 第1列 – <strong>步数计数器</strong> 第2列 – <strong>奖励值</strong> 第3列 – <strong>观测值</strong> 第4列 – <strong>动作值</strong> 另有一组列： 第1列 – <strong>仿真时长</strong>（单位：秒） 第2列 – <strong>吞吐量</strong>（单位：Mbit/100ms） <strong>文件夹内容#2：</strong> Flowmonitor输出：对应配置场景的运行结果，各列含义如下： 第1列 – <strong>仿真时长</strong>（单位：秒） 第2列 – <strong>吞吐量</strong>（单位：Mbit/100ms） <strong>文件夹内容#3 - </strong>数据来源：https://zenodo.org/record/3713271#.YjjBVDXLdhE<strong>: </strong> · <em>date_time</em><strong>.cfg</strong>：实验配置详情文件 · <em>date_time_NodeID</em><strong><em><sup>[1]</sup></em></strong><em>_SenderID</em><strong><em><sup>[2]</sup></em></strong><em>_ReceiverID</em><strong><em><sup>[3]</sup></em></strong><em>_FlowType</em><strong><em><sup>[4]</sup></em></strong><em>_Params</em><strong><em><sup>[5]</sup></em></strong><strong>.snr</strong>：信噪比日志文件（每个节点/流量对应一个文件） · <em>date_time_NodeID_SenderID_ReceiverID_FlowType_Params</em><strong>.stats</strong>：数据包接收日志文件（每个节点/流量对应一个文件） <sub>[1]</sub> 日志记录节点的ID <sub>[2]</sub> 发送节点的ID <sub>[3]</sub> 接收节点的ID <sub>[4]</sub> 流量类型：单向、双向或多址单向 <sub>[5]</sub> 可配置参数：发送/接收节点发射功率与数据速率（如适用） <strong>参考文献</strong> 1. F. FietKau, "Minstrel_HT: 面向802.11n的新型速率控制模块[LWN.net]". Mrt-2010. 2. "ns-3: ns3::IdealWifiManager Class Reference," Jan 2021, [Online; accessed 23. Jun. 2021]. Available: https://www.nsnam.org/docs/release/3.33/doxygen/classns3_1_1_ideal_wifi manager.html
