Codes of paper: AI-based Resource Allocation in End-to-End Network Slicing under Demand and CSI Uncertainties

Abstract—Network slicing (NwS) is one of the main technologiesin the h-generation of mobile communication andbeyond (5G+). One of the important challenges in the NwSis information uncertainty which mainly involves demandand channel state information (CSI). Demand uncertainty isdivided into three types: number of users requests, amountof bandwidth, and requested virtual network functions workloads.Moreover, the CSI uncertainty is modeled by threemethods: worst-case, probabilistic, and hybrid. In this paper,our goal is to maximize the utility of the infrastructureprovider by exploiting deep reinforcement learning algorithmsin end-to-end NwS resource allocation under demandand CSI uncertainties. e proposed formulation is a nonconvexmixed-integer non-linear programming problem. Toperform robust resource allocation in problems that involveuncertainty, we need a history of previous information. Tothis end, we use a recurrent deterministic policy gradient(RDPG) algorithm, a recurrent and memory-based approachin deep reinforcement learning. en, we compare the RDPGmethod in dierent scenarios with so actor-critic (SAC),deep deterministic policy gradient (DDPG), distributed, andgreedy algorithms. e simulation results show that the SACmethod is better than the DDPG, distributed, and greedymethods, respectively. Moreover, the RDPG method out performsthe SAC approach on average by 70%.Index Terms— End-to-end network slicing, Resource allocation,Soware-dened networking (SDN), Network functionvirtualization (NFV), Demand uncertainty, Channel state information(CSI) uncertainty, Recurrent deterministic policygradient (RDPG).

摘要—网络切片（NwS）是未来通信世代（包括5G+）的核心技术之一。在网络切片领域，信息不确定性是一个重要的挑战，其主要涉及需求与信道状态信息（CSI）。需求不确定性可细分为用户请求数量、带宽需求和所请求的虚拟网络功能工作负载三种类型。此外，CSI不确定性通过三种方法进行建模：最坏情况、概率性和混合方法。本研究旨在通过在端到端网络切片资源分配中利用深度强化学习算法，在需求和CSI不确定性条件下最大化基础设施提供商的效用。所提出的公式是一个非凸混合整数非线性规划问题。为了在涉及不确定性的问题中执行鲁棒资源分配，我们需要历史信息记录。为此，我们采用了一个循环确定性策略梯度（RDPG）算法，这是一种基于循环和记忆的深度强化学习方法。随后，我们将RDPG方法在不同场景下与SAC（软 Actor-Critic）、深度确定性策略梯度（DDPG）、分布式和贪婪算法进行了比较。仿真结果表明，SAC方法在性能上优于DDPG、分布式和贪婪方法。此外，RDPG方法在平均性能上比SAC方法高出70%。关键词—端到端网络切片、资源分配、软件定义网络（SDN）、网络功能虚拟化（NFV）、需求不确定性、信道状态信息（CSI）不确定性、循环确定性策略梯度（RDPG）。