AI-based Radio Resource Management and Trajectory Design for PD-NOMA Communication in IRS-UAV Assisted Networks

In this paper, we consider that the unmanned aerial vehicles (UAVs) with attached intelligent reflecting surfaces (IRSs) play the role of flying reflectors that reflect the signal of users to the destination, and utilize the power-domain non-orthogonal multiple access (PD-NOMA) scheme in the uplink. We investigate the benefits of the UAV-IRS on the internet of things (IoT) networks that improve the freshness of collected data of the IoT devices via optimizing power, sub-carrier, and trajectory variables, as well as, the phase shift matrix elements. We consider minimizing the average age-of-information (AAoI) of users subject to the maximum transmit power limitations, PD-NOMA-related restriction, and the constraints related to UAV’s movement. The optimization problem consists of discrete and continuous variables. Hence, we divide the resource allocation problem into two sub-problems and use two different reinforcement learning (RL) based algorithms to solve them, namely the double deep Qnetwork (DDQN) and a proximal policy optimization (PPO). Our numerical results illustrate the performance gains that can be achieved for IRS enabled UAV communication systems. Moreover, we compare our deep RL (DRL) based algorithm with matching algorithm and random trajectory, showing the combination of DDQN and PPO algorithm proposed in this paper performs 10% and 15% better than matching algorithm and random-trajectory algorithm, respectively.

本文探讨搭载智能反射表面（IRS）的无人驾驶航空器（UAV）在信号反射至目的地的过程中所扮演的飞行反射器角色，并采用上链路中的功率域非正交多址接入（PD-NOMA）方案。本研究旨在分析无人机-IRS对物联网（IoT）网络的优势，通过优化功率、子载波、轨迹变量以及相位移矩阵元素，提升物联网设备收集数据的更新速度。我们考虑在最大传输功率限制、PD-NOMA相关约束以及无人机移动相关约束条件下，最小化用户平均信息年龄（AAoI）。该优化问题包含离散和连续变量。因此，我们将资源分配问题划分为两个子问题，并使用两种基于强化学习（RL）的算法来解决，即双深度Q网络（DDQN）和近端策略优化（PPO）。我们的数值结果展示了通过智能反射表面技术实现的无人机通信系统所能达到的性能提升。此外，我们将基于深度强化学习（DRL）的算法与匹配算法以及随机轨迹算法进行了比较，表明本文提出的DDQN与PPO算法的组合，相较于匹配算法和随机轨迹算法，分别提高了10%和15%的性能。