MADDPG and P2P-VFRL for minimizing AoI in NTN network under CSI uncertainty

Non-terrestrial networks (NTNs) are emerging asa promising solution to handle the increment of computationrequirements. Due to the importance of time in computationintensive applications, the criterion of age of information (AoI)has been introduced, which gives a better view of the freshness ofinformation. To this aim, in this paper we develop a hierarchicalaerial computing framework composed of high altitude platform(HAP) and unmanned aerial vehicles (UAVs) to compute the fulloffloaded tasks of terrestrial mobile users who are connected byuplink non-orthogonal multiple access (UL-NOMA). In particular, the problem is formulated to minimize the AoI of all users,whose tasks can take more than one time slot to be received,by UAV’s trajectory and resource allocation on both UAVsand HAP, which is restricted by the channel state information(CSI) uncertainty and multiple resource constraints of UAVs andHAP. In order to solve this non-convex optimization problem,two methods of multi-agent deep deterministic policy gradient(MADDPG) and federated reinforcement learning (FRL) areconsidered to design the UAVs’ trajectory, channel, power andCPU allocation. The complexity of algorithms and the impactsof different parameters are analyzed which verifies the efficiencyof MADDPG approach compared to other method