Distributed Joint Resource Allocation for Underwater Acoustic Communication Networks: An Extended Bandit Learning Game Approach

This paper copes with a joint discrete-channel and continuous-power allocation problem for underwater acoustic communication networks. Hybrid discrete-continuous solution spaces, inevitable interference among multiple users, and unknown time-varying underwater-acoustic communication environments make the proposed problem full of challenges. Firstly, combining multi-armed bandit theory and game theory, an adversarial multiplayer bandit game model is developed to enable the proposed problem to be solved distributedly without any prior perfect channel information. Secondly, a dynamic finite discrete strategy pool learning structure is proposed, which makes the finite discrete strategy pool evolve efficiently over time. Namely, at each learning time, the player can quickly decide based on a finite discrete strategy pool, thereby improving the learning efficiency. With the development of learning time, the dynamic strategy pool can efficiently evolve to extend the whole discrete-continuous space, thereby avoiding missing the real best solution.  Consequently, the bottleneck, which bandit learning algorithms can not be applied to continuous solution spaces, is broken by the proposed learning structure to achieve the best solution of the hybrid discrete-continuous space in an efficient low-cost fashion. Thirdly, a selection probability setting rule is proposed to make the real utility level of strategies for the dynamic strategy pool able to be rapidly identified, thereby significantly enhancing the quick search ability of dynamic best strategies. Finally, the superiority of the proposed algorithm is demonstrated by extensive simulation results.