Resource Scheduling Based on Deep Reinforcement Learning in UAV Assisted Emergency Communication Networks

Unmanned aerial vehicle (UAV) assisted emergency communication is an important technique for future B5G/6G scenario. The UAV is usually considered as a mobile relay to forward information from the macro base station (MBS) to the users in emergency area. In this paper, the MBS power allocation, the UAV service zone selection, and the user scheduling are jointly investigated to enhance the sum spectrum efficiency. We formulate the MBS power allocation and UAV service zone selection problem as an Markov Decision Process (MDP) in the delay ignored system (DIS) and propose a deep reinforcement learning (DRL) algorithm based on Q-learning and Convolutional Neural Networks (CNN). Then the proposed DRL-based scheme is extended in time delay system (TDS) to estimate the current optimal action with the outdated channel information. We also formulate the user scheduling as a 0-1 optimization problem and solve it by dividing into sub-problems. Simulation results demonstrate that the proposed DRL-based resource scheduling scheme can effectively improve the spectrum efficiency compared with the existing schemes.