Joint Optimization of Time Slot and Power Allocation in Underwater Acoustic Communication Networks

This paper proposes a joint optimization algorithm based on reinforcement learning to address the time slot and power allocation problem in underwater acoustic communication networks (UACNs). By maximizing the total capacity of successful transmissions as the optimization objective, two sub-objectives are formulated corresponding to time-slot scheduling and power allocation. The sub-objective corresponding to time-slot scheduling is addressed by constructing a Markov Decision Process (MDP) model based on Deep Q-Network (DQN) learning. In this model, the agent learns the time slot allocation policy with the goal of increasing the number of successfully transmitted links while reducing the collision. For the sub-objective corresponding to power allocation, another MDP model is developed, solved by the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, in which each underwater transmission node acts as an independent agent. The MADDPG approach enables the system to improve channel capacity under energy limitation, which maximizes the total capacity of successfully transmitted links. In terms of model execution, the DQN adopts a centralized training and time slot allocation, while MADDPG uses a centralized training and distributed execution to select the transmission power by each node. Simulation results show that the proposed joint optimization algorithm demonstrates better performance in the number of successfully transmitted links and channel capacity compared to TDMA, Slotted ALOHA, and other algorithms.