Reinforcement Learning-Based Distributed Channel Access for Delay Optimization

As new applications evolve rapidly, wireless networks increasingly require low-delay communication to significantly enhance the quality of user experience. In response, the evolution of the medium access control (MAC) layer has gained more attention, particularly through the application of reinforcement learning to optimize access strategies. In order to meet the low-delay requirements, we propose a reinforcement learning-based MAC protocol, named soft actor-critic multiple access (SAC-MA). To mitigate frequent collisions caused by the exploratory behavior, we propose a multiple waiting actions mechanism that allows stations to wait for multiple time slots. This mechanism enables the agent to develop a more flexible and intelligent access strategy, thereby effectively reducing delay. Additionally, we introduce an innovative formulation in which the head-of-line packet is treated as the agent, enabling more timely feedback and observations. We conduct extensive simulations to demonstrate that SAC-MA: 1) reduces delay by approximately 27.9% and 56.5% compared to the conventional MAC protocol with standard parameters under the collision and capture models, respectively; 2) adapts to environmental changes in dynamic scenarios; 3) coexists harmoniously with legacy stations and reduces the network delay in heterogeneous scenarios. Finally, we perform ablation studies to evaluate the effectiveness of the proposed mechanisms.