Distributed Handoff Problem in Heterogeneous Networks With End-to-End Network Slicing: Decentralized Markov Decision Process-Based Modeling and Solution

Heterogeneous networks (HetNets) with end-to-end (E2E) network slicing are regarded as effective approaches to meet diverse service requirements from vertical industries. Due to the dense deployment of base stations (BSs) and the complicated associations between BSs and E2E network slices (NSs) in the scenario, the handoff problem faces challenges of the huge system state space and handoff action space and the considerable communication overhead. In this paper, we take these issues into account and consider a distributed E2E NS handoff decision framework in the HetNet. A decentralized Markov decision process (DEC-MDP)-based model is formulated for the distributed E2E NS handoff problem, and the jointly observable and random characteristics of the DEC-MDP are analyzed. To obtain a theoretical performance reference, the original distributed E2E NS handoff problem is simplified, and a Nash equilibrium-based performance bound is given. More practically, the multi-agent double deep Q-network-based distributed handoff (MA-DDQN-DH) algorithm with the centralized training and decentralized executing framework is proposed. Simulation results show that the Nash equilibrium-based performance bound is reasonable, and the proposed MA-DDQN-DH algorithm performs well in the comparison.