Energy Minimization of the Cell-Free MEC Networks With Two-Timescale Resource Allocation

In this paper, we investigate the energy minimization problem in cell-free mobile edge computing (MEC) networks under dynamic channel conditions and random arrival tasks. We aim to minimize the total energy consumption of user equipment (UEs) and MEC servers (MECSs) by jointly optimizing MECS active/sleep mode selection, UE-MECS association decision (MSAD), and task offloading, communication, and computation resource allocation (TORA) across two different timescales. Considering that the involved optimization variables affect the system performance in different timescales, we decouple the formulated stochastic optimization problem into two subproblems: MSAD operating in the large timescale and TORA in the small timescale. Leveraging the Lyapunov method, the stochastic TORA problem is decoupled into a series of deterministic problems, of which the closed-form solutions are presented. Furthermore, the MSAD problem is reformulated as a constrained Markov decision process (MDP). Then, we propose a double dueling deep Q-network (D3QN) to learn the optimal MSAD based on the TORA results. Numerical results demonstrate that the proposed online TORA-assisted MSAD learning algorithm has effective convergence and achieves substantial energy reductions for the MEC networks compared with the benchmark schemes.