Adaptive Task Offloading and Resource Allocation for Tasks With Time-Varying Statistical Characteristics in MEC Systems

Considering the random access behaviour of mobile devices (MDs), heterogeneous service requirements, and dynamic completion or discarding of tasks, the statistical characteristics of tasks to be scheduled, transmitted, and computed vary over time in practical multi-access edge computing (MEC) systems. Therefore, it is necessary to design a dynamic task offloading and resource allocation (TORA) strategy to adaptively match these time-varying task characteristics. To achieve the goal, we first formulate the dynamic TORA problem as a Markov decision process (MDP) with time-varying extension of state space and action space, which cannot be effectively solved by conventional deep reinforcement learning (DRL) algorithms. To address this challenge, we propose a general state-action space adaptive (SASA) DRL framework by exploiting the advantages of the Transformer architecture and its multi-head attention (MHA) mechanism. This framework facilitates the integration of available actor-critic DRL algorithms to efficiently solve MDPs with time-varying state and action spaces. Based on the proposed SASA DRL framework, we further develop the SASA-based TORA algorithm, referred to as SASA-TORA, which is adaptable to not only dynamic network conditions but also time-varying statistical characteristics of tasks. Simulation results demonstrate the superiority of SASA-TORA over baseline algorithms and highlight the limitations of conventional DRL algorithms in handling MDPs with time-varying state and action spaces.