GPU-accelerated In-Network Computing for Split-AI in 6G: A Trade-Off Between Inference Time and Energy Consumption

Future 6G networks will be called upon to support increasingly sophisticated applications based on the massive use of Artificial Intelligence (AI). This poses serious challenges to mobile devices that do not have adequate computational and energy capacity to support the effective execution of AI tasks. According to the emerging Split-AI paradigm, 6G networks can help user devices to perform complex AI tasks by distributing and executing a Neural Network (NN) collaboratively among different processing nodes, including network nodes in the 6G User Plane (UP). However, the extent to which these offloading mechanisms can improve the end-to-end latency of AI tasks is limited by the computational resources available in an operator’s network. The presence of Graphical Processing Units (GPUs) in some elements of the UP can certainly help, but it is necessary to quantify the cost of the task acceleration in terms of increased energy consumption entailed by GPU usage. In this work, we study on the example of Split-AI, the impact of GPU acceleration on inference time and energy consumption. Thereby, we demonstrate that GPU inclusion can reduce the latency induced by the DNN computation by up to 90%, with moderate energy consumption increase of 22% at most.