Fast- and memory-efficient convolution on ARM for computer vision neural networks

Modern convolutional neural networks are central to computer vision applications such as medical imaging, autonomous driving, and surveillance. However, their computational demands can limit deployment in practice, especially in resource-constrained environments such as mobile and embedded systems. The widely used approach for such devices is to transform convolution into matrix multiplication, thereby benefiting from hardware capabilities such as parallelism and vector extensions. The basic algorithm, im2col, suffers from high memory usage, which limits its effectiveness on devices with constrained resources. The recently proposed algorithm, im2win, consumes significantly less memory but has not been experimentally evaluated on ARM processors, which are commonly used in the devices under consideration. In this paper, we demonstrate that im2win is approximately twice as slow as the optimized im2col algorithm for typical lightweight convolutions on ARM. So, we propose an optimized im2win implementation based on matrix multiplication. Our implementation remains memory-efficient and has a running time comparable to im2col and other state-of-the-art algorithms, though it is slightly slower than the highly computationally efficient p-im2col algorithm for specific convolution parameters.