Pose-Based Video Sign Language Recognition on Edge Devices Using Deep Neural Networks

This paper presents an efficient, pose-based deep neural network model for real-time American Sign Language (ASL) recognition on edge devices. Using MediaPipe to extract skeletal key points, we reduce input dimensionality and enable low-complexity model inference. A DenseNet-169 model is trained on the word level American sign language dataset and systematically compressed using structured pruning, ONNX (Open Neural Network Exchange) export, and INT8 quantization. Our ablation study evaluates the trade-offs between accuracy degradation and efficiency gains across Raspberry Pi 4, Raspberry Pi 5, and Intel Core i9 platforms. Compared to the baseline, our most optimized model reduces memory usage by over 84% and inference latency by up to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$7.6 \times$</tex>, with only a moderate <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\sim 5 \%$</tex> drop in accuracy. These results demonstrate that pose-based video sign language recognition can be deployed on resource-constrained hardware without sacrificing practical performance, enabling real-time, offline communication assistance.