Tokenization Induced Morphological Degradation in Arabic Language Models and a Structured Embedding Alternative: Root-Pattern Tensor Subspaces

In our paper, we explore the limitations of byte pair encoding (BPE) in Arabic language models, revealing significant token inflation and a failure to preserve morphological relationships. We introduce root pattern tensor subspaces (RPTS) as a more efficient and linguistically informed alternative, achieving a remarkable reduction in embedding parameters while maintaining structural integrity.