The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems

Model Embedding CTR Size per 100M (Compression) Dimension Lift Embeddings SBERT [18] 512 (baseline) 204.8 GB Nomic [14] 768 +4.86% 307.2 GB Nomic (Matryoshka) 64 +1.89% 25.6 GB Nomic (CompresSAE) 4096* +3.44% 25.6 GB *Sparse embeddings with 32 nonzero entries.Figure 1: Comparison of embedding models used for candidate retrieval.We report online recommendation performance on a downstream task, relative to SBERT [18], with anytime-valid 99% confidence intervals.