IndoBERT-Relevancy: A Context-Conditioned Relevancy Classifier for Indonesian Text

We introduce IndoBERT-Relevancy, a context-conditioned relevancy classifier for Bahasa Indonesia built on IndoBERT Large (335M parameters). Given a topical context and a candidate text, the model predicts whether the text is relevant to the context. Trained on a novel dataset of 31,360 labeled pairs spanning 188 topics, the model achieves an F1 score of 0.948 and accuracy of 96.5%. Through an iterative, failure-driven data construction process combining formal text, informal social media text, and targeted synthetic examples, we demonstrate that multi-source training data is essential for robust cross-register relevancy classification. The model is publicly available at https://huggingface.co/apriandito/indobert-relevancy-classifier