Introducing Self-Supervised Learning Models for Spoken Query-Spoken Term Detection

This study was undertaken to improve the performance of Spoken Query Spoken Term Detection (SQ-STD) by introducing Self-Supervised Learning (SSL) models. To construct posteriorgrams, the SSL models are augmented with a Connectionist Temporal Classification (CTC) layer and extracted posterior probability vectors, which represent phonemes or syllables from the layer immediately preceding the CTC output layer. Posteriorgram matching is performed using posteriorgrams generated from SSL models of three types known for their high speech recognition accuracy: wav2vec 2.0, HuBERT, and WavLM. Experiment results obtained on the NTCIR evaluation sets demonstrate that our approach achieves retrieval accuracy exceeding 92 %, which is the state of the art for the evolution sets.