Disfluency Disentanglement Enhancement in Spoken-Text-Style Transfer for Spontaneous Speech Synthesis

Spoken-text-style transfer (STST) aims to convert a given text to a desired style while preserving its semantic content. In this paper, we focus on modeling the disfluency in spoken text, which can be useful as a preprocessing step in spontaneous speech synthesis. Previous methods suffer from unknown words and confusion between normal and disfluent words. Our proposed solutions to the above-mentioned problems include a masked language model (MLM) approach to temporally replace unknown words, and a disfluency symbol representation (DSR) to increase the discrepancy against normal words. Experimental evaluation results show that MLM improves the robustness against unknown words, and the use of DSR achieves a higher transfer accuracy. Finally, we show the potential of speaker-modeling to achieve speaker-wise disfluency control in STST.