Survey on the Impact of Listening to Audio for Adaptive Japanese Subtitles and Captions Ruby

Subtitles and closed captions, which are prepared for hearing-impaired users, are now widely used by users without hearing concerns. In this paper, we focus on the adaptation of subtitles and captions for non-hearing-impaired users, particularly the adaptation of the kanji ruby. From our experiments on non-hearing-impaired adults, Welch’s t-test was used to clarify whether listening to audio with the same content affects the necessity of kanji ruby. In addition, we proposed and evaluated an adaptive model to predict whether ruby should be added to kanji captions based on the experimental results. The experimental results suggest that not only the difficulty of the kanji and the user’s kanji ability, but also the content of the audio is important for the optimization of kanji ruby.