Multimodal Advantage in Android Emotional Expressions

Multimodal emotional expressions play an essential role in real-life communication. Previous psychological studies have reported that emotional expressions can be more effective using trimodal channels, including verbal, vocal, and facial expressions, than those using only bimodal or unimodal channels. Although robots are expected to interact with humans, it remains unclear whether the multimodal advantage could be observed in the emotional expressions of a robot. We investigated this issue with the android Nikola. Nikola showed unimodal, bimodal, and trimodal expressions of negative and positive emotions using verbal, vocal, and facial channels in a face-to-face situation. Participants rated the expressed emotions regarding emotional valence and humanlikeness. Nikola’s trimodal expressions were rated as more emotional than the unimodal and bimodal expressions. The trimodal expressions were also more humanlike than the unimodal and bimodal expressions. These results suggest that robots can display emotional expressions in a more intense and humanlike manner using multimodal channels.