Talking Like Us: A Multimodal Design for Human-Like Conversation Flow in Robots

Social robots often struggle with natural conversational flow due to rule-based systems that cause pauses and interruptions. We present MESA (Modular Empathetic Social Assistant), a campus robot that synchronizes verbal responses with non-verbal cues. It uses UltraVAD to distinguish thoughtful pauses from turn completions and implements turn-yielding and floor-holding mechanisms. A distributed Retrieval-Augmented Generation (RAG) architecture ensures accurate responses for students and faculty. Validated on a proxy platform with ~600 ms latency, MESA demonstrates that combining context-aware audio and gaze enables more fluid conversation, even as hardware assembly nears completion.