From Speech to Delivery: Human-Centered Multimodal Interaction for Indoor Service Robotics

This paper presents the design and implementation of a voice-controlled autonomous robotic delivery system that integrates physical human-robot interaction (HRI), perception, manipulation, and navigation. At the core of the system is an orchestration module implemented as a finite state machine, responsible for managing task execution and coordinating subsystem communication through ROS2 topics and services. The robot interprets natural language commands using a GPT-based Physical AI module, detects objects via perception input, manipulates them with a robotic arm, and navigates to user-defined destinations. We describe the full task pipeline—from voice input to final delivery—highlighting the orchestration logic, system robustness strategies, and real-time feedback mechanisms. Our results demonstrate that modular ROS2-based orchestration enables reliable multi-step execution in collaborative HRI scenarios.