From Movement to Manipulation: Multimodal Control of a Mobile Assistive Robot

We present a hybrid brain-computer interface (hBCI) system designed to control a 3D-printed robot car equipped with a 6-joint robotic arm, named CARM. The proposed hBCI system integrates four distinct control inputs: steady-state visual evoked potentials (SSVEP) for longitudinal vehicle control (forward/backward), motor imagery (MI) for lateral turning (left/right turns), computer vision (CV) for robotic arm manipulation, and electromyography (EMG) for mode switching. The system’s performance was evaluated in an internal BCI robot control competition. The competition comprised two online tasks: "Speed and Precision" and "Obstacle Environment." The proposed multimodal strategy successfully completed these two tasks, demonstrating the complementary strengths of the integrated modalities and their positive impact on task performance.