Sample-Efficient Robot Learning for Supervised Effect Prediction Tasks

In self-supervised robot learning, data is acquired through active interaction with the environment, which is costly. Therefore, sample-efficient exploration is vital. To this end, intrinsic motivation (IM) methods such as learning progress (LP) have been adopted in robot learning, with variable success across tasks; whereas in machine learning, active learning (AL) is considered the go-to solution, especially for classification tasks. However, there is no systematic method for fusing both approaches for continuous regression tasks encountered in robot learning. To this end, we propose MUSEL (Model Uncertainty for Sample-Efficient Learning), a novel AL framework tailored for regression tasks in robotics, such as action-effect prediction. MUSEL introduces a novel model uncertainty metric that combines total predictive uncertainty, learning progress, and input diversity to guide experience gathering. We choose Stochastic Variational Deep Kernel Learning (SVDKL) as the base learning model and validate our approach by showing its efficacy in effect prediction tasks where a manipulator robot interacts with objects on a confined tabletop. Experiments comparing MUSEL with strong baselines show that MUSEL improves learning accuracy and sample efficiency. Overall, this study offers MUSEL as an effective online learning model applicable to any robot self-learning task where experience gathering is costly.