Apprentissage profond par Renforcement et Démonstrations, pour le comportement de robots manipulateurs

Despite having known great success, reinforcement-learning algorithms still need to become more sample-efficient, particularly for robotics where it is much harder to train an agent outside of simulation. As the community leans towards data-driven approaches (offline reinforcement learning, decision transformers, etc.), in this thesis we focus on off-policy reinforcement learning, and explore different ways of incorporating additional data into the algorithms. In particular, we rely on expert demonstrations, which can help with efficiency as well as overall performance. The goal is to design efficient algorithms to solve a range of robotic-manipulation tasks, such as flipping a switch or sliding a cube on a table.After a thorough review of the reinforcement-learning and imitation-learning frameworks, we first introduce our reward-relabeling method, which can be seen as a form of reward shaping that happens in hindsight, once the entire episode is collected. This approach can easily extend any off-policy algorithm to benefit from both reinforcement and imitation signals. Building on this method, we then introduce a more efficient algorithm that aggregates previous and concurrent works that also address similar concerns.Finally, we move onto the more realistic setting of vision-based reinforcement learning. To tackle this problem, we design a two-stage training pipeline: first learn a visual representation of the scene by pre-training an encoder from multiple supervised computer-vision objectives, then train a reinforcement-learning agent which can focus solely on solving the task. Despite all the data being collected in simulation, the experiments include one sim-to-real example to show that these techniques can translate to real-world controlled environments.