Sample-Efficient Deep Reinforcement Learning for Continuous Control

Abstract

Sample efficiency in deep reinforcement learning (DRL) is measured by the amount of new data collected to learn a task. It is one of the most important research topics in DRL. Collecting new data points with DRL can be expensive and time-consuming, especially in complex continuous control tasks such as robotic manipulation. So a low sample efficiency can severely limit the application of DRL to challenging real-world problems. In this work, we try to improve the sample efficiency of DRL algorithms from three aspects: 1) Improve robustness and performance with aggressive data exploitation and regularization. 2) Take a data-driven approach, make use of out-of-domain data, offline and online RL data. 3) Obtain a better fundamental understanding with analysis and ablations. We show that with the right design, we can significantly improve sample efficiency on popular simulated robotics benchmarks such as OpenAI Gym MuJoCo and Adroit, reducing the time and money required to learn complex control tasks, and providing novel DRL solutions that are more effective, practical, and accessible.