/rlbase

Clean PyTorch implementations of RL Algorithms forked from OpenAI Spinning Up

Primary LanguagePythonOtherNOASSERTION

RL Base

Clean PyTorch implementations of Reinforcement Learning algorithms forked from OpenAI Spinning Up. This serves as a resource for understanding deep RL methods.

This fork has all the PyTorch implementations from the original repo, plus additional algorithms. The new implementations are all following the Spinning Up code format.

Algorithms

The following algorithms have been implemented.

Algorithm Implementation box discrete Multi Processing
REINFORCE [1] RLBase ✔️ ✔️ ✔️
VPG [2, 3] Spinning Up ✔️ ✔️ ✔️
PPO [4] Spinning Up ✔️ ✔️ ✔️
Double DQN [5, 6] RLBase 🔲 ✔️ 🔲
DDPG [7] Spinning Up ✔️ 🔲 🔲
TD3 [8] Spinning Up ✔️ 🔲 🔲
SAC [9] Spinning Up ✔️ 🔲 🔲
HER [10] RLBase ✔️ ✔️ 🔲

Note: This is a work in progress and more algorithms will be added over time.

Installation and Instructions

  • You can install the package using pip. This installs the package and all its dependencies.
# From ~/rlbase
pip install -e .

References

  1. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
  2. Policy Gradient Methods for Reinforcement Learning with Function Approximation
  3. High-Dimensional Continuous Control Using Generalized Advantage Estimation
  4. Proximal Policy Optimization Algorithms
  5. Human-level control through deep reinforcement learning
  6. Deep Reinforcement Learning with Double Q-learning
  7. Continuous control with deep reinforcement learning
  8. Addressing Function Approximation Error in Actor-Critic Methods
  9. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
  10. Hindsight Experience Replay

Algorithms to be Added

  • REINFORCE
  • DQN and Double DQN
  • Hindsight Experience Replay (HER)
  • Option-Critic
  • FeUdal Networks

ToDos

  • Refactor the code to share components among algorithms of the same class
  • Add CNN option for policies
  • Add Tensorboard support