Note: This repository is an attempt to convert the OpenAI's SpinningUP from Tensorflow to Pytorch.
Status: In-progress, see the list of what's working at the bottom of the README.
Status OpenAI: Active (under active development, breaking changes may occur)
Welcome to Spinning Up in Deep RL with Pytorch!
This is an educational resource originally produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).
For the unfamiliar: reinforcement learning (RL) is a machine learning approach for teaching agents how to solve tasks by trial and error. Deep RL refers to the combination of RL with deep learning.
This module contains a variety of helpful resources, including:
- a short introduction to RL terminology, kinds of algorithms, and basic theory,
- an essay about how to grow into an RL research role,
- a curated list of important papers organized by topic,
- a well-documented code repo of short, standalone implementations of key algorithms,
- and a few exercises to serve as warm-ups.
Get started at spinningup.openai.com!
Working:
- Setup process (now in python 3.7)
- Logger (with model saving)
- VPG (CPU, single GPU)
- PPO (CPU, single GPU)
- DDPG (CPU, single GPU)
- TD3 (CPU, single GPU)
- SAC (CPU, single GPU)
TODO (up next):
- TRPO (Comming soon!)
- Tensorflow vs Pytorch Benchmark
- MPI pytorch
- Excercise section
Known issues:
- A difference has been noticed when training a simple environment, for TF default parameters would get Cartpole AverageEpRet to 100 for pytorch it barely gets over 40, (if the lr is changed to 0.01 for both PI and V learners, then it easily reaches 200)