Welcome to Spinning Up in Deep RL!

This is an educational resource produced by OpenAI that makes it easier to learn about deep reinforcement learning (deep RL).

For the unfamiliar: reinforcement learning (RL) is a machine learning approach for teaching agents how to solve tasks by trial and error. Deep RL refers to the combination of RL with deep learning.

This module contains a variety of helpful resources, including:

a short introduction to RL terminology, kinds of algorithms, and basic theory,
an essay about how to grow into an RL research role,
a curated list of important papers organized by topic,
a well-documented code repo of short, standalone implementations of key algorithms,
and a few exercises to serve as warm-ups.

Get started at spinningup.openai.com!

Installation

Installing OpenMPI

Ubuntu

sudo apt-get update && sudo apt-get install libopenmpi-dev

Mac OS X

Installation of system packages on Mac requires Homebrew. With Homebrew installed, run the follwing:

brew install openmpi

Installing Spinning Up Pytorch

git clone https://github.com/KongCDY/spinningup_pytorch.git
cd spinningup_pytorch
pip install -e .

Check Your Install

To see if you’ve successfully installed Spinning Up, try running PPO in the LunarLander-v2 environment with

python -m spinup_pt.run ppo --hid "[32,32]" --env LunarLander-v2 --exp_name installtest --gamma 0.999

This might run for around 10 minutes, and you can leave it going in the background while you continue reading through documentation. This won’t train the agent to completion, but will run it for long enough that you can see some learning progress when the results come in.

After it finishes training, watch a video of the trained policy with

python -m spinup_pt.run test_policy data/installtest/installtest_s0

And plot the results with

python -m spinup_pt.run plot data/installtest/installtest_s0

Other options see official webpage.