Meta-Reinforcement Learning Algorithms

A PyTorch implementation of meta-reinforcement learning algorithms, RL^2 PPO, MGRL, SNAIL, and VariBAD.

Setup

Install the packages using the requirements.txt file.

# using conda
conda create --name meta_rl --file requirements.txt
# Or pip
pip install requirements.txt

Usage

Run experiments by using the following example command:

python main.py --name experiment_name -c configs/rl2_ppo.yml

Algorithms

RL^2 Proximal Policy Optimization (PPO)
Meta-Gradient Reinforcement Learning (A2C)
VariBAD
SNAIL

Ideas for:

Proximal Policy Optimization with Episodic Planning Networks (EPNs)

Results

Initial results showing the convergence of meta-gradient reinforcement learning with A2C. The inner-loop optimizes on the CartPole environment and in the outer-loop the gamma value is cross-validated and updated by gradient descent. The current setting shows similar performance to the regular A2C algorithm and the implementation might benefit from conditioning the value and policy on gamma value embeddings.

References