/DeepRL_PyTorch

Deep Reinforcement learning codes for study. Currently, there are only codes for algorithms: DQN, C51, QR-DQN, IQN, QUOTA.

Primary LanguagePython

Deep Reinforcement Learning Codes

Currently there are only the codes for distributional reinforcement learning here.

The codes for C51, QR-DQN, and IQN are a slight change from sungyubkim. QUOTA is implemented based on the work of the algorithm's author: Shangtong Zhang. I recently noticed that my DQN code may not get an ideal performance, while other codes run well. I would much appreciate it if someone could point out the errors in my code.

Always up for a chat -- shoot me an email if you'd like to discuss anything.

Dependency:

  • pytorch(>=1.0.0)
  • gym(=0.10.9)
  • numpy
  • matplotlib

Usage:

When your computer's python environment satisfies the above dependencies, you can run the code. For example, enter:

python 3_iqn.py Breakout

on the command line to run the algorithms in the Atari environment. You can change some specific parameters for the algorithms inside the codes.

After training, you can plot the results by running result_show.py with appropriate parameters.

References:

  1. Human-level control through deep reinforcement learning (DQN) [Paper] [Code]

  2. A Distributional Perspective on Reinforcement Learning (C51) [Paper] [Code]

  3. Distributional Reinforcement Learning with Quantile Regression (QR-DQN) [Paper] [Code]

  4. Implicit Quantile Networks for Distributional Reinforcement Learning (IQN) [Paper] [Code]

  5. QUOTA: The Quantile Option Architecture for Reinforcement Learning (QUOTA) [Paper] [Code]