/VizDoom-Keras-RL

Reinforcement Learning in Keras on VizDoom

Primary LanguagePythonMIT LicenseMIT

Implementation of Reinforcement Learning Algorithms in Keras tested on VizDoom

This repo includes implementation of Double Deep Q Network (DDQN), Dueling DDQN, Deep Recurrent Q Network (DRQN) with LSTM, REINFORCE, Advantage Actor Critic (A2C), A2C with LSTM, and C51 DDQN (Distribution Bellman). All implementations are tested on VizDoom Defend the Center scenario, which is a 3D partially observable environment.

For more details on the implementation, you can check out my blog post at https://flyyufelix.github.io/2017/10/12/dqn-vs-pg.html.

Results

Below is the performance chart of 20,000 episodes of DDQN, REINFORCE, and A2C running on Defend the Center. Y-axis is the average number of kills (moving average over 50 episodes).

Performance Chart 1

The performance chart of 15,000 episodes C51 DDQN and DDQN running on Defend the Center.

Performance Chart 2

Usage

First follow this instruction to install VizDoom. If you use python, you can simply do pip install:

$ pip install vizdoom

Second, clone ViZDoom to your machine, copy the python files provided in this repo over to examples/python.

To test if the environment is working, run

$ cd examples/python
$ python ddqn.py

You should see some printouts indicating that the DDQN is running successfully. Errors would be thrown otherwise.

Dependencies