This repo includes implementation of C51 Algorithm describe in this paper. The implementation is tested on the VizDoom Defend the Center scenario, which is a 3D partially observable environment.
For tutorial on Distributional Bellman and step-by-step walkthrough of the implementation, please check out my blog post at https://flyyufelix.github.io/2017/10/24/distributional-bellman.html.
Below is the performance chart of 15,000 episodes of C51 DDQN and DDQN running on Defend the Center. Y-axis is the average number of kills (moving average over 50 episodes).
First follow this instruction to install VizDoom. If you use python, you can simply do pip install:
$ pip install vizdoom
Second, clone ViZDoom to your machine, copy the python files provided in this repo over to examples/python.
Next, edit scenarios/defend_the_center.cfg file. Replace this line
available_game_variables = { AMMO2 HEALTH }
with
available_game_variables = { KILLCOUNT AMMO2 HEALTH }
To test if the environment is working, run
$ cd examples/python
$ python c51_ddqn.py
You should see some printouts indicating that the C51 DDQN is running successfully. Errors would be thrown otherwise.
- Keras 1.2.2 / 2.0.5
- Tensorflow 0.12.0 / 1.2.1
- VizDoom Environment