Code of our ICLR 2018 paper.
The code can be run in a docker container as described below. The dockerfile template in docker/Dockerfile.cuda.template lists all requirements that may be needed to set up a non-dockerised environment. The core requirements are pytorch, gym, and sacred.
The configuration and logging is handled by Sacred. Results are stored by the FileStorageObserver as .json's in results/. We recommend a MongoObserver (requires pymongo) to organise larger numbers of experiments.
Valid configuration options are documented in conf/default.yaml.
The default settings correspond to our Atari experiments on Seaquest with TreeQN, depth 2.
To run a default setup with the configuration specified in conf/default.yaml, simply execute:
python treeqn/nstep_run.py
Further parameters can be specified using with:
python treeqn/nstep_run.py with env_id=Qbert architecture=dqn
Configuration files can also be used. Our box-pushing experiment defaults are given in conf/push.yaml:
python treeqn/nstep_run.py with config=./conf/push.yaml
If you have Docker installed, you can build a docker image tagged treeqn with:
cd docker
./build.sh
cd ..
To run an experiment in a detached docker container named treeqn-$GPU_ID, use:
./docker/run.sh $GPU_ID python treeqn/nstep_run.py
@inproceedings{farquhar2018treeqn,
title={TreeQN and ATreeC: Differentiable Tree-Structured Models for Deep Reinforcement Learning},
author={Farquhar, Gregory and Rockt{\"a}schel, Tim and Igl, Maximilian and Whiteson, Shimon},
booktitle={ICLR 2018},
year={2018}
}