An Exploration of Count-Based Exploration Methods in Text-Based Games

Framework and model code for the paper "An Exploration of Count-Based Exploration Methods in Text-Based Games", which was also used as a submission for Microsoft TextWorld competition.

Paper is added to the repository under the name report.pdf

Installation

TextWorld

Follow these steps to set up TextWorld. Adapted from the TextWorld repo.

Get required system libraries. For macOS:

brew install libffi curl git

Create and activate a Conda environment:

conda create --name textworld python=3.7
conda activate textworld

Make sure the environment is activated by checking that the terminal input says (textworld) before continuing. Install Python packages using pip:

pip install https://github.com/Microsoft/TextWorld/archive/master.zip

To create a game:

tw-make custom --world-size 5 --nb-objects 10 --quest-length 5 --seed 1234 --output tw_games/custom_game.ulx

To play the game:

tw-play tw_games/custom_game.ulx

Training data

Get the data from the CodaLab Competition at: Participate tab > Files > Public Data.

Unzip it into the repo's root directory (where this file is) and name the folder train so that it gets ignored by git.

LSTM-DQN

With the textworld conda environment activated, and making sure that which pip points to the pip inside conda:

pip install spacy torch

Download the English language model:

python -m spacy download en

Training

Make sure to edit config.yaml; use the following naming convention for experiments: yyyy_mm_dd_name_experiment, e.g. 2019_02_20_leon_initial_experiments.

To train on all games:

python train.py ../../train

train.py ../../train/some-game.ulx -c  config.yaml