Active Neural Localization

This is a PyTorch implementation of the ICLR-18 paper:

Active Neural Localization
Devendra Singh Chaplot, Emilio Parisotto, Ruslan Salakhutdinov
Carnegie Mellon University

Project Website: https://devendrachaplot.github.io/projects/Neural-Localization

This repository contains:

Code for the Maze2D Environment which generates random 2D mazes for active localization.
Code for training an Active Neural Localization agent in Maze2D Environment using A3C.

Dependencies

PyTorch (v0.3)

Usage

Training

For training an Active Neural Localization A3C agent with 16 threads on 7x7 mazes with maximum episode length 30:

python a3c_main.py --num-processes 16 --map-size 7 --max-episode-length 30 --dump-location ./saved/ --test-data ./test_data/m7_n1000.npy

The code will save the best model at ./saved/model_best and the training log at ./saved/train.log. The code uses ./test_data/m7_n1000.npy as the test data and makes sure that any maze in the test data is not used while training.

Evaluation

After training, the model can be evaluated using:

python a3c_main.py --num-processes 0 --evaluate 1 --map-size 7 --max-episode-length 30 --load ./saved/model_best --test-data ./test_data/m7_n1000.npy

Pre-trained models

The pretrained_models directory contains pre-trained models for map-size 7 (max-episode-length 15 and 30), map-size 15 (max-episode-length 20 and 40) and map-size 21 (max-episode-length 30 and 60). The test data used for training these models is provided in the test_data directory.

For evaluating a pre-trained model on maze size 15x15 with maximum episode length 40:

python a3c_main.py --num-processes 0 --evaluate 1 --map-size 15 --max-episode-length 40 --load ./pretrained_models/m15_l40 --test-data ./test_data/m15_n1000.npy

Generating test data

The repository contains test data of map-sizes 7, 15 and 21 with 1000 mazes each in the test_data directory.

For generating more test data:

python generate_test_data.py --map-size 7 --num-mazes 100 --test-data-location ./test_data/ --test-data-filename my_new_test_data.npy

This will generate a test data file at test_data/my_new_test_data.npy containing 100 7x7 mazes.

All arguments

All arguments for a3c_main.py:

  -h, --help            show this help message and exit
  -l L, --max-episode-length L
                        maximum length of an episode (default: 30)
  -m MAP_SIZE, --map-size MAP_SIZE
                        m: Size of the maze m x m (default: 7), must be an odd
                        natural number
  --lr LR               learning rate (default: 0.001)
  --num-iters NS        number of training iterations per training thread
                        (default: 10000000)
  --gamma G             discount factor for rewards (default: 0.99)
  --tau T               parameter for GAE (default: 1.00)
  --seed S              random seed (default: 1)
  -n N, --num-processes N
                        how many training processes to use (default: 8)
  --num-steps NS        number of forward steps in A3C (default: 20)
  --hist-size HIST_SIZE
                        action history size (default: 5)
  --load LOAD           model path to load, 0 to not reload (default: 0)
  -e EVALUATE, --evaluate EVALUATE
                        0:Train, 1:Evaluate on test data (default: 0)
  -d DUMP_LOCATION, --dump-location DUMP_LOCATION
                        path to dump models and log (default: ./saved/)
  -td TEST_DATA, --test-data TEST_DATA
                        Test data filepath (default: ./test_data/m7_n1000.npy)

Cite as

Chaplot, Devendra Singh, Parisotto, Emilio and Salakhutdinov, Ruslan. Active Neural Localization. In International Conference on Learning Representations, 2018. (PDF)

Bibtex:

@inproceedings{chaplot2018active,
  title={Active Neural Localization},
  author={Chaplot, Devendra Singh and Parisotto, Emilio and Salakhutdinov, Ruslan},
  booktitle={International Conference on Learning Representations},
  year={2018}
}

Acknowledgements

The implementation of A3C is borrowed from https://github.com/ikostrikov/pytorch-a3c.

devendrachaplot/Neural-Localization