VLN-Trans: Translator for the Vision and Language Navigation Agent
Paper (ACL 2023)
VLN-Trans (https://arxiv.org/pdf/2302.09230.pdf)
Installation
Install the Matterport3D Simulator.
Data Preparation
Please follow the instructions below to prepare the data in directories:
-
MP3D navigability graphs: connectivity maps.
-
Processed fine-grained R2R data and augmented: Fine-grained r2r data.
-
Processed fine-grained R4R data and augmented: Fine-grained r4r data.
-
R2R-Last: R2R-Last.
-
Translator pre-train data (SyFiS): Translator pre-train data.
-
MP3D image features: img features (ResNet-152-Places365).
Initial weights for VLN-trans and Translator
- VLN-trans: Download the
pytorch_model.binfrom here. - Translator: Download the pre-train weights from here.
Trained Network Weights
- VLN-trans trained-weights
R2R Navigation
Please read Peter Anderson's VLN paper for the R2R Navigation task.
Test Navigator
To replicate the performance reported in our paper, load the trained network weights and run validation:
bash run_translator/test_agent.bash
Train Navigator
To train the network, run:
bash run_translator/train_agent.bash