This repository hosts the open source code for Pathdreamer, presented at ICCV 2021.
Paper | Project Webpage | Colab Demo
Set up virtualenv, and install required libraries:
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt
Add the Pathdreamer library to PYTHONPATH:
export PYTHONPATH=$PYTHONPATH:/home/path/to/pathdreamer_root/
We provide a pretrained checkpoint which can be acquired by running:
wget https://storage.googleapis.com/gresearch/pathdreamer/ckpt.tar -P data/
tar -xf data/ckpt.tar --directory data/
The results will be extracted to the data/ckpt directory. Two checkpoints are provided, one for the Stage 1 model (Structure Generator), and another for the Stage 2 model (Image Generator).
Pathdreamer_Example_Colab.ipynb [click to launch in Google Colab] shows how to setup and run the pretrained Pathdreamer model for inference. It includes examples on synthesizing image sequences and continuous video sequences for arbitrary navigation trajectories.
If you find this work useful, please consider citing:
@inproceedings{koh2021pathdreamer,
title={Pathdreamer: A World Model for Indoor Navigation},
author={Koh, Jing Yu and Lee, Honglak and Yang, Yinfei and Baldridge, Jason and Anderson, Peter},
journal={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
year={2021}
}
Pathdreamer is released under the Apache 2.0 license. The Matterport3D dataset is governed by the Matterport3D Terms of Use.
Not an official Google product.