/LEAStereo

Hierarchical Neural Architecture Searchfor Deep Stereo Matching (NeurIPS 2020)

Primary LanguagePythonMIT LicenseMIT

LEAStereo

This repository contains the code for our NeurIPS 2020 paper Hierarchical Neural Architecture Searchfor Deep Stereo Matching [NeurIPS 20]

alt text

Requirements

Environment

  1. Python 3.8.*
  2. CUDA 10.0
  3. PyTorch
  4. TorchVision

Install

Create a virtual environment and activate it.

conda create -n leastereo python=3.8
conda activate leastereo

The code has been tested with PyTorch 1.6 and Cuda 10.2.

conda install pytorch=1.6.0 torchvision=0.7.0 cudatoolkit=10.2 -c pytorch
conda install matplotlib path.py tqdm
conda install tensorboard tensorboardX
conda install scipy scikit-image opencv

Install Nvidia Apex

Follow the instructions here. Apex is required for mixed precision training. Please do not use pip install apex - this will not install the correct package.

Dataset

To evaluate/train our LEAStereo network, you will need to download the required datasets.

Change the first column path in file create_link.sh with your actual dataset location. Then run create_link.sh that will create symbolic links to wherever the datasets were downloaded in the datasets folder. For Middlebury 2014 dataset, we perform our network on half resolution images.

├── datasets
    ├── SceneFlow
        ├── camera_data
        ├── disparity
        ├── frames_finalpass
    ├── kitti2012
        ├── testing
        ├── training
    ├── kitti2015
        ├── testing
        ├── training
    ├── MiddEval3
        ├── testH
        ├── trainingH

Prediction

You can evaluate a trained model using prediction.sh for each dataset, that would help you generate *.png or *.pfm images correspoding to different datasets.

sh predict_sf.sh
sh predict_md.sh
sh predict_kitti12.sh
sh predict_kitti15.sh

Results of our model on three benchmark datasets could also be found here

Architecture Search

Three steps for the architecture search:

1. Search

sh search.sh

2. decode

sh decode.sh

3. retrain

sh train_sf.sh
sh train_md.sh
sh train_kitti12.sh
sh train_kitti15.sh

Acknowledgements

This repository makes liberal use of code from [AutoDeeplab] [pytorch code(Non-official)].

Citing

If you find this code useful, please consider to cite our work.

@article{cheng2020hierarchical,
  title={Hierarchical Neural Architecture Search for Deep Stereo Matching},
  author={Cheng, Xuelian and Zhong, Yiran and Harandi, Mehrtash and Dai, Yuchao and Chang, Xiaojun and Li, Hongdong and Drummond, Tom and Ge, Zongyuan},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  year={2020}
}