DeepLiDAR

This repository contains the code (in PyTorch) for "DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image" paper (CVPR 2019) by Jiaxiong Qiu, Zhaopeng Cui, Yinda Zhang, Xingdi Zhang, Shuaicheng Liu, Bing Zeng and Marc Pollefeys.

Introduction

In this work, we propose an end-to-end deep learning system to produce dense depth from sparse LiDAR data and a color image taken from outdoor on-road scenes leveraging surface normal as the intermediate representation.

Requirements

Python2.7
PyTorch(0.4.0+)
torchvision 0.2.0 (higher version may cause issues)
KITTI Depth Completion

Pretrained Model

※NOTE: The pretrained model were saved in '.tar'; however, you don't need to untar it. Use torch.load() to load it.

Download Link

Train

Get the surface normal of Lidar dataset by run the code in the folder named 'surface_normal'.
Use the training strategy in the folder named 'trainings'.

Evaluation

Fill the names of the folders in 'test.py':

'gt_fold': the location of your groundtruth folder;
'left_fold': the location of your RGB image folder;
'lidar2_raw': the location of your Sparse(LiDAR) depth folder.

Use the following command to evaluate the trained on your own data.

python test.py --loadmodel (your trained model)

Citation

If you use our code or method in your work, please cite the following:

@inproceedings{qiu2018deeplidar,
  title={DeepLiDAR: Deep Surface Normal Guided Depth Prediction for Outdoor Scene from Sparse LiDAR Data and Single Color Image},
  author={Qiu, Jiaxiong and Cui, Zhaopeng and Zhang, Yinda and Zhang, Xingdi and Liu, Shuaicheng and Zeng, Bing and Pollefeys, Marc},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year={2019}
}

Please direct any questions to Jiaxiong Qiu at qiujiaxiong727@gmail.com

youngguncho/DeepLiDAR