A Surface Geometry Model for LiDAR Depth Completion (IEEE RA-L)
Abstract:
We develop a non-learning depth completion pipeline with the assumption that the empty pixel shares the same surface with the nearest value. The pipeline has two major technical modules: an outlier removal and a surface model to explicitly calculate the depth with a mathematical equation. Please check our paper for more details:
The arxiv version: https://arxiv.org/pdf/2104.08466.pdf
The IEEE RAL version: https://ieeexplore.ieee.org/abstract/document/9387169?casa_token=m6hbd7riBioAAAAA:bsyIXoghUPIyvK_oASnZAATLqq1h1yt2gAEQbMPLSvdOl0zRsFadmPPuYTPnIEDwlrdW8bM
Method Pipeline:
Outlier Removal Examples:
Data Structure
Download the validation and test dataset from KITTI depth completion benchmark website, then put them under the fodler depth_selection with the stucture:
----- depth_selection
|
-------val_selection_cropped
|
-------test_depth_completion_anonymous
Environment Settings
Note: we have two different outlier removal methods. The function outlier_removal() in tools.py is a simple opencv filter-based algorithm. It can work on 64 line LiDAR but need mannually adjust two hyperparameters, thus can not directly extend on sparser 16 line or 32 line LiDAR. To design a more general outlier removal algoritem, we think from first principal rule and have an alternative one in non_learning_completion_multi_res.py. The function outlier_removal_mask() is parameter-free, that we believe more general. However, we only have one month to prepare the reply of our RA-L submission, so the current implementation utilize the extract_patch function in tensorflow. This is why the tensorflow 2.0 is temporary needed. we will implement a c++ version just like the convd operator in OpenCV in the short future.
Pull the docker image:
docker pull tensorflow/tensorflow:2.1.0-gpu-py3-jupyter
Activate the docker container with mounting the code in this repository.
Install packages with the certain version in the docker container:
bash install_dependency.sh
To subsample the 32 line and 16 line LiDAR, please run:
python subsample_Lidar_val.py
After installing all packages, you can open the jupyter note book by bash jupyter.sh, and use depth_completion.ipynb to explore each step of our method.
Run the Code
Run the completion with 64 line LiDAR on KITTI validation (simple outlier removal):
python non_learning_completion_CPU_64line.py
Run the completion with 64 line LiDAR on KITTI test (simple outlier removal):
python non_learning_completion_CPU_64line_test.py
Run the completion with 64 line LiDAR on KITTI validation (general outlier removal):
python non_learning_completion_multi_res.py
Run the completion with 64 line LiDAR on KITTI test (general outlier removal):
python non_learning_completion_multi_res_test.py
Publication
Please cite the paper if you use this code:
@article{zhao2021surface,
title={A Surface Geometry Model for LiDAR Depth Completion},
author={Zhao, Yiming and Bai, Lin and Zhang, Ziming and Huang, Xinming},
journal={IEEE Robotics and Automation Letters},
volume={6},
number={3},
pages={4457--4464},
year={2021},
publisher={IEEE}
}