LIGA-Stereo

Introduction

This is the official implementation of the paper LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector, In ICCV'21, Xiaoyang Guo, Shaoshuai Shi, Xiaogang Wang and Hongsheng Li.

[project page] [paper] [code]

Overview

Installation
Getting Started
Pretrained Models
Citation

Installation

Requirements

All the codes are tested in the following environment:

Linux (tested on Ubuntu 14.04 / 16.04)
Python 3.7
PyTorch 1.6.0
Torchvision 0.7.0
CUDA 9.2 / 10.1
spconv (commit f22dd9)

Installation Steps

a. Clone this repository.

git clone https://github.com/xy-guo/LIGA.git

b. Install the dependent libraries as follows:

Install the dependent python libraries:

pip install -r requirements.txt

Install the SparseConv library, we use the implementation from [spconv].

git clone https://github.com/traveller59/spconv
git reset --hard f22dd9
git submodule update --recursive
python setup.py bdist_wheel
pip install ./dist/spconv-1.2.1-cp37-cp37m-linux_x86_64.whl

Install modified mmdetection from [mmdetection_kitti]

git clone https://github.com/xy-guo/mmdetection_kitti
python setup.py develop

c. Install this library by running the following command:

python setup.py develop

Getting Started

The dataset configs are located within configs/stereo/dataset_configs, and the model configs are located within configs/stereo for different datasets.

Dataset Preparation

Currently we only provide the dataloader of KITTI dataset.

Please download the official KITTI 3D object detection dataset and organize the downloaded files as follows (the road planes are provided by OpenPCDet [road plane], which are optional for training LiDAR models):

LIGA_PATH
├── data
│   ├── kitti
│   │   │── ImageSets
│   │   │── training
│   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes)
│   │   │── testing
│   │   │   ├──calib & velodyne & image_2
├── configs
├── liga
├── tools

You can also choose to link your KITTI dataset path by

YOUR_KITTI_DATA_PATH=~/data/kitti_object
ln -s $YOUR_KITTI_DATA_PATH/training/ ./data/kitti/
ln -s $YOUR_KITTI_DATA_PATH/testing/ ./data/kitti/

Generate the data infos by running the following command:

python -m liga.datasets.kitti.kitti_dataset create_kitti_infos
python -m liga.datasets.kitti.kitti_dataset create_gt_database_only

Training & Testing

Test and evaluate the pretrained models

To test with multiple GPUs:

./scripts/dist_test_ckpt.sh ${NUM_GPUS} ./configs/stereo/kitti_models/liga.yaml ./ckpt/pretrained_liga.pth

Train a model

Train with multiple GPUs

./scripts/dist_train.sh ${NUM_GPUS} 'exp_name' ./configs/stereo/kitti_models/liga.yaml

Pretrained Models

Google Drive

Citation

@InProceedings{Guo_2021_ICCV,
    author = {Guo, Xiaoyang and Shi, Shaoshuai and Wang, Xiaogang and Li, Hongsheng},
    title = {LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month = {October},
    year = {2021}
}

Acknowledgements

Part of codes are migrated from OpenPCDet and DSGN.

xy-guo/LIGA-Stereo