/BEVStereo

Official code for BEVStereo

Primary LanguagePythonMIT LicenseMIT

BEVStereo

BEVStereo is a new multi-view 3D object detector using temporal stereo to enhance depth estimation.

Updates!!

  • 【2022/09/22】 We released our paper on Arxiv.
  • 【2022/08/24】 We submitted our result on nuScenes Detection Task and achieved the SOTA.

Quick Start

Installation

Step 0. Install pytorch(v1.9.0).

Step 1. Install MMDetection3D(v1.0.0rc4).

Step 2. Install requirements.

pip install -r requirements.txt

Step 3. Install BEVDepth(gpu required).

python setup.py develop

Data preparation

We use the same data format as BEVDepth, if you have processed it before, please skip.

Step 0. Download nuScenes official dataset.

Step 1. Symlink the dataset root to ./data/.

ln -s [nuscenes root] ./data/

The directory will be as follows.

BEVDepth
├── data
│   ├── nuScenes
│   │   ├── maps
│   │   ├── samples
│   │   ├── sweeps
│   │   ├── v1.0-test
|   |   ├── v1.0-trainval

Step 2. Prepare infos.

python scripts/gen_info.py

Step 3. Prepare depth gt.

python scripts/gen_depth_gt.py

Tutorials

Train.

python [EXP_PATH] --amp_backend native -b 8 --gpus 8

Eval.

python [EXP_PATH] --ckpt_path [CKPT_PATH] -e -b 8 --gpus 8

Benchmark

Exp Frames EMA CBGS mAP mATE mASE mAOE mAVE mAAE NDS weights
R50 key + sweep4 0.3427 0.6560 0.2784 0.5982 0.5347 0.2228 0.4423 github
R50 key + sweep4 √ 0.3435 0.6585 0.2757 0.5792 0.5034 0.2163 0.4485 github
R50 key + key 0.3456 0.6589 0.2774 0.5500 0.4980 0.2278 0.4516 github
R50 key + key √ 0.3494 0.6671 0.2785 0.5606 0.4686 0.2295 0.4543 github
R50 key + key √ 0.3576 0.6071 0.2684 0.4157 0.3928 0.2021 0.4902 github
R50 key + key √ √ 0.3721 0.5980 0.2701 0.4381 0.3672 0.1898 0.4997 github

Acknowledgments

This project exists thanks to all the people who instruct. @Haotian-Zh @xavierwu95 @Tai-Wang