/stream3dppe

StreamPETR with 3dppe Extension

Primary LanguagePythonApache License 2.0Apache-2.0

StreamPETR with 3dppe Extension

Introduction

This repository is an implementation of StreamPETR with 3dppe.

Getting Started

  1. Prepare nuScenes dataset and generate 2D annotations and temporal information for training & evaluation. (see streamPETR)

  2. Conda env

conda create -n xxx python=3.8 -y
conda activate xxx
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install flash-attn==0.2.2  # (Tesla v100 is not compatible)

pip install mmcv-full==1.6.0
pip install mmdet==2.28.2
pip install mmsegmentation==0.30.0
git clone https://github.com/open-mmlab/mmdetection3d.git
cd mmdetection3d
git checkout v1.0.0rc6 
pip install -v -e .

Note : make sure that numba 0.53.0 numpy 1.23.5
(if not, reinstall numba==0.53.0).

Catalogue: tree -d -L 1 

.
├── ckpts
├── data
├── mmdetection3d
├── projects
└── tools
  1. Train & Infer
tools/dist_train.sh [-config] [-num_gpus]
tools/dist_test.sh [-config] [-model] [-num_gpus] --eval bbox

Results on NuScenes Val Set

Model Setting Pretrain Lr Schd Training Time NDS mAP Config Download
StreamPETR V2-99-900q-800x320 FCOS3D 24ep 13h 57.1 48.3 config model/log
Stream3dppe V2-99-900q-800x320 FCOS3D 24ep 16h 58.45/58.45 49.95/50.04 config （model1,model2)/(log1,log2)
Stream3dppe_gt_detph V2-99-900q-800x320 FCOS3D 24ep 22h 61.7 55.3 config model/log
StreamPETR V2-99-900q-1600x640 FCOS3D 24ep
Stream3DPPE V2-99-900q-1600x640 FCOS3D 24ep

Note : Stream3dppe is trained on 4 x RTX 3090 with bs4 ,while Stream3dppe_gt_detph is trained on 4 x RTX 2080Ti with bs2 .

More result please refer to https://github.com/drilistbox/3DPPE.

Acknowledgement

Many thanks to the authors of PETR and StreamPETR.

Citation

If you find this project useful for your research, please consider citing:

@article{shu20233DPPE,
  title={3DPPE: 3D Point Positional Encoding for Multi-Camera 3D Object Detection Transformers},
  author={Shu, Changyong and Deng, Jiajun and Yu, Fisher and Liu, Yifan},
  journal={arXiv preprint arXiv:2211.14710},
  year={2023}
}