/BtcDet

Behind the Curtain: Learning Occluded Shapes for 3D Object Detection

Primary LanguagePythonApache License 2.0Apache-2.0

Behind the Curtain: Learning Occluded Shapes for 3D Object Detection (AAAI-2022)

Reference

Please cite our paper if you are interested to use this implementation,
Behind the Curtain: Learning Occluded Shapes for 3D Object Detection.

@article{xu2021behind,
  title={Behind the Curtain: Learning Occluded Shapes for 3D Object Detection},
  author={Xu, Qiangeng and Zhong, Yiqi and Neumann, Ulrich},
  journal={arXiv preprint arXiv:2112.02205},
  year={2021}
}

The implementatin is also inspired by the ICCV-2021 paper,
SPG: Unsupervised domain adaptation for 3d object detection via semantic point generation.

@inproceedings{xu2021spg,
  title={Spg: Unsupervised domain adaptation for 3d object detection via semantic point generation},
  author={Xu, Qiangeng and Zhou, Yin and Wang, Weiyue and Qi, Charles R and Anguelov, Dragomir},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={15446--15456},
  year={2021}
}

Acknowledgement

Our model, BtcDet, is implemented based on [OpenPcdet 0.3.0]. We thank Shaohuai Shi for the discussion during our implementation.

Installation

Requirements

All the codes are tested in the following environment:

Install

b. Install the dependent libraries as follows:

  • Install the dependent python libraries:
pip install -r requirements.txt 
  • Install the SparseConv library, we use the implementation from [spconv].
    • The very recent spconv 2.+ has significant interface changes and some issues in operators. Therefore, we do not support it for now.
    • If you use PyTorch 1.1, then make sure you install the spconv v1.0 with (commit 8da6f96) instead of the latest one.
    • If you use PyTorch 1.3+, then you need to install the spconv v1.2. As mentioned in v1.2.1 of spconv (https://github.com/traveller59/spconv/tree/v1.2.1)
    git clone -b v1.2.1  https://github.com/traveller59/spconv.git --recursive
    
    cd spconv
    
    sudo apt-get install libboost-all-dev
    
    python setup.py bdist_wheel
    
    cd ./dist 
    
    Inside the dist directory, use pip to install generated whl file.
    pip install spconv-1.2.1-{your system info}.whl
    
    After that, you should first get out of the spconv directory, then do python import spconv to see if you installed it correctly.

c. Install the operators in BtcDet by running the following command:

cd BtcDet
python setup.py develop

Preparation

Use Our Preprocessed Data:

you can use our generated kitti's data including the generated complete object points, download it [here (about 31GBs)] and put the zip file inside data/kitti/ and unzip it as detection3d directory.

Alternatively, Generate Data by Yourself:

####KITTI Dataset

  • Please download the official KITTI 3D object detection dataset and organize the downloaded files as follows (the road planes could be downloaded from [road plane], which are optional for data augmentation in the training):
BtcDet
├── data
│   ├── kitti
    │   │   │──detection3d  │── ImageSets
                    │   │   │── training
                    │   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes) & (optional: depth_2)
                    │   │   │── testing
                    │   │   │   ├──calib & velodyne & image_2
  • Generate the data infos by running the following command:
python -m btcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml

Generate Approximated complete object points:

(Under BtcDet directory, execute:)

python -m btcdet.datasets.multifindbestfit

Run Training:

cd tools/

Single gpu training

mkdir output

mkdir output/kitti_car

python train.py --cfg_file ./cfgs/model_configs/btcdet_kitti_car.yaml --output_dir ../output/kitti_car/ --batch_size 2 --gpu_str "0"

Multi gpu training, assuming you have 4 gpus:

bash scripts/dist_train.sh 4  --batch_size 8 --gpu_str "0,1,2,3" --cfg_file ./cfgs/model_configs/btcdet_kitti_car.yaml --output_dir ../output/kitti_car/

Run Testing:

cd tools/

Single gpu testing for all saved checkpoints, assuming you have 4 gpus:

python test.sh 4 --eval-all --cfg_file ./cfgs/model_configs/btcdet_kitti_car.yaml --gpu_str "0" --batch_size 2 --output_dir ../output/kitti_car/ --ckpt_dir  ../output/kitti_car/ckpt/

Multi gpu testing for all saved checkpoints, assuming you have 4 gpus:

bash scripts/dist_test.sh 4 --eval-all --cfg_file ./cfgs/model_configs/btcdet_kitti_car.yaml --gpu_str "0,1,2,3" --batch_size 8 --output_dir ../output/kitti_car/ --ckpt_dir  ../output/kitti_car/ckpt/

Multi gpu testing a specific checkpoint, assuming you have 4 gpus and checkpoint_39 is your best checkpoint :

bash scripts/dist_test.sh 4  --cfg_file ./cfgs/model_configs/btcdet_kitti_car.yaml --gpu_str "0,1,2,3" --batch_size 8 --output_dir ../output/kitti_car/ --ckpt  ../output/kitti_car/ckpt/checkpoint_epoch_39.pth