Generalizable-BEV
A plug-and-play BEV generalization framework that can leverage both unlabeled and labeled data.
Get Started
BEVDet.
step 1. Prepare the environment refer tostep 2. Prepare PDBEV repo by.
git clone https://github.com/EnVision-Research/Generalizable-BEV.git
cd Generalizable-BEV
pip install -v -e .
step 3. Prepare datasets:
The preparation of the dataset is actually to generate the corresponding index (pkl files), which can then be used with the dataset that we have created.
Details
nuScenes dataset pkl file generation refers toDetails, and then use ./tools/Deepaccident_converter.py to convert to a uniform format.
DeepAccident dataset pkl file generation refers toLyft use ./tools/Lyft_converter.py to convert to a uniform format.
The pre-processed pkl of the three data sets can be downloaded directly [here].
step 4. Train for domain generalization:
bash tools/dist_train.sh $confige_file$ $Gpus_num$
For example:
bash tools/dist_train.sh ./configs/PDBEV/pdbev-r50-cbgs-NUS2X-dg.py 8 # nuScenes as source domain, using 8 gpus
bash tools/dist_train.sh ./configs/PDBEV/pdbev-r50-cbgs-LYFT2X-dg.py 8 # Lyft as source domain, using 8 gpus
bash tools/dist_train.sh ./configs/PDBEV/pdbev-r50-cbgs-DA2X-dg.py 8 # DeepAccident as source domain, using 8 gpus
step 5. Train for unspuervised domain adapataion:
bash tools/dist_train.sh $confige_file$ c --checkpoint $the pretrained models on source domain$
For example:
bash tools/dist_train.sh ./configs/PDBEV/pcbev-uda-NUS2LYFT.py 8 --checkpoint ./work_dirs/pdbev-r50-cbgs-NUS2X-dg/epoch_23.pth
# nuScenes as source domain, LYFT as target domain, using 8 gpus, loading DG pretrain models at 23 epoch
# You only need to modify the path of the configuration file of different data set D and the corresponding model M to test the performance of model M on the corresponding data set D. It is worth mentioning that none of our algorithms change the model infrastructure, so they are only used for BEVDepth evaluation.
step 6. Test at target domain:
bash ./tools/dist_test.sh &test dataset config_file& &model_path& $Gpus_num$ --eval bbox --out $output_path$
For example:
bash ./tools/dist_test.sh ./configs/bevdet_our/bevdepth-r50-cbgs-pc-lyft.py ./work_dirs/pdbev-r50-cbgs-NUS2X-dg/epoch_24.pth 8 --eval bbox --out ./work_dirs/bevdepth-r50-cbgs-pc-nus/nus.pkl
Acknowledgement
This project is not possible without multiple great open-sourced code bases. We list some notable examples: BEVDet, DeepAccident, Lyft.
Citation
@InProceedings{PD-BEV,
author = {Hao LU, Yunpeng ZHANG, Qing LIAN, Dalong DU, Ying-Cong CHEN},
title = {Towards Generalizable Multi-Camera 3D Object Detection via Perspective Debiasing},
booktitle = {arXiv preprint arXiv:2310.11346},
year = {2023},
}