This repository implements multimodal models trained on aiMotive Multimodal Dataset. For more details, please refer to our paper on Arxiv. The code is built on the top of BEVDepth repository. The code has been tested on python 3.8.
Create a conda environment
conda create --name mmtraining python=3.8
conda activate mmtraining
Step 0. Install pytorch(v1.9.0).
pip install torch==1.9.0+cu102 torchvision==0.10.0+cu102 -f https://download.pytorch.org/whl/torch_stable.htmlStep 1. Install MMDetection3D(v1.0.0rc4).
pip install -U openmim
mim install mmcv-full==1.7.0
mim install mmsegmentation==0.28.0
mim install mmdet==2.25.1
pip install mmdet3d==1.0.0.rc4Step 2. Install requirements.
pip install -r requirements.txtIf you get an error message claiming 'No matching distribution found for waymo-open-dataset', then you can comment out that line from mmdetection3d/requirements/optional.txt.
Step 3. Install mm_training (gpu required).
python setup.py developStep 1. Download aiMotive Multimodal Dataset.
Step 2. Change data_root variable in exps/conf_aim.py to the path of downloaded dataset.
The config parameters are contained by exps/conf_aim.py. You need to override its values. Example config files can be found under exps/configs.
Train.
PYTHONPATH=$PYTHONPATH: python exps/mm_training_aim.py
Eval.
PYTHONPATH=$PYTHONPATH: python exps/eval.py
Inference.
PYTHONPATH=$PYTHONPATH: python exps/inference.py
| Model | Checkpoint |
|---|---|
| LiDAR | link |
| LiDAR+cam | link |
| LiDAR+radar | link |
| LiDAR+radar+cam | link |
If you use this code or aiMotive Multimodal Dataset in your research, please cite our work by using the following BibTeX entry:
@article{matuszka2022aimotivedataset,
title = {aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception},
author = {Matuszka, Tamás and Barton, Iván and Butykai, Ádám and Hajas, Péter and Kiss, Dávid and Kovács, Domonkos and Kunsági-Máté, Sándor and Lengyel, Péter and Németh, Gábor and Pető, Levente and Ribli, Dezső and Szeghy, Dávid and Vajna, Szabolcs and Varga, Bálint},
doi = {10.48550/ARXIV.2211.09445},
url = {https://arxiv.org/abs/2211.09445},
publisher = {arXiv},
year = {2022},
}
@inproceedings{matuszka2023aimotive,
title={aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception},
author={Tamas Matuszka},
booktitle={International Conference on Learning Representations 2023 Workshop on Scene Representations for Autonomous Driving},
year={2023},
url={https://openreview.net/forum?id=LW3bRLlY-SA}
}