Bench2DriveZoo (with Think2Drive as Teacher Model)

Introduction

This repo contains the training, open-loop evaluation, and closed-loop evaluation code for BEVFormer, UniAD , VAD in Bench2Drive. All models are student models of the world model RL teacher - Think2Drive.
We merge multiple dependencies of UniAD and VAD including mmcv, mmseg, mmdet, and mmdet3d (v0.17.1) into a single library. As a result, it could support latest pytorch and advanced frameworks like deepspeed for acceleration.
Use "git checkout tcp/admlp" to obtain their corresponding training and evaluation code.
To calculate smoothness and efficiency, remember to write self.metric_info related codes in your own team code agent. This two metrics require the state of the ego vehicle at each step in 20Hz. You may comment the lines about saving sensor data to save disk space.

Citation

Please consider citing our papers if the project helps your research with the following BibTex:

@article{jia2024bench,
  title={Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving},
  author={Xiaosong Jia and Zhenjie Yang and Qifeng Li and Zhiyuan Zhang and Junchi Yan},
  journal={arXiv preprint arXiv:2406.03877},
  year={2024}
}

@inproceedings{li2024think,
  title={Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)},
  author={Qifeng Li and Xiaosong Jia and Shaobo Wang and Junchi Yan},
  booktitle={ECCV},
  year={2024}
}

Getting Started

Results and Pre-trained Models

UniAD and VAD

As stated in the news at 2024/08/27, there are several fixed bugs and changed protocols. Thus, the old version of closed-loop performance is deprecated.

Method	L2 (m) 2s	Driving Score	Success Rate(%)	Config	Download	Eval Json
UniAD-Tiny	0.80	40.73 (deprecated 32.00)	13.18 (deprecated 9.54)	config	Hugging Face/Baidu Cloud	New Version
UniAD-Base	0.73	45.81 (deprecated 37.72)	16.36 (deprecated 9.54)	config	Hugging Face/Baidu Cloud	New Version
VAD	0.91	42.35 (deprecated 39.42)	15.00 (deprecated 10.00)	config	Hugging Face/Baidu Cloud	New Version

BEVFormer

Method	mAP	NDS	Config	Download
BEVFormer-Tiny	0.37	0.43	config	Hugging Face/Baidu Cloud
BEVFormer-Base	0.63	0.67	config	Hugging Face/Baidu Cloud

Failure Cases Analysis

We provide some visualization videos and qualitatively analysis for TCP-traj, UniAD-Base, VAD-Base at here. You may refer to https://github.com/Thinklab-SJTU/Bench2DriveZoo/blob/uniad/vad/team_code/vad_b2d_agent_visualize.py to write your own visualization code.

Thinklab-SJTU/Bench2DriveZoo