/VAD

[ICCV 2023] VAD: Vectorized Scene Representation for Efficient Autonomous Driving

Primary LanguagePythonApache License 2.0Apache-2.0

VAD v1 & v2

project page

vis_vad.mp4
vis_vad_carla.mp4

VAD: Vectorized Scene Representation for Efficient Autonomous Driving

Bo Jiang1*, Shaoyu Chen1*, Qing Xu2, Bencheng Liao1, Jiajie Chen2, Helong Zhou2, Qian Zhang2, Wenyu Liu1, Chang Huang2, Xinggang Wang1,†

1 Huazhong University of Science and Technology, 2 Horizon Robotics

*: equal contribution, †: corresponding author.

arXiv Paper, ICCV 2023

News

  • 20 Sep, 2024: Core code of VADv2 (config and model) is available in the VADv2 folder. Easy to integrade it into the VADv1 framework for training and inference.
  • 17 June, 2024: CARLA implementation of VADv1 is available on Bench2Drive.
  • 20 Feb, 2024: VADv2 is available on arXiv paper project page.
  • 1 Aug, 2023: Code & models are released!
  • 14 July, 2023: VAD is accepted by ICCV 2023🎉! Code and models will be open source soon!
  • 21 Mar, 2023: We release the VAD paper on arXiv. Code/Models are coming soon. Please stay tuned! ☕️

Introduction

VAD is a vectorized paradigm for end-to-end autonomous driving.

  • We propose VAD, an end-to-end unified vectorized paradigm for autonomous driving. VAD models the driving scene as a fully vectorized representation, getting rid of computationally intensive dense rasterized representation and hand-designed post-processing steps.
  • VAD implicitly and explicitly utilizes the vectorized scene information to improve planning safety, via query interaction and vectorized planning constraints.
  • VAD achieves SOTA end-to-end planning performance, outperforming previous methods by a large margin. Not only that, because of the vectorized scene representation and our concise model design, VAD greatly improves the inference speed, which is critical for the real-world deployment of an autonomous driving system.

Models

Method Backbone avg. L2 avg. Col. FPS Config Download
VAD-Tiny R50 0.78 0.38 16.8 config model
VAD-Base R50 0.72 0.22 4.5 config model

Results

  • Open-loop planning results on nuScenes. See the paper for more details.
Method L2 (m) 1s L2 (m) 2s L2 (m) 3s Col. (%) 1s Col. (%) 2s Col. (%) 3s FPS
ST-P3 1.33 2.11 2.90 0.23 0.62 1.27 1.6
UniAD 0.48 0.96 1.65 0.05 0.17 0.71 1.8
VAD-Tiny 0.46 0.76 1.12 0.21 0.35 0.58 16.8
VAD-Base 0.41 0.70 1.05 0.07 0.17 0.41 4.5
  • Closed-loop simulation results on CARLA.
Method Town05 Short DS Town05 Short RC Town05 Long DS Town05 Long RC
CILRS 7.47 13.40 3.68 7.19
LBC 30.97 55.01 7.05 32.09
Transfuser* 54.52 78.41 33.15 56.36
ST-P3 55.14 86.74 11.45 83.15
VAD-Base 64.29 87.26 30.31 75.20

*: LiDAR-based method.

Getting Started

Catalog

  • Code & Checkpoints Release
  • Initialization

Contact

If you have any questions or suggestions about this repo, please feel free to contact us (bjiang@hust.edu.cn, outsidercsy@gmail.com).

Citation

If you find VAD useful in your research or applications, please consider giving us a star 🌟 and citing it by the following BibTeX entry.

@article{jiang2023vad,
  title={VAD: Vectorized Scene Representation for Efficient Autonomous Driving},
  author={Jiang, Bo and Chen, Shaoyu and Xu, Qing and Liao, Bencheng and Chen, Jiajie and Zhou, Helong and Zhang, Qian and Liu, Wenyu and Huang, Chang and Wang, Xinggang},
  journal={ICCV},
  year={2023}
}

@article{chen2024vadv2,
  title={Vadv2: End-to-end vectorized autonomous driving via probabilistic planning},
  author={Chen, Shaoyu and Jiang, Bo and Gao, Hao and Liao, Bencheng and Xu, Qing and Zhang, Qian and Huang, Chang and Liu, Wenyu and Wang, Xinggang},
  journal={arXiv preprint arXiv:2402.13243},
  year={2024}
}

License

All code in this repository is under the Apache License 2.0.

Acknowledgement

VAD is based on the following projects: mmdet3d, detr3d, BEVFormer and MapTR. Many thanks for their excellent contributions to the community.