Goal Driven Multi-Modal Motion Prediction

Introduction

In this work, we introduce a novel framework known as the Goal-Driven Motion Prediction Network (GoDNet). GoDNet is engineered to excel in autonomous driving motion prediction tasks by learning from short-term historical observations and vectorized map information. To harness both spatial and temporal dimensions, the framework employs 1D-CNN on historical trajectories, as well as graph convolution on a sparse lane graph. To incorporate interaction-aware information, we introduce specialized cross-attention modules that facilitate nuanced interactions between the map and the actors, as well as among the actors themselves. Additionally, we have designed a goal-driven unit to enhance the model's capacity for long-term predictions across varying scales. Our proposed approach sets a new benchmark, achieving state-of-the-art performance in terms of minimum Final Displacement Error (minFDE) and minimum Average Displacement Error (minADE) on the Waymo Open Dataset.

Architecture

Get started

Recommended: Ubuntu 20.04 or higher.

1. Installation

A step-by-step installation guide.

Create a conda virtual environment and activate it.

conda create -n waymo python=3.8 -y
conda activate waymo

Install PyTorch following the official instructions.

conda install pytorch==1.5.1 torchvision cudatoolkit=10.2 -c pytorch

Install TensorFlow following the official instructions.

pip install tensorflow==2.4

Install waymo open dataset dependencies according to the reference.

pip install waymo-open-dataset-tf-2-11-0==1.6.0

Install Google Protocol Buffers.

pip install protobuf

Clone the GoDNet repository.

git clone https://github.com/LiamTheronC/GoDNet.git

2. Prepare dataset

The motion dataset is provided as sharded TFRecord format files containing protocol buffer data. The data are split into training, test, and validation sets with a split of 70% training, 15% testing and 15% validation data.

cd GoDNet/working
mkdir dataset && cd dataset
mkdir train val test

Download the full dataset from Google Cloud to the directories respectively following the official instructions.
Make sure the folder structure is:

GoDNet
├── working/
    ├── dataset/
        ├── train/
        ├── val/
        └──test/
    ├── dataLoader/
    ├──model/
    ├── preprocess/
        ├── preprocess_exe.py
        |...
    ├── train.py
    |...

Preprocess the dataset following Waymo-Motion-Preprocess.

python preprocess_exe.py train --downsample-factor=10 --type-feats=vp

Train and evaluation

Prerequisites

Please ensure you have prepared the environment and the Waymo Open Dataset dataset.

Train

Train GoDnet with 4 GPUs.

./dist_train.sh vp 4

Visualization

see plot.py.

Results

Result in metrics

Methods	minFDE(k=6)↓	minADE(k=6) ↓	MR↓
LSTM	2.36	1.00	0.31
laneGCN	2.27	1.04	0.27
GoDNet	2.02	0.91	0.21

Visualization of the results

License

The work is released under the MIT license.

LiamTheronC/GoDNet