This is the code for the paper E-RAFT: Dense Optical Flow from Event Cameras by Mathias Gehrig, Mario Millhäusler, Daniel Gehrig and Davide Scaramuzza.
We also introduce DSEC-Flow (download here), the optical flow extension of the DSEC dataset. We are also hosting an automatic evaluation server and a public benchmark!
Visit our project webpage or download the paper directly here for more details. If you use any of this code, please cite the following publication:
@InProceedings{Gehrig3dv2021,
author = {Mathias Gehrig and Mario Millh\"ausler and Daniel Gehrig and Davide Scaramuzza},
title = {E-RAFT: Dense Optical Flow from Event Cameras},
booktitle = {International Conference on 3D Vision (3DV)},
year = {2021}
}Download the network checkpoints and place them in the folder checkpoints/:
Checkpoint trained on MVSEC 20 Hz
Checkpoint trained on MVSEC 45 Hz
Please install conda. Then, create new conda environment with python3.7 and all dependencies by running
conda env create --file environment.yml
The DSEC dataset for optical flow can be downloaded here.
We prepared a script download_dsec_test.py for your convenience.
It downloads the dataset directly into the OUTPUT_DIRECTORY with the expected directory structure.
download_dsec_test.py OUTPUT_DIRECTORYTo use the MVSEC dataset for our approach, it needs to be pre-processed into the right format. For your convenience, we provide the pre-processed dataset here:
MVSEC Outdoor Day 1 for 20 Hz evaluation
MVSEC Outdoor Day 1 for 45 Hz evaluation
For the evaluation of our method with warm-starting, execute the following command:
python3 main.py --path <path_to_dataset>
For the evaluation of our method without warm-starting, execute the following command:
python3 main.py --path <path_to_dataset> --type standard
For the evaluation of our method with warm-starting, trained on 20Hz MVSEC data, execute the following command:
python3 main.py --path <path_to_dataset> --dataset mvsec --frequency 20
For the evaluation of our method with warm-starting, trained on 45Hz MVSEC data, execute the following command:
python3 main.py --path <path_to_dataset> --dataset mvsec --frequency 45
--path : Path where you stored the dataset
--dataset : Which dataset to use: ([dsec]/mvsec)
--type : Evaluation type ([warm_start]/standard)
--frequency : Evaluation frequency of MVSEC dataset ([20]/45) Hz
--visualize : Provide this argument s.t. DSEC results are visualized. MVSEC experiments are always visualized.
--num_workers : How many sub-processes to use for data loading (default=0)