README

neuromorphic_au_drone_racing

This repository contains the public datasets and code for event camera perception for drone racing. The dataset is comprised of two parts: N-AU-DR-sim containing simulated data and N-AU-DR-real containing real event data.

Paper: link

To cite this paper:

  @INPROCEEDINGS{kristoffer2022ecc,
  author={Andersen, Kristoffer Fogh and Pham, Huy Xuan and Ugurlu, Halil Ibrahim and Kayacan, Erdal},
  booktitle={2022 European Control Conference (ECC)}, 
  title={Event-based Navigation for Autonomous Drone Racing with Sparse Gated Recurrent Network}, 
  year={2022}}

Download the dataset

The entire dataset can be downloaded from: N-AU-DR (10GB compressed and 33GB uncompressed).
Each dataset is split into a training set and a validation set.

Dataset description

The overall structure of the dataset is aligned with how the Gen1 Automotive dataset is structured, and all tools compatible with this dataset can also be used for our dataset.
The dataset is organized into .dat files each containing 60 seconds of event data, either simulated or real. Each dat file is a binary file in which events are encoded using 4 bytes (unsigned int32) for the timestamps and 4 bytes (unsigned int32) for the data
The data is composed of 14 bits for the x position, 14 bits for the y position and 1 bit for the polarity (encoded as -1/1).
For N-AU-DR-sim, the .dat filenames identify the date, timestamp and the simulated contrast threshold used. For example: 29-04-2021_15-21-46_1619702295_0.3_td.dat.
For N-AU-DR-real train split, the .dat filenames identify the date, timestamp, number of drone rotations and drone velocity. For example: 18-06-2021_10-57-20_1624006640_6rot_075ms_td.dat
For N-AU-DR-real validation split, the .dat filenames identify the date, timestamp, environment lighting and whether gates have the same placement as in the training data. For example: 10-07-2021_13-58-01_1625918281_50light_gatemoved_td.dat
For each .dat file there is a corresponding .npy file containing the annotated bounding boxes of gates. Each bounding box consist of x abscissa of the top left corner in pixels, y ordinate of the top left corner in pixels, w width of the boxes in pixel, h height of the boxes in pixel, ts timestamp of the box in the sequence in microseconds, class_id 0 for gates.

Dataset usage

For visualizing and manipulating the dataset we recommend using the prophesee-automotive-dataset-toolbox. Clone this repository and it then provides a variety of tools for visualizing the data with annotations.

Event-based gate detector with ROS

Requirements

Tested on ROS Melodic + Ubuntu 18.04
Event camera running ros driver rpg_dvs_ros: https://github.com/uzh-rpg/rpg_dvs_ros

Trained models

We provided 3 trained models: our sparse_RNN and two baselines dense_VGG and sparse_VGG under the following link
Download the respective model and put them into folder "trained_models" (currently empty)

Installation

Executing develop.sh will install all dependencies and external libraries (including Facebook's SparseConvNet):

cd ros/
./develop.sh

Usage

Launch package "event_preprocessor" for creating event histogram:

roslaunch event_preprocessor event_preprocessor.launch

Then, launch detector:

roslaunch event_detector_rt detector.launch

open-airlab/neuromorphic_au_drone_racing