Dynamic Vision Sensors (DVS), offer a unique advantage in control applications, due to their high temporal resolution, and asynchronous event-based data. Still, their adoption in machine learning algorithms remains limited. To address this gap, and promote the development of models that leverage the specific characteristics of DVS data, we introduce the Multi-Modal Dynamic-Vision-Sensor Line Following dataset (MMDVS-LF). This comprehensive dataset, is the first to integrate multiple sensor modalities, including DVS recordings, RGB video, odometry, and Inertial Measurement Unit (IMU) data, from a small-scale standardized vehicle. Additionally, the dataset includes eye-tracking and demographic data of drivers performing a Line Following task on a track. With its diverse range of data, MMDVS-LF opens new opportunities for developing deep learning algorithms, and conducting data science projects across various domains, supporting innovation in autonomous systems and control applications.
We provide two flavors of datasets:
- Noisy: Includes frames with limited amount of noise
- No Noise: All frames and observations with noise are removed
Due to constraints in storage capacity, not all datasets are available yet. We will upload the remaining files, as soon as possible.
The sizes in the tables refer to the uncompressed size of the downloadable archives.
| Noisy | 128x256 | 256x512 |
|---|---|---|
| 60Hz (with RGB) | 136,484 Frames | |
| 4.92 GB | 10.98 GB | |
| 100Hz (DVS only) | 227,375 Frames | |
| 1.56 GB | 4.98 GB | |
| 120Hz (DVS only) | 272,838 | |
| 1.73 GB | 5.42 GB |
Checksums:
7da517b07d6700a4cf033e5b44490243 dvs_only_256p_100hz_noisy.tar.bz2
a0f4331dbb2a9eaabc2c40514f17d4ea dvs_only_256p_120hz_noisy.tar.bz2
7f5b91a090fd25059774825a52ebea47 dvs_only_512p_100hz_noisy.tar.bz2
3effbba0af49999c4b672ce36910a120 dvs_only_512p_120hz_noisy.tar.bz2| No Noise | 128x256 | 256x512 |
|---|---|---|
| 60Hz (with RGB) | 96,161 Frames | |
| 2.926 GB | 7.462 GB | |
| 100Hz (DVS only) | 160,127 Frames | |
| 1.08 GB | 3.45 GB | |
| 120Hz (DVS only) | 192,127 Frames | |
| 1.19 GB | 3.75 GB |
Checksums:
cc51c86dae91e8aa818ad1f42bf6092b dvs_only_256p_100hz.tar.bz2
3bac60150f799a58da2627d54bdeb5e5 dvs_only_256p_120hz.tar.bz2
01cdc95effbf52b88c4c760a9a52f8d9 dvs_only_512p_100hz.tar.bz2
- CNN
- CNN + Simple RNN
- CNN + MGU
- CNN + GRU
- CNN + LSTM
- CNN + LTC
@misc{resch2024mmdvslfmultimodaldynamicvisionsensorline,
title={MMDVS-LF: A Multi-Modal Dynamic-Vision-Sensor Line Following Dataset},
author={Felix Resch and Mónika Farsang and Radu Grosu},
year={2024},
eprint={2409.18038},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2409.18038},
}